Target detection method and device based on spatial semantic similar knowledge distillation and medium

Abstract

This invention belongs to the field of target detection technology, and relates to a target detection method, apparatus, and medium based on spatial semantic similarity knowledge distillation. Image samples from the training set are input into a teacher model and a student model respectively for feature extraction, outputting teacher feature maps and student feature maps. The semantic similarity matrix and spatial distance similarity matrix of each feature map are obtained. The semantic similarity matrix and spatial distance similarity matrix of each feature map are fused to obtain a global affinity matrix for each feature map. A local neighborhood mask is generated from the global affinity matrix of each feature map to obtain a spatial semantic similarity matrix for each feature map. Target detection and recognition are performed based on the teacher feature map and its spatial semantic similarity matrix, and the student feature map and its spatial semantic similarity matrix. A target detection distillation loss is constructed, and the student model is trained. The trained student model is then used for target detection, improving the accuracy of target detection.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a target detection method, apparatus, and computer-readable storage medium based on spatial semantic similarity knowledge distillation. Background Technology

[0002] Target detection is a technology that automatically identifies and locates specific targets (such as aircraft, vehicles, buildings, ships, etc.) in images. It combines image processing, pattern recognition, and deep learning methods to achieve target category judgment and spatial location labeling. It is widely used in fields such as urban planning, environmental monitoring, and disaster assessment, and is a key link in information extraction and application.

[0003] With the continuous development of deep learning technology, object detection models can achieve high detection accuracy due to their powerful expressive capabilities. However, as detection accuracy improves, models often suffer from large parameter counts and high computational complexity, making them unsuitable for deployment in resource-constrained scenarios (such as drones and edge devices). Knowledge distillation is an effective model compression and knowledge transfer method. Its core idea is to transfer the knowledge learned by a well-trained, complex teacher model to a simpler, fewer-parameter student model. This allows the student model to maintain performance close to that of the teacher model while significantly reducing computational costs and storage requirements, facilitating deployment on resource-constrained devices.

[0004] Existing knowledge distillation-based object detection methods utilize teacher and student models to extract image features and perform object detection. A distillation loss function is constructed based on the detection results of both models, and through iterative training, the student model mimics the teacher model's output, achieving detection results with similar accuracy. However, object detection essentially involves joint modeling of spatial location and semantic category. Existing methods ignore the impact of spatial and semantic characteristics of feature maps on detection accuracy, directly using the extracted features and then constructing a distillation loss function. This prevents the teacher model from learning the relationship between the spatial continuity and semantic consistency of the target and the detection results, and also prevents the transfer of this knowledge to the student model. The student model can only mimic the teacher model's output. When there are changes in lighting, viewpoint shifts, or background interference in the detection scene, the extracted features may change due to environmental factors. Because the student model has not learned the spatial-semantic coupling relationship with the target, it may exhibit problems such as target fragmentation and misclassification of background as targets, resulting in low object detection accuracy. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem of low detection accuracy in the existing target detection methods based on knowledge distillation.

[0006] To address the aforementioned technical problems, this invention provides a target detection method based on spatial semantic similarity knowledge distillation, comprising:

[0007] Image samples from the training set are input into the teacher model and student model respectively for feature extraction, and the teacher feature map and student feature map are output.

[0008] Each feature map is flattened into a positional feature vector and normalized. The similarity between each feature value in the normalized positional feature vector is calculated to obtain the semantic similarity matrix of each feature map.

[0009] Each feature map is mapped to a spatial coordinate vector, and the similarity between each feature value in the spatial coordinate vector is calculated to obtain the spatial distance similarity matrix of each feature map.

[0010] The semantic similarity matrix and spatial distance similarity matrix of each feature map are fused to obtain the global affinity matrix of each feature map;

[0011] For each feature map, a local neighborhood mask is generated from the global affinity matrix. The neighborhood position association features of each pixel in the feature map are obtained. The affinity of all the neighborhood position association features is aggregated to obtain the spatial semantic similarity matrix of each feature map.

[0012] Target detection and recognition are performed based on teacher feature maps and their spatial semantic similarity matrices and student feature maps and their spatial semantic similarity matrices. A target detection distillation loss is constructed, and the student model is trained. The trained student model is then used for target detection.

[0013] Preferably, image samples from the training set are input into the teacher model and student model respectively for feature extraction, and the output teacher feature maps and student feature maps are generated, including:

[0014] Image samples are input into the multi-scale feature extraction networks of the teacher model and the student model, respectively, and multi-level teacher feature maps and multi-level student feature maps are output.

[0015] Preferably, the location-level feature vector is represented as:

[0016] ,

[0017] ,

[0018] in, This represents the positional feature vector after flattening the teacher feature map of level l; This indicates operations that flatten out spatial dimensions; This represents the teacher feature map at level l. This represents the positional feature vector after flattening the student feature map of level l; This represents the student feature map at level l.

[0019] The normalized positional feature vector is represented as:

[0020] ,

[0021] ,

[0022] in, Indicates to Normalized position-level feature vectors; This indicates a normalization operation; Indicates to Normalized position-level feature vectors;

[0023] The semantic similarity matrix of the feature maps is represented as follows:

[0024] ,

[0025] ,

[0026] in, The semantic similarity matrix represents the teacher feature map at level l; This indicates a batch matrix multiplication operation; Indicates transpose; This represents the semantic similarity matrix of the student feature maps at level l.

[0027] Preferably, the spatial coordinate vector is represented as:

[0028] ,

[0029] ,

[0030] in, The spatial coordinate vector representing the teacher feature map of level l; This represents the operation of generating and flattening grid coordinates; This represents the coordinate sequence along the width direction of the teacher feature map at level l; This represents the coordinate sequence along the height direction of the teacher feature map at level l; This represents the spatial coordinate vector of the student feature map at level l; This represents the coordinate sequence along the width direction of the student feature map at level l; This represents the coordinate sequence along the height direction of the student feature map at level l;

[0031] The spatial distance similarity matrix of the feature maps is represented as follows:

[0032] ,

[0033] ,

[0034] ,

[0035] ,

[0036] in, This indicates the operation for calculating Euclidean distance; Represents the spatial distance similarity matrix of the teacher feature maps at level l; Represents the spatial Gaussian kernel parameters; This represents the spatial distance similarity matrix of the student feature maps at level l. This represents the Euclidean distance between each pixel and other pixels in the teacher feature map of level l. This represents the Euclidean distance between each pixel and other pixels in the student feature map of level l;

[0037] The global affinity matrix of the feature map is represented as:

[0038] ,

[0039] ,

[0040] in, Represents the global affinity matrix of the teacher feature map at level l; The semantic similarity matrix represents the teacher feature map at level l; This represents an element-wise multiplication operation; The global affinity matrix represents the student feature map at level l; This represents the semantic similarity matrix of the student feature maps at level l.

[0041] Preferably, the spatial semantic similarity matrix of the feature map is represented as:

[0042] ,

[0043] ,

[0044] ,

[0045] ,

[0046] in, The spatial semantic similarity matrix represents the teacher feature map at level l; Represents the local neighborhood radius; Indicates an indicator function, when hour, ,when hour, ; The spatial semantic similarity matrix represents the student feature map at level l; The spatial neighborhood mask represents the feature map of the teacher at level l; This represents the spatial neighborhood mask of the student feature map at level l. This indicates element-wise summation.

[0047] Preferably, target detection and recognition are performed based on the teacher feature map and its spatial semantic similarity matrix and the student feature map and its spatial semantic similarity matrix, and a target detection distillation loss is constructed, including:

[0048] Input the teacher feature map and student feature map of each level into the detection head, and output the first target detection result of the teacher model and the second target detection result of the student model at that level;

[0049] Based on the first target detection results of the teacher model and the second target detection results of the student model at each level, construct the detection loss for that level; construct the target detection loss based on the detection losses of all levels.

[0050] Based on the spatial semantic similarity matrix of the teacher feature map and the spatial semantic similarity matrix of the student feature map at each level, the similarity structure distillation loss of that level is constructed; based on the similarity structure distillation loss of all levels, the target similarity structure distillation loss is constructed.

[0051] Based on the first target detection results of the teacher model, the second target detection results of the student model, and the spatial semantic similarity matrix of the teacher feature map at each level, a perceptual classification distillation loss for that level is constructed; and a target perceptual classification distillation loss is constructed based on the perceptual classification distillation losses of all levels.

[0052] The target detection distillation loss is obtained by weighted summing of the target detection loss, the target similarity structure distillation loss, and the target perception classification distillation loss.

[0053] Preferably, based on the first object detection result of the teacher model, the second object detection result of the student model, and the spatial semantic similarity matrix of the teacher feature map at each level, a perceptual classification distillation loss for that level is constructed, including:

[0054] Based on the first target detection result of the teacher model and the second target detection result of the student model at each level, calculate the target detection KL divergence loss of that level.

[0055] The spatial semantic similarity matrix of the teacher feature map at each level is used as the weight of the KL divergence loss for object detection at that level, thus obtaining the perceptual classification distillation loss for that level.

[0056] Preferably, the target similarity structure distillation loss The calculation formula is:

[0057] ,

[0058] in, Indicates the number of layers in a multi-scale feature extraction network; The spatial semantic similarity matrix represents the teacher feature map at level l; The spatial semantic similarity matrix represents the student feature map at level l; express Norm;

[0059] Target perception classification distillation loss The calculation formula is:

[0060] ,

[0061] in, This represents the target detection results of the teacher model; This represents the target detection results of the student model; , This represents the spatial dimensions of the feature map at level l; Indicates the number of target categories; This represents the first object detection result of the teacher model at level l; This represents the second object detection result of the student model at level l.

[0062] The present invention also provides a target detection device based on spatial semantic similarity knowledge distillation, comprising:

[0063] The feature extraction module is used to input image samples from the training set into the teacher model and student model respectively for feature extraction, and output teacher feature maps and student feature maps.

[0064] The semantic similarity acquisition module is used to flatten each feature map into a positional feature vector and normalize it, calculate the similarity between each feature value in the normalized positional feature vector, and obtain the semantic similarity matrix of each feature map.

[0065] The spatial similarity acquisition module is used to map each feature map into a spatial coordinate vector, calculate the similarity between each feature value in the spatial coordinate vector, and obtain the spatial distance similarity matrix of each feature map.

[0066] The similarity fusion module is used to fuse the semantic similarity matrix and spatial distance similarity matrix of each feature map to obtain the global affinity matrix of each feature map;

[0067] The spatial semantic similarity acquisition module is used to generate a local neighborhood mask from the global affinity matrix of each feature map, obtain the neighborhood position association features of each pixel in the feature map, aggregate the affinity of the neighborhood position association features of all pixels, and obtain the spatial semantic similarity matrix of each feature map.

[0068] The model training and object detection module is used to perform object detection and recognition based on teacher feature maps and their spatial semantic similarity matrices and student feature maps and their spatial semantic similarity matrices, and to construct object detection distillation loss. The student model is then trained and the trained student model is used for object detection.

[0069] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the target detection method based on spatial semantic similarity knowledge distillation described above.

[0070] The target detection method based on spatial semantic similarity knowledge distillation provided in this application has the following beneficial effects:

[0071] 1. After obtaining the teacher and student feature maps, the semantic similarity matrix of the feature maps is obtained by flattening them into positional feature vectors. This captures the semantic association strength between different spatial locations in the feature maps. For example, the semantic association strength between the car front and wheel positions in an image sample is strong, indicating they belong to the same target, thus modeling the semantic consistency of target features. Then, the feature maps are mapped to spatial coordinate vectors to obtain the spatial distance similarity matrix, capturing the spatial dependencies between different locations in the feature maps. For example, pixels of the same target are spatially continuous, thus modeling the spatial continuity of target features. The semantic similarity matrix and the spatial distance similarity matrix are then fused to obtain... The global affinity matrix is ​​used to form coupled spatial and semantic features. Then, local neighborhood masks and affinity aggregation are performed on the global affinity features to obtain a spatial semantic similarity matrix. This allows for the focus on fine-grained feature information of key regions while ensuring the semantic consistency and spatial continuity of target features. Finally, target detection is performed based on the feature map and its spatial semantic similarity matrix, and a target detection distillation loss is constructed. This enables the teacher model and student model to learn the perceptual logic from space and semantics to target detection results. Even in complex scenarios such as changes in lighting or viewpoint shifts, the model can still output accurate target detection results based on the spatial and semantic characteristics of target features.

[0072] 2. A multi-scale feature extraction network is used to extract teacher and student feature maps at multiple levels. The spatial semantic similarity matrix of the feature map at each level is obtained, and the detection results of the teacher model and the student model at each level are obtained. Since the feature information extracted from the feature maps at different levels is different, the corresponding detection results may also be different. This application directly constructs the detection distillation loss of each level based on the detection results of the teacher model, the spatial semantic similarity matrix, the detection results of the student model, and the spatial semantic similarity matrix at each level. The hierarchical loss constraint realizes the efficient transfer of target-related knowledge in the teacher model. Finally, the target detection distillation loss is constructed based on the detection distillation loss of all levels, realizing the hierarchical knowledge distillation and transfer of the teacher model and the student model.

[0073] 3. When constructing the detection distillation loss for each level, in addition to building a simple detection loss based on the detection results of the teacher model and the student model, a similarity structure distillation loss is also constructed based on the spatial semantic similarity matrix of the teacher feature map and the semantic similarity matrix of the student feature map. This allows the student model to not only imitate the target detection results of the teacher model, but also to imitate the teacher model's ability to learn the spatial continuity and semantic consistency of features. In addition, the spatial semantic similarity matrix of the teacher feature map is used as a dynamic weight to calculate the KL divergence loss of the detection results of the teacher model and the student model. This allows the student model to focus on learning the semantic information of key spatial locations during the process of learning the detection experience of the teacher model, thereby accurately capturing the semantic features and spatial distribution patterns of the target. This makes up for the problem of insufficient feature expression ability caused by the limitation of network depth and parameter size in lightweight models. Attached Figure Description

[0074] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0075] Figure 1 Flowchart of the target detection method based on spatial semantic similarity knowledge distillation provided in this application;

[0076] Figure 2 A schematic diagram illustrating the principle of target detection based on spatial semantic similarity knowledge distillation provided in this application;

[0077] Figure 3 The target detection heatmap based on the spatial semantic similarity matrix provided in this application; wherein, Figure 3 (a) in the image is the target detection heatmap for the first scenario. Figure 3 (b) in the image is the target detection heatmap for the second scenario. Figure 3 (c) in the image is the target detection heatmap for the third scenario. Figure 3 (d) in the image represents the target detection heatmap for the fourth scenario.

[0078] Figure 4 This is a comparison chart of the detection results of this application and the target detection baseline model in the prior art; wherein, Figure 4 (a) in the figure is a schematic diagram of the target detection results of this application in the first scenario. Figure 4 (b) in the diagram is a schematic diagram of the target detection results of the target detection baseline model in the first scenario. Figure 4 (c) in the diagram is a schematic diagram of the target detection results of this application in the second scenario. Figure 4 (d) in the diagram is a schematic diagram of the target detection results of the target detection baseline model in the second scenario. Detailed Implementation

[0079] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0080] Please see Figure 1 and Figure 2 , Figure 1 The diagram shown is a flowchart of the target detection method based on spatial semantic similarity knowledge distillation provided in this application. Figure 2 The diagram shown illustrates the target detection principle based on spatial semantic similarity knowledge distillation provided in this application. The method specifically includes steps S10-S60:

[0081] S10: Input the image samples in the training set into the teacher model and student model respectively for feature extraction, and output the teacher feature map and student feature map.

[0082] S20: Flatten each feature map into a positional feature vector and normalize it. Calculate the similarity between each feature value in the normalized positional feature vector to obtain the semantic similarity matrix of each feature map.

[0083] S30: Map each feature map to a spatial coordinate vector, calculate the similarity between each feature value in the spatial coordinate vector, and obtain the spatial distance similarity matrix of each feature map.

[0084] S40: Fuse the semantic similarity matrix and spatial distance similarity matrix of each feature map to obtain the global affinity matrix of each feature map.

[0085] S50: Generate a local neighborhood mask for the global affinity matrix of each feature map, obtain the neighborhood position association features of each pixel in the feature map, aggregate the affinity of the neighborhood position association features of all pixels, and obtain the spatial semantic similarity matrix of each feature map.

[0086] S60: Target detection and recognition are performed based on teacher feature maps and their spatial semantic similarity matrices and student feature maps and their spatial semantic similarity matrices. Target detection distillation loss is constructed, and the student model is trained. The trained student model is then used for target detection.

[0087] Furthermore, S10 includes: inputting image samples into the multi-scale feature extraction network of the teacher model and the multi-scale feature extraction network of the student model respectively, and outputting multi-level teacher feature maps and multi-level student feature maps.

[0088] For example, the teacher model is a ResNet-101-based FCOS (Fully Convolutional One-Stage Object Detection) model, and the student model is a ResNet-18-based FCOS model. In this embodiment, the training set is selected from the publicly available DOTA 1.0 dataset. After the image samples are processed by the teacher model, five levels of teacher feature maps are output. After processing the student model, five levels of student feature maps are output. , .

[0089] Specifically, the location-level feature vector is represented as:

[0090] ,

[0091] ,

[0092] in, This represents the positional feature vector after flattening the teacher feature map of level l; This indicates a flattening operation on the spatial dimensions, which flattens the original two-dimensional spatial structure of the feature map. This represents the teacher feature map at level l. This represents the positional feature vector after flattening the student feature map of level l; This represents the student feature map at level l.

[0093] Furthermore, the positional feature vectors are normalized along the channel dimension. The normalized positional feature vectors are represented as follows:

[0094] ,

[0095] ,

[0096] in, Indicates to Normalized position-level feature vectors; This indicates a normalization operation; Indicates to Normalized positional feature vectors.

[0097] Specifically, the semantic similarity matrix of the feature maps is represented as follows:

[0098] ,

[0099] ,

[0100] in, The semantic similarity matrix represents the teacher feature map at level l; This indicates a batch matrix multiplication operation; Indicates transpose; This represents the semantic similarity matrix of the student feature maps at level l.

[0101] Specifically, by calculating the matrix product of the normalized position-level feature vectors, the semantic similarity matrix of the feature map is obtained, which is used to measure the degree of semantic association between features at different spatial locations on the feature map.

[0102] Furthermore, the spatial coordinate vector is represented as:

[0103] ,

[0104] ,

[0105] in, The spatial coordinate vector representing the teacher feature map of level l; This represents the operation of generating and flattening grid coordinates; This represents the coordinate sequence along the width direction of the teacher feature map at level l; This represents the coordinate sequence along the height direction of the teacher feature map at level l; This represents the spatial coordinate vector of the student feature map at level l; This represents the coordinate sequence along the width direction of the student feature map at level l; This represents the coordinate sequence of the student feature map at level l along the height direction.

[0106] The spatial distance similarity matrix of the feature maps is represented as follows:

[0107] ,

[0108] ,

[0109] ,

[0110] ,

[0111] in, This indicates the operation for calculating Euclidean distance; Represents the spatial distance similarity matrix of the teacher feature maps at level l; Represents the spatial Gaussian kernel parameters; This represents the spatial distance similarity matrix of the student feature maps at level l. This represents the Euclidean distance between each pixel and other pixels in the teacher feature map of level l. This represents the Euclidean distance between each pixel in the student feature map of level l and other pixels.

[0112] Specifically, the spatial distance similarity matrix of a feature map can characterize the spatial proximity between different locations on the feature map.

[0113] Furthermore, the global affinity matrix of the feature map is represented as:

[0114] ,

[0115] ,

[0116] in, Represents the global affinity matrix of the teacher feature map at level l; The semantic similarity matrix represents the teacher feature map at level l; This represents an element-wise multiplication operation; The global affinity matrix represents the student feature map at level l; This represents the semantic similarity matrix of the student feature maps at level l.

[0117] Specifically, by fusing the semantic similarity matrix and the spatial distance similarity matrix of the feature map, the dual similarity of spatial distance and feature semantic distance at different locations on the feature map can be integrated.

[0118] Furthermore, the spatial semantic similarity matrix of the feature maps is represented as:

[0119] ,

[0120] ,

[0121] ,

[0122] ,

[0123] in, The spatial semantic similarity matrix represents the teacher feature map at level l; Represents the local neighborhood radius; Indicates an indicator function, when hour, ,when hour, ; The spatial semantic similarity matrix represents the student feature map at level l; The spatial neighborhood mask represents the feature map of the teacher at level l; This represents the spatial neighborhood mask of the student feature map at level l. This indicates element-wise summation.

[0124] Specifically, by generating a local neighborhood mask, only the neighborhood position association features of each pixel on the feature map within a specified local neighborhood radius are retained, and the neighborhood position association features of all pixels within a specified local neighborhood radius are aggregated. The resulting spatial semantic similarity matrix can quantify the local similarity strength of each spatial location.

[0125] Furthermore, in S60, target detection and recognition are performed based on the teacher feature map and its spatial semantic similarity matrix and the student feature map and its spatial semantic similarity matrix, and the target detection distillation loss is constructed, including S600~S604:

[0126] S600: Input the teacher feature map and student feature map of each level into the detection head, and output the first target detection result of the teacher model and the second target detection result of the student model of that level.

[0127] Specifically, the first target detection result of the level l teacher model The second object detection results of the student model They are represented as follows:

[0128] ,

[0129] ,

[0130] in, This indicates the detection head.

[0131] S601: Construct the detection loss for each level based on the first target detection result of the teacher model and the second target detection result of the student model; construct the target detection loss based on the detection losses of all levels.

[0132] Specifically, S601 includes steps 1-1 to 1-4:

[0133] Step 1-1: Calculate the detection loss of each level of teacher model based on the first target detection result and the label of the image sample.

[0134] Step 1-2: Calculate the detection loss of each level of student model based on the second target detection results and the labels of the image samples.

[0135] Steps 1-3: Weighted summation of the detection loss of the teacher model and the detection loss of the student model at each level to obtain the detection loss for each level.

[0136] Steps 1-4: Weighted summation of the detection losses at all levels to obtain the target detection loss.

[0137] It should be noted that since the object detection result includes the object prediction category, the object prediction location, and the object prediction angle, the detection loss of both the teacher model and the student model can be the loss between the object prediction category and the true object category in the label of the image sample, the loss between the object prediction location and the true object location in the label of the image sample, or the loss between the object prediction angle and the true object angle in the label of the image sample, or a weighted sum of two or three of the above three losses. Through the object detection loss, the local spatial features contained in the low-level features of the teacher model, the high-level semantic knowledge of the features at that level, and the prior knowledge of angle prediction can be transferred to the student model.

[0138] For example, FcoalLoss and RotatedIoULoss can be chosen as the target prediction category loss and the target prediction location loss, respectively.

[0139] S602: Based on the spatial semantic similarity matrix of the teacher feature map and the spatial semantic similarity matrix of the student feature map at each level, construct the similarity structure distillation loss for that level; construct the target similarity structure distillation loss based on the similarity structure distillation loss for all levels.

[0140] S603: Based on the first target detection result of the teacher model, the second target detection result of the student model, and the spatial semantic similarity matrix of the teacher feature map at each level, construct the perceptual classification distillation loss for that level; construct the target perceptual classification distillation loss based on the perceptual classification distillation loss of all levels.

[0141] S604: The target detection distillation loss is obtained by weighted summing of the target detection loss, the target similarity structure distillation loss, and the target perception classification distillation loss.

[0142] Furthermore, in S603, based on the first target detection result of the teacher model, the second target detection result of the student model, and the spatial semantic similarity matrix of the teacher feature map, the perceptual classification distillation loss for that level is constructed, including steps 2-1 to 2-2:

[0143] Step 2-1: Based on the first target detection result of the teacher model and the second target detection result of the student model at each level, calculate the target detection KL divergence loss for that level.

[0144] Step 2-2: Use the spatial semantic similarity matrix of the teacher feature map at each level as the weight of the object detection KL divergence loss at that level to obtain the perceptual classification distillation loss at that level.

[0145] It's important to note that the perceptual classification distillation loss uses the spatial semantic similarity matrix of the teacher feature map at each level as weights to calculate a weighted distillation loss for the target detection results of the teacher model and the teacher detection results of the student model. This dynamic weighting ensures that during training, the student model doesn't simply mimic the teacher model's output, but rather automatically focuses on the key semantically related regions of the teacher model's output, avoiding overfitting to irrelevant backgrounds and thus improving the student model's accuracy in target perception and classification. Furthermore, when calculating the perceptual classification distillation loss, the primary calculation focuses on the distillation loss between the predicted target categories in the target detection results of the teacher and student models.

[0146] Specifically, target similarity structure distillation loss The calculation formula is:

[0147] ,

[0148] in, Indicates the number of layers in a multi-scale feature extraction network; The spatial semantic similarity matrix represents the teacher feature map at level l; The spatial semantic similarity matrix represents the student feature map at level l; express Norm.

[0149] Target perception classification distillation loss The calculation formula is:

[0150] ,

[0151] in, This represents the target detection results of the teacher model; This represents the target detection results of the student model; , This represents the spatial dimensions of the feature map at level l; Indicates the number of target categories; This represents the first object detection result of the teacher model at level l; This represents the second object detection result of the student model at level l.

[0152] Target detection distillation loss Represented as:

[0153] ,

[0154] in, Indicates the target detection loss; Represents the structural distillation loss of target similarity The weights; Represents target perception classification distillation loss The weight.

[0155] It is worth noting that, through experimental analysis, this application found that when , That is, when the weights of the three losses are the same, the target detection accuracy of the trained student model reaches its highest level.

[0156] The embodiments of this application also verify the detection accuracy of the trained student model and the baseline model for different categories of targets when the target similarity structure distillation loss and the target perception classification distillation loss are used alone and in combination, as shown in Table 1:

[0157] Table 1

[0158]

[0159] Table 2 shows the comparison of target detection accuracy when neither the target similarity structure distillation loss nor the target perception classification distillation loss is applied (baseline model), when they are applied individually, and when they are applied together:

[0160] Table 2

[0161]

[0162] As can be seen from the data in Table 2, the average target detection accuracy of the student model obtained after training with both distillation losses is significantly improved.

[0163] like Figure 3 The image shown is a heatmap of target detection based on a spatial semantic similarity matrix provided in this application; wherein, Figure 3 (a) in the image is the target detection heatmap for the first scenario. Figure 3 (b) in the image is the target detection heatmap for the second scenario. Figure 3 (c) in the image is the target detection heatmap for the third scenario. Figure 3 (d) in the diagram represents the target detection heatmap for the fourth scenario. It can be seen that the student model trained using the spatial semantic similarity matrix proposed in this application is better able to focus on small target regions during target detection, avoiding the problem of missed detections due to the small size of the target.

[0164] like Figure 4 The figure shown is a comparison of the detection results of this application and the target detection baseline model in the prior art; wherein, Figure 4 (a) in the figure is a schematic diagram of the target detection results of this application in the first scenario. Figure 4 (b) in the diagram is a schematic diagram of the target detection results of the target detection baseline model in the first scenario. Figure 4(c) in the diagram is a schematic diagram of the target detection results of this application in the second scenario. Figure 4 (d) in the diagram illustrates the target detection results of the baseline model in the second scenario. It can be seen that the student model trained using the spatial semantic similarity knowledge distillation method provided in this application achieves significantly higher detection accuracy for small targets than the baseline model.

[0165] This application constructs a global affinity matrix by fusing the semantic similarity and spatial distance similarity of the feature maps of teachers and students at each level, and further generates a local neighborhood spatial semantic similarity matrix. Based on the feature maps of teachers and students at each level and their spatial semantic similarity matrices, target detection and recognition are performed, and a target detection distillation loss is constructed. This hierarchical loss constraint enables efficient transfer of target association knowledge from the teacher model. Simultaneously, a similarity-weighted classification distillation mechanism allows the student model to accurately capture the semantic features and spatial distribution patterns of targets while learning from the teacher model's detection experience, thus overcoming the insufficient feature representation capability of lightweight models due to limitations in network depth and parameter size. Furthermore, the student model obtained through this transfer significantly improves the prediction accuracy of targets, especially in the detection of small and dense targets, while maintaining high prediction efficiency. This adapts to the real-time and lightweight requirements of application scenarios, providing an effective solution for deploying image target detection technology on resource-constrained devices.

[0166] Based on the target detection method based on spatial semantic similarity knowledge distillation provided in the above embodiments, this application also provides a target detection device based on spatial semantic similarity knowledge distillation, the device specifically including:

[0167] The feature extraction module is used to input image samples from the training set into the teacher model and student model respectively for feature extraction, and output teacher feature maps and student feature maps.

[0168] The semantic similarity acquisition module is used to flatten each feature map into a positional feature vector and normalize it, calculate the similarity between each feature value in the normalized positional feature vector, and obtain the semantic similarity matrix of each feature map.

[0169] The spatial similarity acquisition module is used to map each feature map into a spatial coordinate vector, calculate the similarity between each feature value in the spatial coordinate vector, and obtain the spatial distance similarity matrix of each feature map.

[0170] The similarity fusion module is used to fuse the semantic similarity matrix and spatial distance similarity matrix of each feature map to obtain the global affinity matrix of each feature map.

[0171] The spatial semantic similarity acquisition module is used to generate a local neighborhood mask from the global affinity matrix of each feature map, obtain the neighborhood position association features of each pixel in the feature map, and perform affinity aggregation on the neighborhood position association features of all pixels to obtain the spatial semantic similarity matrix of each feature map.

[0172] The model training and object detection module is used to perform object detection and recognition based on teacher feature maps and their spatial semantic similarity matrices and student feature maps and their spatial semantic similarity matrices, and to construct object detection distillation loss. The student model is then trained and the trained student model is used for object detection.

[0173] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the target detection method based on spatial semantic similarity knowledge distillation described above.

[0174] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0176] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0177] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0178] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A target detection method based on spatial semantic similarity knowledge distillation, characterized in that, include: Image samples from the training set are input into the teacher model and student model respectively for feature extraction, and the teacher feature map and student feature map are output. Specifically, this includes: inputting image samples into the multi-scale feature extraction network of the teacher model and the multi-scale feature extraction network of the student model respectively, and outputting multi-level teacher feature maps and multi-level student feature maps; Each feature map is flattened into a positional feature vector and normalized. The similarity between each feature value in the normalized positional feature vector is calculated to obtain the semantic similarity matrix of each feature map. Each feature map is mapped to a spatial coordinate vector, and the similarity between each feature value in the spatial coordinate vector is calculated to obtain the spatial distance similarity matrix of each feature map. The semantic similarity matrix and spatial distance similarity matrix of each feature map are fused to obtain the global affinity matrix of each feature map; For each feature map, a local neighborhood mask is generated from the global affinity matrix. The neighborhood position association features of each pixel in the feature map are obtained. The affinity of all the neighborhood position association features is aggregated to obtain the spatial semantic similarity matrix of each feature map. Target detection and recognition are performed based on teacher feature maps and their spatial semantic similarity matrices, and student feature maps and their spatial semantic similarity matrices. A target detection distillation loss is then constructed, which specifically includes: Input the teacher feature map and student feature map of each level into the detection head, and output the first target detection result of the teacher model and the second target detection result of the student model at that level; Based on the first target detection results of the teacher model and the second target detection results of the student model at each level, construct the detection loss for that level; construct the target detection loss based on the detection losses of all levels. Based on the spatial semantic similarity matrix of the teacher feature map and the spatial semantic similarity matrix of the student feature map at each level, the similarity structure distillation loss of that level is constructed; based on the similarity structure distillation loss of all levels, the target similarity structure distillation loss is constructed. Based on the first object detection results of the teacher model, the second object detection results of the student model, and the spatial semantic similarity matrix of the teacher feature map at each level, the perceptual classification distillation loss for that level is constructed, which specifically includes: Based on the first target detection result of the teacher model and the second target detection result of the student model at each level, calculate the target detection KL divergence loss of that level. The spatial semantic similarity matrix of the teacher feature map at each level is used as the weight of the object detection KL divergence loss at that level to obtain the perceptual classification distillation loss at that level. Construct a target perception classification distillation loss based on perception classification distillation loss at all levels; The target detection distillation loss is obtained by weighted summing of the target detection loss, the target similarity structure distillation loss, and the target perception classification distillation loss; The student model is trained, and the trained student model is used for object detection.

2. The target detection method based on spatial semantic similarity knowledge distillation according to claim 1, characterized in that, The location-level feature vector is represented as: ， ， in, This represents the positional feature vector after flattening the teacher feature map of level l; This indicates operations that flatten out spatial dimensions; This represents the teacher feature map at level l. This represents the positional feature vector after flattening the student feature map of level l; This represents the student feature map at level l. The normalized positional feature vector is represented as: ， ， in, Indicates to Normalized position-level feature vectors; This indicates a normalization operation; Indicates to Normalized position-level feature vectors; The semantic similarity matrix of the feature maps is represented as follows: ， ， in, The semantic similarity matrix represents the teacher feature map at level l; This represents a batch matrix multiplication operation; Indicates transpose; This represents the semantic similarity matrix of the student feature maps at level l.

3. The target detection method based on spatial semantic similarity knowledge distillation according to claim 1, characterized in that, Spatial coordinate vectors are represented as: ， ， in, The spatial coordinate vector representing the teacher feature map of the l-th level; This represents the operation of generating and flattening grid coordinates; This represents the coordinate sequence along the width direction of the teacher feature map at level l; This represents the coordinate sequence along the height direction of the teacher feature map at level l; This represents the spatial coordinate vector of the student feature map at level l; This represents the coordinate sequence along the width direction of the student feature map at level l; This represents the coordinate sequence along the height direction of the student feature map at level l; The spatial distance similarity matrix of the feature maps is represented as follows: ， ， ， ， in, This indicates the operation for calculating Euclidean distance; Represents the spatial distance similarity matrix of the teacher feature maps at level l; Represents the spatial Gaussian kernel parameters; This represents the spatial distance similarity matrix of the student feature maps at level l. This represents the Euclidean distance between each pixel and other pixels in the teacher feature map of level l. This represents the Euclidean distance between each pixel and other pixels in the student feature map of level l; The global affinity matrix of the feature map is represented as: ， ， in, Represents the global affinity matrix of the teacher feature map at level l; The semantic similarity matrix represents the teacher feature map at level l; This represents an element-wise multiplication operation; The global affinity matrix represents the student feature map at level l; This represents the semantic similarity matrix of the student feature maps at level l.

4. The target detection method based on spatial semantic similarity knowledge distillation according to claim 3, characterized in that, The spatial semantic similarity matrix of the feature maps is represented as follows: ， ， ， ， in, The spatial semantic similarity matrix represents the teacher feature map at level l; Represents the local neighborhood radius; Indicates an indicator function, when hour, ,when hour, ; The spatial semantic similarity matrix represents the student feature map at level l; The spatial neighborhood mask represents the feature map of the teacher at level l; This represents the spatial neighborhood mask of the student feature map at level l. This indicates element-wise summation.

5. The method of claim 1, wherein the method is based on spatial semantic similarity knowledge distillation. Target similarity structure distillation loss The calculation formula is: ， in, Indicates the number of layers in a multi-scale feature extraction network; The spatial semantic similarity matrix represents the teacher feature map at level l; The spatial semantic similarity matrix represents the student feature map at level l; express Norm; Target perception classification distillation loss The calculation formula is: ， in, This represents the target detection results of the teacher model; This represents the target detection results of the student model; , This represents the spatial dimensions of the feature map at level l; Indicates the number of target categories; This represents the first object detection result of the teacher model at level l; This represents the second object detection result of the student model at level l.

6. An apparatus for target detection based on spatial semantic similarity knowledge distillation, comprising: The apparatus is used to implement the target detection method based on spatial semantic similarity knowledge distillation as described in any one of claims 1 to 5, comprising: The feature extraction module is used to input image samples from the training set into the teacher model and student model respectively for feature extraction, and output teacher feature maps and student feature maps. The semantic similarity acquisition module is used to flatten each feature map into a positional feature vector and normalize it, calculate the similarity between each feature value in the normalized positional feature vector, and obtain the semantic similarity matrix of each feature map. The spatial similarity acquisition module is used to map each feature map into a spatial coordinate vector, calculate the similarity between each feature value in the spatial coordinate vector, and obtain the spatial distance similarity matrix of each feature map. The similarity fusion module is used to fuse the semantic similarity matrix and spatial distance similarity matrix of each feature map to obtain the global affinity matrix of each feature map; The spatial semantic similarity acquisition module is used to generate a local neighborhood mask from the global affinity matrix of each feature map, obtain the neighborhood position association features of each pixel in the feature map, aggregate the affinity of the neighborhood position association features of all pixels, and obtain the spatial semantic similarity matrix of each feature map. The model training and object detection module is used to perform object detection and recognition based on teacher feature maps and their spatial semantic similarity matrices and student feature maps and their spatial semantic similarity matrices, and to construct object detection distillation loss. The student model is then trained and the trained student model is used for object detection.

7. A computer readable storage medium characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the target detection method based on spatial semantic similarity knowledge distillation as described in any one of claims 1 to 5.

Images