Small target detection method based on image two-dimensional entropy segmentation and knowledge distillation

Abstract

The invention provides a small target detection method based on image two-dimensional entropy segmentation and knowledge distillation, and belongs to the technical field of image small target detection. In order to solve the technical problems of insufficient small target feature resolution, lack of significant features and insufficient model generalization ability in a complex traffic scene in the existing small target detection technology, the adopted technical scheme is as follows: constructing a small target detection model in an automatic driving environment of the complex traffic scene; designing an image segmentation selection strategy based on a candidate region selection method of image two-dimensional entropy and density map estimation; taking the pre-trained visual language model as a teacher model to perform knowledge distillation processing, transmitting knowledge to the lightweight student model, and constructing a target detection head; a two-stage target detection model is constructed, and potential small targets in a complex traffic scene are detected based on the target detection model; the method is applied to small target detection of automatic driving.

Description

Technical Field

[0001] This invention provides a small target detection method based on two-dimensional image entropy segmentation and knowledge distillation, belonging to the field of image small target detection technology. Background Technology

[0002] With the rapid development of artificial intelligence, deep learning and environmental perception technologies are now being applied to autonomous driving, effectively improving the safety and intelligence of road traffic. However, complex and ever-changing traffic scenarios also bring new challenges to autonomous driving technology, placing higher demands on its reliability and safety.

[0003] In complex traffic scenarios, it is necessary to detect small targets such as distant traffic signs and pedestrians, which often occupy only a few pixels in the detected image. As one of the important research directions in the field of computer vision, small target image detection in autonomous driving can not only effectively reduce the incidence of traffic accidents and improve road traffic efficiency, but also play a key role in new transportation modes such as smart city construction and shared mobility. Therefore, small target detection, as a key capability of autonomous driving perception system, can provide a reliable basis for decision-making.

[0004] Current methods for small object detection mainly include: sampling-based methods, scale-aware methods, attention-based methods, feature mimicry methods, and context modeling methods. However, these methods have significant limitations in perceiving and detecting small objects in images.

[0005] Sampling-oriented methods: These methods aim to address the problem of insufficient training samples for small targets, typically employing data augmentation strategies or optimized label allocation strategies. In addition to traditional data augmentation methods, some studies also utilize techniques such as GANs, image inpainting, and image fusion to synthesize high-quality small target sample data, thereby increasing the number of small targets. These methods are simple and direct, and can be combined with existing network models to effectively alleviate the problem of small target sample scarcity. However, these methods do not consider the characteristics of the small targets themselves, and the quality of the generated small target samples needs further analysis, as they may introduce noise information and lead to false detections. Therefore, they are difficult to directly adapt to the high-precision requirements of complex traffic scenarios.

[0006] Scale-aware methods: In complex traffic application scenarios, targets in images often vary greatly in scale. Single-scale detection methods are usually difficult to solve this problem. Scale-aware methods aim to alleviate this problem. They usually adopt the construction of detectors at specific scales or feature fusion methods, which are commonly found in feature pyramid modules and their improved methods. These methods are specifically designed for targets of different scales and significantly improve the visibility and representation of small target features. However, while improving the performance of small target detection, these methods are usually accompanied by an increase in computational and memory overhead.

[0007] Attention-based methods: By assigning different weights to different parts of the feature map, attention mechanisms emphasize valuable regions in the image, thereby suppressing relatively minor regions. Such methods can combine spatial and channel information to effectively suppress background noise. However, the design of the attention range and hierarchy is relatively complex, and performance improvements are often accompanied by heavy computational overhead, which to some extent limits the further practical application of such methods.

[0008] Feature mimicry methods: Small objects in images contain less effective information and are often difficult to represent with high quality, while relatively large instances tend to have better representations. Based on this idea, feature mimicry methods add additional similarity loss or super-resolution architecture to existing detectors, enabling models to mine the intrinsic correlation between small and large objects, thereby enhancing the semantic representation of small objects. Such methods can enhance the semantic information of low-level features, which is beneficial for improving the feature representation ability of small objects. However, such methods tend to increase training complexity and training time, and it is also difficult to guarantee feature diversity, which to some extent limits the further practical application of such methods.

[0009] Contextual modeling approach: For small object detection, context can sometimes provide more decision support. Based on this idea, some researchers have tried to use contextual information to improve the performance of small object detection models. By enhancing the local and global context of the model, additional information can be provided for the detection and recognition of small objects, which can effectively improve the recognition accuracy of small objects in complex backgrounds. However, introducing inappropriate context may introduce noise and lead to false detections. In addition, the relevant information of the context has limited effect on small object detection in dense scenes, which limits the further application of such methods.

[0010] To address the shortcomings in small target detection mentioned above, it is necessary to research a detection method with strong model generalization ability and improved accuracy in locating potential small targets in images, so as to better adapt to different complex traffic scenarios. Summary of the Invention

[0011] To address the technical problems of insufficient feature resolution, lack of salient features, and insufficient model generalization ability of existing small target detection technologies in complex traffic scenarios, this invention provides a small target detection method based on two-dimensional image entropy segmentation and knowledge distillation, comprising the following detection steps:

[0012] Step 1: Construct a small target detection model in an autonomous driving environment with complex traffic scenarios;

[0013] Step 2: Based on the candidate region selection method of image two-dimensional entropy and density map estimation, design an image segmentation selection strategy;

[0014] Step 3: Use the pre-trained visual language model as the teacher model to perform knowledge distillation, and transfer the knowledge to the lightweight student model to build the object detection head;

[0015] Step 4: Construct a two-stage target detection model and use this model to detect potential small targets in complex traffic scenarios.

[0016] The specific method for step 1 is as follows:

[0017] The expression for defining small target features is:

[0018] ;

[0019] in, Indicates the width and height of the target. Indicates the width and height of the image;

[0020] The characteristics of small targets in autonomous driving are defined as follows:

[0021] Given an image of a complex traffic scene The output of object detection is a set of objects. This contains potential small target objects, each target Represented by category and bounding box:

[0022] ;

[0023] ;

[0024] ;

[0025] in, Indicates the target category. Represents the target bounding box.

[0026] The specific method for step 2 is as follows:

[0027] Step 2.1: Calculate the two-dimensional entropy of the image and convert the RGB image to a grayscale image. The expression is:

[0028] ;

[0029] For each local window of size k*k Calculate the local gray-level histogram and local probability The distribution and calculation formulas are as follows:

[0030] ;

[0031] ;

[0032] Then calculate the local two-dimensional entropy of the image. The calculation formula is:

[0033] ;

[0034] in, Used to prevent the argument of a logarithm from being zero;

[0035] By calculating the local two-dimensional entropy of the image, the amount of information in the local region of the image is determined, and the high entropy region where there is a potential small target is found.

[0036] Step 2.2: Calculate the density map using image two-dimensional entropy and intensity-weighted density estimation. The calculation formula is as follows:

[0037] ;

[0038] in, Representing the normalized color features, we can find regions with high density and high image entropy values ​​where potential small targets exist.

[0039] Then the density map is thresholded, as expressed by:

[0040] ;

[0041] Where T is the adaptive threshold;

[0042] This guides the arbitrary segmentation model to perform fine segmentation of the potential region of small targets.

[0043] The specific method for step 3 is as follows:

[0044] Step 3.1: To make the spatial features of students more closely resemble those of teachers in semantically important positions, feature distillation is performed to align the teacher's projected features. The calculation formula is as follows:

[0045] ;

[0046] Projected teacher characteristics With student characteristics Spatial alignment, after positional alignment, is denoted as... ;

[0047] Using teacher response graphs The weighted L2 loss is calculated using the following formula:

[0048] ;

[0049] ;

[0050] in, , used to amplify weights at high response points; N represents the number of spatial locations;

[0051] Step 3.2: Output distillation:

[0052] The teacher's global or proposal-level soft-label is passed to the student's classification logits, expressed as:

[0053] ;

[0054] in, These represent the corresponding logits output by the student model and the teacher model, respectively, where T represents temperature;

[0055] Step 3.3: Calculate the loss function:

[0056] Define the total training loss as The calculation formula is:

[0057] ;

[0058] in, This indicates the loss of standard testing tasks;

[0059] The above formula specifies the values ​​of the initial hyperparameters. , Subsequently, comparative distillation loss was added, and the hyperparameter values ​​were adjusted to obtain the final target detection model.

[0060] The advantages of this invention compared to existing technologies are as follows: This invention provides a small target detection method based on two-dimensional image entropy segmentation and knowledge distillation. This method combines a large model with knowledge distillation technology and applies it to small target detection in complex traffic scenarios, promoting the integration and development of artificial intelligence and intelligent transportation systems. By adopting the SAM segmentation model and designing segmentation strategies to guide the segmentation results, the segmentation model as a whole adapts to the small target detection problem in complex traffic scenarios, enhancing the accuracy of potential localization of small targets in images. This invention not only considers the accuracy of the detection model in small target detection in autonomous driving, but also further improves the generalization ability of the model by introducing a large model, enabling the detection model to better adapt to different complex traffic scenarios. Attached Figure Description

[0061] The present invention will be further described below with reference to the accompanying drawings:

[0062] Figure 1 This is a flowchart illustrating the steps of the small target detection method based on two-dimensional image entropy segmentation and knowledge distillation of the present invention.

[0063] Figure 2 This is a flowchart illustrating the steps of the segmentation strategy based on image two-dimensional entropy and density map estimation in an embodiment of the present invention.

[0064] Figure 3 This is a flowchart illustrating the steps of knowledge distillation based on VLM in an embodiment of the present invention. Detailed Implementation

[0065] like Figures 1 to 3 As shown, this invention proposes a two-stage small target detection method based on image 2D entropy segmentation and knowledge distillation for small target detection in complex traffic scenarios. It primarily utilizes a candidate region selection method based on image 2D entropy and density map estimation. A target detection head is constructed using a Virtual Model (VLM) and knowledge distillation. A pre-trained VLM serves as the teacher model, and knowledge is transferred to a lightweight student model via knowledge distillation, ultimately building a two-stage target detection model. Since small targets cover a small area, the SAM model tends to focus on larger targets in the image while neglecting smaller ones. This invention, based on the SAM model, employs image 2D entropy and density map estimation to reflect the amount of information contained in the image and the local target density, thereby evaluating the segmentation results and further refining them. Furthermore, this invention fully considers the model's generalization ability, using a pre-trained large model as the teacher model and constructing a lightweight student model through knowledge distillation, effectively improving the model's generalization ability. Finally, the above methods are integrated into a two-stage target detection model.

[0066] To achieve the above technical effects, the small target detection method based on two-dimensional image entropy segmentation and knowledge distillation provided by this invention specifically includes the following detection steps:

[0067] Step 1: Describe and define small target detection:

[0068] Small target detection is a sub-problem in target detection, focusing on small-sized, low-pixel-ratio, and easily obscured areas in an image. Its aim is to accurately identify and locate relatively small targets within the image. Small targets are typically defined as targets with relatively small dimensions in an image (e.g., occupying an area <1%-2% of the image), and are defined as follows:

[0069] ;

[0070] in, Indicates the width and height of the target. This indicates the width and height of the image.

[0071] In complex traffic scenarios, small object detection in autonomous driving is defined as follows:

[0072] Given an image of a complex traffic scene The output of object detection is a set of objects. This includes potential small target objects (cars and pedestrians in the distance, as well as traffic lights, traffic signs, etc.), each target Represented by category and bounding box:

[0073] ;

[0074] ;

[0075] ;

[0076] in, Indicates the target category. Represents the target bounding box.

[0077] Step 2: Design a segmentation strategy based on image 2D entropy and density map estimation:

[0078] With the emergence of large-scale models, some researchers have adopted the Segment Anything (SAM) model to identify potential target regions. However, because small targets cover relatively small areas, the SAM model tends to focus on larger targets in the image while ignoring the existence of small targets. Therefore, this invention designs the following segmentation selection strategy, such as... Figure 2 As shown, a two-dimensional entropy and graph density map estimation method for images is introduced. The specific steps are as follows:

[0079] Step 2.1: Calculation of the two-dimensional entropy of the image:

[0080] The two-dimensional entropy of an image can reflect the average amount of information contained in the entire image. The larger the entropy value, the more information the image contains; conversely, the smaller the entropy value, the less information the image contains. The two-dimensional entropy of an image can be used to measure the amount of information in a local region of the image. First, the RGB image is converted into a grayscale image, as shown in formula (1):

[0081] (1);

[0082] For each local window Calculate the local grayscale histogram (size k*k). and local probability The distribution is shown in formulas (2) and (3):

[0083] (2);

[0084] (3);

[0085] Based on this, the local entropy of the image is calculated. As shown in formula (4):

[0086] (4);

[0087] in, This is used to prevent the argument of the logarithm from being zero. By calculating the local two-dimensional entropy of an image, regions with high entropy values ​​can be identified, potentially indicating the presence of small targets.

[0088] Step 2.2: Density map calculation:

[0089] Density map estimation is commonly used in target counting problems. It generates a density map using a Gaussian kernel or other kernel functions, where each pixel value represents the target density at that location. Therefore, the density map emphasizes the clustering of high-information regions. This invention employs an image two-dimensional entropy and intensity-weighted density estimation method, as shown in formula (5):

[0090] (5);

[0091] in, The normalized color features represent regions that simultaneously satisfy both high density and high image entropy, indicating a greater likelihood of potential small targets. Finally, the density map is thresholded, as shown in formula (6) (where T is the adaptive threshold), and this threshold guides the SAM model to perform fine segmentation of potential regions for small targets.

[0092] (6);

[0093] Step 3: Knowledge Distillation

[0094] Vision Language Models (VLMs) are often used as base models and exhibit strong generalization ability in many downstream applications. To further enhance the generalization ability of the model, knowledge distillation is used. A pre-trained VLM is used as a teacher model, which transfers knowledge from a larger teacher model to a lightweight target model, thereby further improving the model's generalization ability. The knowledge distillation process based on VLM in this invention is as follows: Figure 3 As shown, it specifically includes:

[0095] Step 3.1: Characteristic distillation:

[0096] Feature distillation aims to make the spatial features of students closer to those of teachers in semantically important positions. First, the teacher's features are projected and aligned, as shown in formula (7):

[0097] (7);

[0098] Projected teacher characteristics With student characteristics Spatial alignment, after positional alignment, is denoted as... Using teacher response graphs The weighted L2 loss is shown in formulas (8) and (9):

[0099] (8);

[0100] (9);

[0101] in , used to amplify weights at high response points; N represents the number of spatial locations.

[0102] Step 3.2: Output distillation:

[0103] The teacher's global or proposal-level soft-label is passed to the student's classification logits, as shown in formula (10):

[0104] (10);

[0105] in, These represent the corresponding logits output by the student model and the teacher model, respectively, where T represents temperature.

[0106] Step 3.3: Loss function calculation:

[0107] Total training loss is recorded as As shown in formula (11):

[0108] (11);

[0109] in This represents the loss of the standard detection task. In the experiment, the initial hyperparameters... , Later, we can add a comparison of distillation losses and adjust the hyperparameter values.

[0110] Step 4: Finally, a two-stage target detection model is constructed, and the potential small targets in complex traffic scenarios are detected based on this target detection model.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A small target detection method based on image two-dimensional entropy segmentation and knowledge distillation, characterized in that: The detection steps include the following steps: Step 1: constructing a small target detection model in an automatic driving environment of a complex traffic scene; Step 2: designing an image segmentation selection strategy based on an image two-dimensional entropy and a density map estimation candidate region selection method; Step 3: using a pre-trained visual language model as a teacher model to perform knowledge distillation processing and transfer knowledge to a lightweight student model to construct a target detection head; Step 4: constructing a two-stage target detection model to detect potential small targets in a complex traffic scene.

2. The small target detection method based on image two-dimensional entropy segmentation and knowledge distillation according to claim 1, characterized in that: The specific method of step 1 is as follows: The expression of the small target feature is defined as: ； wherein, denotes the width and height of the target, denotes the width and height of the image; The feature of the small target in the automatic driving is defined as: Given an image of a complex traffic scene , the output of object detection is a set of objects , which contains potentially small objects, each object is represented by a class and a bounding box. ； ； ； wherein, represents a target class, represents a target bounding box.

3. The small target detection method based on image two-dimensional entropy segmentation and knowledge distillation according to claim 2, characterized in that: The specific method of step 2 is as follows: Step 2.1: calculate the image two-dimensional entropy, convert the RGB image to a grayscale image, and the expression is: ； For each local window of size k*k , the local gray level histogram and local probability distribution are calculated, respectively as follows: ； ； The local two-dimensional entropy of the image is then calculated with the formula: ； wherein, for preventing the log's true number from taking zero; By calculating the local two-dimensional entropy of the image, the information amount of the local region of the image is determined, and the high entropy value region where the potential small target exists is found. Step 2.2: calculate the density map by using the image two-dimensional entropy and intensity weighted density estimation, and the calculation formula is: ； wherein, The normalized color feature indicates that the region with high density and high image entropy value exists potential small target. Then, the density map is thresholded, and the expression is: ； Where T is the adaptive threshold. This guides any segmentation model to perform fine segmentation on the potential region of the small target.

4. The small target detection method based on image two-dimensional entropy segmentation and knowledge distillation according to claim 3, characterized in that: The specific method of step 3 is as follows: Step 3.1: to make the spatial features of the student closer to the teacher in the semantic important position, perform feature distillation processing, project and align the teacher features, and the calculation formula is: ； projected teacher features with student features spatially aligned, position-wise aligned and written as ; Using teacher response graphs Weighted L2 loss, computed as: ； ； wherein, a weight used for amplification at a high response; N represents the number of spatial positions; Step 3.2: output distillation: Transfer the teacher's global or proposal level soft-label to the student classification logits, and the expression is: ； wherein, respectively denote the corresponding logits output by the student and teacher models, and T denotes the temperature; Step 3.3: calculate the loss function: The total training loss is defined as The calculation formula is: ； wherein, represents a standard detection task loss; The initial hyperparameters in the above formula are taken as , , the contrast distillation loss is added subsequently, and the values of the hyperparameters are adjusted to obtain the final target detection model.

Images