A small target detection method and system

Abstract

The application provides a small target detection method and system, and belongs to the field of computer vision. The method comprises the following steps: acquiring historical images and real label samples, wherein the sample categories include extreme small targets, relatively small targets, ordinary small targets and normal size targets; constructing a small target detection model; inputting the historical images into a convolutional neural network to extract feature maps; inputting the feature maps into an improved CRPN; obtaining candidate samples based on different sample categories, and then mapping the candidate samples on the feature maps to obtain regions of interest and features; obtaining the predicted categories and regression positions of the samples based on the features; simultaneously, screening positive samples and typical negative samples from the features of the regions of interest; training the small target detection model based on the predicted categories, the regression positions, a positive sample teacher set and a typical negative sample teacher set; and finally inputting the current image to be detected into the trained small target detection model to output a detection result. The application improves the precision and accuracy of small target detection.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to a small target detection method and system. BACKGROUND

[0002] With the rapid development of computer vision technology, target detection is increasingly widely used in daily life. Small object detection (SOD) is an important branch of target detection task, aiming to identify and locate the targets occupying a small number of pixels from images or videos. The small object detection task is crucial in many practical applications, for example, in autonomous driving, the accurate identification of small targets such as pedestrians and traffic signs is of key significance to driving safety; in security monitoring, the detection of suspicious behavior in a small range relies on the accurate positioning of small targets at a distance; in remote sensing images, detecting small buildings, ships and vehicles also belongs to the typical application scenarios of small target detection task. However, compared with conventional target detection, small targets usually have a small size and occupy a limited number of pixels in the image. This characteristic causes the features of small targets to be easily lost in the convolution process, significantly increasing the difficulty of detection. In addition, small targets have blurred appearance features, insufficient information and are greatly disturbed by noise, especially in complex background and multi-target scenes, traditional target detection methods are difficult to achieve ideal detection results. How to efficiently and accurately detect small targets has become an important research direction in the field of target detection.

[0003] In the prior art, the mainstream method of small target detection can be divided into one-stage detection method and two-stage detection method. These two types of methods have their own advantages in feature extraction, sample allocation and detection accuracy improvement of small targets. Among them, the single-stage detector in the one-stage target detection method usually adopts an end-to-end architecture, directly performs dense prediction on the input image without generating candidate regions, the main advantage is fast detection speed, suitable for application scenarios with high real-time requirements, and due to the simple structure, it is easy to implement and optimize; while the two-stage target detection method adopts a more detailed detection process, which is usually divided into two stages of candidate region generation and candidate region classification, the main advantage is high detection accuracy, especially in handling complex scenes and multi-scale targets, it can capture the detailed features of the target more effectively. However, due to the small size of small targets, there are problems such as insufficient training samples, and the screening criteria for different size targets cannot be unified, which makes it difficult for the above methods to capture enough effective information in the feature extraction process; in addition, small targets carry limited information and have low feature discrimination, which are often easily confused with the background or other objects, greatly increasing the risk of model misjudgment, resulting in low detection accuracy and accuracy of small targets. SUMMARY

[0004] In order to solve the above problems, the application provides a small target detection method and system, which improves the threshold calculation method of dynamic sample screening, is more suitable for sample screening of small size targets, collects high-quality positive sample features and typical negative sample features in training for positive and negative sample feature learning, uses the features as teacher features for positive and negative sample feature learning, improves the quality of small target feature representation, and thus improves the accuracy and precision of small target detection and improves the detection effect of small target tasks.

[0005] In order to achieve the above purpose, the technical solutions adopted by the embodiments of the application are as follows:

[0006] In a first aspect, the embodiments of the application provide a small target detection method, which comprises the following steps:

[0007] Step S1, historical images and real labeled samples in the historical images are obtained as training data sets, and the training data sets comprise categories of real labeled samples divided according to target sizes, including extreme small targets eS, relatively small targets rS, ordinary small targets gS and normal size targets N;

[0008] Step S2, a small target detection model is constructed, which comprises a convolutional neural network, an improved coarse and fine pipeline network CRPN, a mapping module, a classification head, a detection head and a loss function module;

[0009] Step S3, the historical images are input into the convolutional neural network to extract feature maps of the images, and the feature maps contain real labeled samples and category information;

[0010] Step S4, the feature maps are input into the improved CRPN to generate candidate samples through a coarse pipeline, optimize the candidate samples through a fine pipeline, and calculate classification scores of each candidate sample;

[0011] Step S5, the optimized candidate samples are mapped on the feature maps to obtain regions of interest, and the original features of the regions of interest are pooled and aligned to obtain features of the regions of interest;

[0012] Step S6, the features of the regions of interest are input into the classification head and the detection head to obtain predicted categories and regression positions of all real labeled samples; meanwhile, positive samples and typical negative samples are selected from the features of the regions of interest to establish a positive sample teacher set and a typical negative sample teacher set;

[0013] Step S7, based on the predicted categories, the regression positions, the positive sample teacher set and the typical negative sample teacher set, classification loss, regression loss, positive sample comparison loss and negative sample comparison loss of feature imitation are calculated, a comprehensive loss is obtained through the four kinds of loss calculation, and the training of the small target detection model is completed based on the comprehensive loss;

[0014] Step S8, inputting the current image to be detected into the small target detection model trained, and outputting all target detection results, wherein the target detection results include small target detection results.

[0015] As a preferred embodiment of the present application, step S4 generates candidate samples through a rough pipeline, specifically comprising:

[0016] Step S41, judging the classes of all real labeled samples in the feature map; for extreme small targets eS, performing step S42; for relative small targets rS and general small targets gS, performing step S43; for normal size targets N, performing step S44.

[0017] Step S42, calculating the IoU values of a predetermined number M of anchor boxes corresponding to each real labeled sample belonging to eS, sorting the anchor boxes in descending order of the IoU values, and selecting the first Q anchor boxes as candidate samples, wherein Q≤M.

[0018] Step S43, calculating the IoU values of a predetermined number M of anchor boxes corresponding to each real labeled sample belonging to rS and gS; constructing a nonlinear function of the area of each real labeled sample to calculate an IoU threshold value; and selecting anchor boxes with IoU values greater than the IoU threshold value T as candidate samples to realize the screening of samples in the rS and gS classes.

[0019] Step S44, calculating the IoU values of a predetermined number M of anchor boxes corresponding to each real labeled sample belonging to N; presetting an IoU threshold value, and selecting anchor boxes with IoU values greater than the IoU threshold value as candidate samples to realize the screening of samples in the N class.

[0020] As a preferred embodiment of the present application, the nonlinear function of the area of each real labeled sample is as follows:

[0021] T=min(T max , max(αC, βC+δγ×f(wh))) (1)

[0022] In formula (1), T max represents the upper limit of the IoU threshold value, C represents the IoU standard value, a represents the lower limit coefficient of the IoU threshold value, β represents the starting coefficient of the nonlinear function, w and h represent the width and height of the real labeled sample, f(wh) represents the nonlinear function of the product of the width and height of the real sample, γ represents the slope of the nonlinear function, and δ represents the slope coefficient.

[0023] As a preferred embodiment of the present application, the convolutional neural network in step S2 adopts a backbone network and adopts a Faster RCNN model.

[0024] As a preferred embodiment of the present application, the improved fine pipeline network in step S2 is constructed based on a dynamic threshold method, including a coarse pipeline network and a fine pipeline network, which respectively generate and optimize the candidate regions of the input image in sequence, and the threshold calculation process of the coarse pipeline and the fine pipeline is optimized to improve the CRPN.

[0025] As a preferred embodiment of the present application, step S6 screens positive samples and typical negative samples from the features of the region of interest, specifically including the following steps:

[0026] Obtain the IoU value and classification score of the candidate sample corresponding to the feature of the region of interest;

[0027] Define IQ_H = IoU x classification score, and preset a positive sample threshold T 正 , and the feature with IQ_H > T 正 is taken as a positive sample;

[0028] Define IQ_L = (1-IoU) x classification score, and preset a negative sample threshold T 负 , and the feature with IQ_L > T 负 is taken as a typical negative sample.

[0029] As a preferred embodiment of the present application, the positive sample contrast loss in step S7 adopts the loss function as shown below:

[0030]

[0031] In formula (2), p represents a candidate sample set, v represents a current candidate sample, and p u represents a current positive sample, v represents a sample in the positive sample set, and tau represents a temperature coefficient for controlling contrast learning.

[0032] As a preferred embodiment of the present application, the typical negative sample contrast loss in step S7 adopts the loss function as shown below:

[0033]

[0034] In formula (3), p represents a candidate sample set, v represents a current candidate sample, and n u represents a current negative sample, v represents a sample in the negative sample set, and tau represents a temperature coefficient for controlling contrast learning.

[0035] In a second aspect, the embodiments of the present application also provide a small target detection system, which comprises a data acquisition module, a model construction module, a feature map extraction module, a CRPN processing module, a mapped feature map module, a classification and detection module, a positive and negative sample set construction module, a model training module and a result output module, wherein

[0036] The data acquisition module is configured to acquire historical images and real labeled samples in the historical images as a training data set and input the feature map extraction module; the training data set comprises categories of real labeled samples divided according to target size, including extreme small target eS, relatively small target rS, general small target gS and normal size target N; and is also configured to acquire a current image to be detected and input the model construction module after model training is completed;

[0037] The model construction module is configured to construct a small target detection model, wherein the model comprises a convolutional neural network, an improved coarse and fine pipeline network CRPN, a mapping module, a classification head, a detection head and a loss function module; and is also configured to input the current image to be detected into the small target detection model after training is completed and send the detection result to the result output module;

[0038] The feature map extraction module is configured to input the historical images into the convolutional neural network to extract feature maps of the images, wherein the feature maps contain real labeled samples and category information;

[0039] The CRPN processing module is configured to input the feature maps into the improved CRPN, generate candidate samples through a coarse pipeline, optimize the candidate samples through a fine pipeline and calculate classification scores of each candidate sample;

[0040] The mapped feature map module is configured to map the optimized candidate samples on the feature maps to obtain regions of interest, pool and align original features of the regions of interest to obtain features of the regions of interest;

[0041] The classification and detection module is configured to map the optimized candidate samples on the feature maps to obtain regions of interest, pool and align original features of the regions of interest to obtain features of the regions of interest;

[0042] The positive and negative sample set construction module is configured to screen positive samples and typical negative samples from the features of the regions of interest to establish a positive sample teacher set and a typical negative sample teacher set;

[0043] The model training module is configured to calculate classification loss, regression loss, positive sample comparison loss and negative sample comparison loss of feature imitation based on the predicted category, the regression position, the positive sample teacher set and the typical negative sample teacher set, obtain a comprehensive loss through the four kinds of loss calculation, and complete the training of the small target detection model based on the comprehensive loss.

[0044] The result output module is configured to output all target detection results, wherein the target detection results include small target detection results.

[0045] The scheme of the embodiment of the present application has the following beneficial effects:

[0046] The small target detection method and system provided by the embodiment of the present application can more accurately screen out samples suitable for small target detection by improving the dynamic threshold calculation method of sample screening, especially for small targets with small size and weak features, and can effectively reduce the interference of background noise and improve the robustness and adaptability of the model in a complex environment by selecting a fixed number of samples from the qualified candidate samples for training. In addition, by introducing high-quality positive samples and typical negative sample features for learning in the training process, the interference of background noise can be further effectively reduced, the probability of false detection can be reduced, and the reliability of the detection result can be improved. The present application optimizes sample selection and feature extraction, reduces the negative impact of redundant samples on training, further improves the discrimination ability of the model in the small target detection process, enhances the sensitivity of the model to small target features, and improves the detection effect of small targets.

[0047] Of course, implementing any product or method of the present application does not necessarily require achieving all the advantages described above at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 1 is a flowchart of the small target detection method described in the embodiment of the present application;

[0050] Figure 2 is a comparison chart of the effect of the small target detection method of the present application and the existing detection method. DETAILED DESCRIPTION

[0051] The present inventors have found the above problems after careful research on existing small target detection methods. Research has found that for the problems existing in small target detection, scholars have proposed corresponding improvement methods, but there are still some deficiencies. For example, to solve the problem of insufficient training samples, the IoU-based training sample screening method can improve the sample screening effect; but for small targets, the IoU will change greatly due to the slight difference between the sample and the basic true value, so that the calculated threshold deviates from the common threshold range, resulting in that most of the positive samples are filtered out; some scholars use the method of reducing the threshold to obtain more samples with sizes comparable to small targets, but reducing the threshold by a large margin is not a good solution, although the number of small target training samples will increase, but the training sample set of normal size targets will introduce a large number of low-quality samples, the overall sample quality will actually decrease, and the detection effect of small targets and large size targets cannot be considered, and the problems of different size target screening standards cannot be unified, and the number of small target training samples is severely insufficient. In addition,

[0052] Due to the small size of small targets, the feature information carried by small targets is limited, resulting in that the feature representation obtained in the feature extraction process is often rough and fuzzy. In this case, the model is difficult to effectively capture the unique features of small targets, reducing the recognition ability of small targets. In addition, the existing scheme does not improve the quality of small target feature representation, and the features of small targets are often confused with the features of the background or other objects, further increasing the risk of misjudgment of the model in classification and positioning. Since small targets have fewer pixels in the image, their edge information and detailed features are not obvious, making it difficult for the model to extract enough discriminative features during feature learning; at the same time, the high similarity of features between different small targets also increases the difficulty of model classification.

[0053] It should be noted that the defects in the above prior art solutions are the result of the inventors' practice and careful research, therefore, the discovery process of the above problems and the solutions proposed by the embodiments of the present application to solve the above problems should be the contribution of the inventors to the present application.

[0054] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. It should be noted that the embodiments and features in the embodiments can be combined with each other without conflict.

[0055] It should be noted that similar reference numerals and letters refer to like items in the accompanying drawings, and once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings. In the description of the present application, the terms "first", "second", "third", "fourth" and the like are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0056] Based on the above in-depth analysis, the embodiment of the present application provides a small target detection method and system, which adopts a two-stage detection process, combines the optimization of threshold calculation method, provides different sample screening standards for real labeled samples of different sizes, and improves the feature representation quality of small targets through positive and negative sample feature learning based on feature imitation (FI), to realize more accurate, precise and effective small target detection.

[0057] As shown in Figure 1 The small target detection method of the embodiment includes the following steps:

[0058] Step S1, obtaining historical images and real labeled samples in the historical images as training data sets, and the training data sets include categories of real labeled samples divided by target size, including extreme small target eS, relative small target rS, general small target gS and normal size target N.

[0059] In this step, the historical images come from data in a standard data network; the classification of the extreme small target eS, the relative small target rS, the general small target gS and the normal size target N is classified according to the classification method in the standard image.

[0060] Step S2, constructing a small target detection model, the model including a convolutional neural network, an improved coarse-fine pipeline network, a mapping module, a classification head, a detection head and a loss function module.

[0061] In this step, the convolutional neural network adopts a backbone network for pre-processing of images, and adopts a Faster Region-based Convolutional Neural Network (Faster RCNN) model. The improved coarse-fine pipeline network (CRPN) is constructed based on a dynamic threshold method, including a coarse pipeline network and a fine pipeline network, which respectively generate and optimize the candidate regions of the input image in turn, and the threshold calculation process of the coarse pipeline and the fine pipeline is optimized to improve the CRPN.

[0062] Step S3, inputting the historical image into a convolutional neural network to extract a feature map of the image, wherein the feature map contains real labeled samples and category information.

[0063] Step S4, inputting the feature map into the improved CRPN to generate candidate samples through a coarse pipeline, optimize the candidate samples through a fine pipeline, and calculate a classification score of each candidate sample.

[0064] In this step, candidate samples are generated through the coarse pipeline, specifically including:

[0065] Step S41, judging the category of all real labeled samples in the feature map; performing step S42 for an extremely small target eS; performing step S43 for a relatively small target rS and a general small target gS; and performing step S44 for a normal size target N.

[0066] In this step, based on the classification of the real labeled samples, different candidate sample screening methods are adopted according to the different characteristics of each sample to generate anchor boxes dynamically in stages, so that more samples are screened for model training under the premise of retaining the effectiveness of the samples to the greatest extent, guaranteeing the recall rate of target samples of various categories, and performing more accurate and precise inference on the candidate samples in the Fine stage, so that the obtained model can detect small targets more accurately and precisely.

[0067] Step S42, calculating the IoU value of a predetermined number M of anchor boxes corresponding to each real labeled sample belonging to eS, sorting the anchor boxes in descending order of the IoU value, and selecting the first Q anchor boxes as candidate samples, wherein Q≤M.

[0068] In this step, the extremely small target is screened in the form of sorting to obtain more sample quantities, which is more in line with the actual situation of the extremely small sample and distinguishes it from the other two types of small target samples.

[0069] Step S43, calculating the IoU value of a predetermined number M of anchor boxes corresponding to each real labeled sample belonging to rS and gS; constructing a nonlinear function of the area of each real labeled sample to calculate an IoU threshold value; and taking the anchor boxes with an IoU value greater than the IoU threshold value T as candidate samples to realize the screening of samples in the rS and gS categories.

[0070] In this step, the constructed nonlinear function of the area of each real labeled sample is as follows:

[0071] T = min(T max , max(αC, βC+δγ×f(wh))) (1)

[0072] In formula (1), T maxIoU upper limit, C represents an IoU standard value, a represents an IoU threshold lower limit coefficient, β represents a nonlinear function starting coefficient, w and h represent the width and height of a real labeled sample, f(wh) represents a nonlinear function of the real sample width and height product, γ represents a nonlinear function slope, and δ represents a slope coefficient.

[0073] In this step, the IoU threshold is calculated by constructing a nonlinear function of the area of each real labeled sample. The constructed area nonlinear function includes sufficient consideration of factors affecting sample selection accuracy. The factors are reflected in the calculation of the formula in the form of weights, so that the calculation of the threshold is more in line with the class characteristics of the sample. Under normal circumstances, the threshold calculated by using the ordinary area nonlinear function will be more, so that the number of obtained samples is too small, which is not conducive to the training of the model. The present application assigns a weight value to the parameters in the nonlinear function, excludes the influence of other types of samples on the area nonlinear function, and thus obtains more samples.

[0074] Step S44, calculate the IoU value of the predetermined number M of anchor boxes corresponding to each real labeled sample belonging to N; preset an IoU threshold, and take the anchor box with an IoU value greater than the IoU threshold as a candidate sample, to realize the screening of samples in the N categories.

[0075] Step S5, map the optimized candidate sample on the feature map to obtain a region of interest, and pool and align the original features of the region of interest to obtain the features of the region of interest.

[0076] Step S6, input the features of the region of interest into a classification head and a detection head to obtain the predicted categories and regression positions of all real labeled samples; at the same time, screen out positive samples and typical negative samples from the features of the region of interest to establish a positive sample teacher set and a typical negative sample teacher set.

[0077] In this step, the positive samples and the typical negative samples are screened out from the features of the region of interest, which specifically includes the following steps:

[0078] Obtain the IoU value and the classification score of the candidate sample corresponding to the features of the region of interest.

[0079] Define IQ_H=IoU*classification score (cls_scores), and at the same time, preset a positive sample threshold T 正 , the features with IQ_H>T 正 as positive samples;

[0080] Define IQ_L=(1-IoU)*classification score (cls_scores), and at the same time, preset a negative sample threshold T 负 , the features with IQ_L>T 负 as typical negative samples.

[0081] In small target detection tasks, the interference of background noise and ambiguous regions is one of the main reasons for misidentification. By analyzing the visualization results, it is found that the model often misjudges the regions similar to the ambiguous background as target foreground during the training process. Such samples usually have a low IoU, but the model assigns them a high classification score, leading to the false identification of these samples as targets. This phenomenon is mainly due to the insufficient sensitivity of the model's feature representation ability to the difference between the background and the target, especially in small target detection, the distinction between the background and the target is more difficult. In order to effectively solve this problem, the invention proposes the definition of typical negative samples, that is, by calculating the (1-IoU)*cls_scores of the sample to determine whether it is defined as a negative sample.

[0082] Reasonably combine IoU and classification score to screen typical negative samples in training. By focusing on training typical negative samples, the model can identify and ignore background noise, improving the accuracy and robustness of target detection. This method is particularly effective in small target detection, as small targets are often easily covered by background noise or confused with the background. By accurately defining negative samples, the model can better focus on the true target, avoid being disturbed by the background, and improve the final detection effect.

[0083] In this step, based on the contrast learning, the positive sample teacher set and the typical negative sample teacher set are collected at the same time, based on the feature imitation idea, how to improve the feature representation quality of small targets is considered. Through the analysis of the visualization results, the model misidentifies a large number of blurred backgrounds as foreground. Such samples often have a low IoU, and the model usually calculates a high classification score for such samples, which are misidentified as foreground. How to reduce the misidentification of the model on such samples can effectively improve the detection effect of the model on small targets. By collecting high-quality positive samples in training to establish a teacher set, for the newly generated target feature representation, the similarity between different feature representations is calculated through contrast learning, and guidance and supervision are provided for the model to generate features; at the same time, the typical negative samples that are misidentified are used as teacher features to guide feature imitation. Specifically, the high-quality samples and low-quality samples in the training set are used as guidance for this process, and two dynamically updated sample feature libraries are constructed respectively, the features are mapped into the respective embedding spaces for comparison, and finally the loss is calculated by the respective loss function and the parameters are updated. The introduction of negative sample guidance can complete the feature imitation task when high-quality samples are insufficient. By collecting high-quality positive samples in training, and typical negative samples, a teacher set is established. For the newly generated target feature representation, the similarity between different feature representations is calculated through contrast learning, and guidance and supervision are provided for the model to generate features. For the newly generated target feature representation, the similarity between different feature representations is calculated through contrast learning, and guidance and supervision are provided for the model to generate features. The samples with a product exceeding a certain value introduce negative sample guidance, which can help generate high-quality positive samples to a certain extent. In addition, based on the contrast loss, the loss function of the positive and negative sample FI branches is designed. Among them, the contrast loss of the positive sample is greater as the teacher sample and the student sample are more similar, and the contrast loss of the negative sample is the opposite.

[0084] In step S7, based on the predicted category, the regression position, the positive sample teacher set and the typical negative sample teacher set, the classification loss, the regression loss, the positive sample contrast loss and the negative sample contrast loss of feature imitation (FI) are calculated, the comprehensive loss is obtained through the calculation of the four losses, and the training of the small target detection model is completed based on the comprehensive loss.

[0085] In this step, the positive sample contrast loss adopts the loss function as shown below:

[0086]

[0087] In formula (2), ρ represents a candidate sample set, v represents the current candidate sample, v p v represents the current positive sample, v represents the sample in the positive sample set, and τ represents the temperature coefficient for controlling the contrast learning.

[0088] The typical negative sample contrast loss adopts the loss function as shown in the following formula:

[0089]

[0090] In formula (3), p represents a candidate sample set, represents a current candidate sample, and n represents a current negative sample, represents a sample in a negative sample set, and τ represents a temperature coefficient for controlling contrast learning.

[0091] In this step, the contrast losses of the positive samples and the typical negative samples are respectively calculated based on the loss function, the training process is optimized, the sample representation quality in the training set is further improved through the cooperation of the positive sample learning and the negative sample learning, and the training is optimized and the model convergence speed is accelerated.

[0092] In step S8, the current image to be detected is input into the small target detection model with completed training, and all target detection results are output, wherein the target detection results include small target detection results.

[0093] Table 1 shows the comparison results of the small target detection method of the present application and the most advanced method on the SODA-D data set.

[0094] Table 1: Results of the model on the SODA-D benchmark test set

[0095]

[0096] As shown in Table 1, compared with the existing target detection methods, the method of the present application achieves the most advanced performance on the small target detection data set, wherein the average precision (AP) reaches 31.4%, and all indicators reach the current optimal. Specifically, compared with baseline (Faster-RCNN), the AP is improved by 2.5%; compared with the special small target detection method RFLA, the indicators are obviously ahead (AP: 1.7%, AP50: 1.6%, AP75: 2.7%, APeS: 2.4%, APrS: 1.0%, APgS: 2.0%, APN: 2.1%); most importantly, compared with the small target detection method CFINet with similar structure, the AP is improved by 0.7%, the AP50 is improved by 1.0%, the AP75 is improved by 1.2%, the APeS is improved by 0.9%, the APrS is improved by 0.1%, the APgS is improved by 1.0%, and the APN is improved by 2.1%. The experimental results prove that the method of the present application has superiority and universality, and has better performance and potential on the small target detection task.

[0097] As Figure 2As shown, the method of the present application is visualized on the SODA-D benchmark test set, wherein the boxes of different colors represent different categories of targets, and the visualization result verifies that the method of the present application still has excellent detection effect when facing extremely small targets.

[0098] Based on the same idea, the embodiment of the present application also provides a small target detection system, which comprises:

[0099] The system comprises a data acquisition module, a model construction module, a feature map extraction module, a CRPN processing module, a mapping feature map module, a classification and detection module, a positive and negative sample set construction module, a model training module and a result output module; wherein,

[0100] The data acquisition module is configured to acquire historical images and real labeled samples in the historical images as a training data set, and input the feature map extraction module; the training data set comprises categories of real labeled samples divided according to target size, including extremely small targets eS, relatively small targets rS, ordinary small targets gS and normal size targets N; and is also configured to acquire a current image to be detected, and input the model construction module after the model training is completed;

[0101] The model construction module is configured to construct a small target detection model, wherein the model comprises a convolutional neural network, an improved coarse and fine pipeline network, a mapping module, a classification head, a detection head and a loss function module; and is also configured to input the current image to be detected into the small target detection model after the model training is completed, and send the detection result to the result output module;

[0102] The feature map extraction module is configured to input the historical images into the convolutional neural network, and extract feature maps of the images, wherein the feature maps contain real labeled samples and category information;

[0103] The CRPN processing module is configured to input the feature maps into the improved CRPN, generate candidate samples through a coarse pipeline, optimize the candidate samples through a fine pipeline, and calculate a classification score of each candidate sample;

[0104] The mapping feature map module is configured to map the optimized candidate samples on the feature maps, obtain a region of interest, and pool and align the original features of the region of interest to obtain features of the region of interest;

[0105] The classification and detection module is configured to map the optimized candidate samples on the feature maps, obtain a region of interest, and pool and align the original features of the region of interest to obtain features of the region of interest;

[0106] The positive and negative sample set construction module is configured to screen positive samples and typical negative samples from features of the region of interest, and to establish a positive sample teacher set and a typical negative sample teacher set.

[0107] The model training module is configured to calculate a classification loss, a regression loss, a positive sample contrast loss and a negative sample contrast loss of feature imitation (FI) based on the predicted category, the regression position, the positive sample teacher set and the typical negative sample teacher set, to obtain a comprehensive loss by calculating the four kinds of losses, and to complete training of the small target detection model based on the comprehensive loss.

[0108] The result output module is configured to output all target detection results, wherein the target detection results include small target detection results.

[0109] The modules in the embodiment are implemented by a processor, and a memory is appropriately added when storage is needed. The processor can be, but is not limited to, a microprocessor (MPU), a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the like. The memory can include a random access memory (RAM) and a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0110] In the above embodiment, all or part of the embodiment can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiment can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function described in the embodiment of the present application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode.

[0111] In addition, it should be noted that the small target detection system and the small target detection method described in the embodiment are corresponding. The description and limitation of the method are also applicable to the system, and will not be described here.

[0112] The above description is merely preferred embodiments of the present application and a description of the principles of the technology used, and is not intended to limit the scope of the claimed application, but merely represents the preferred embodiments of the application. Those skilled in the art should understand that the scope of the application involved in the application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

Claims

1. A small target detection method, characterized in that: The method comprises the following steps: Step S1: obtaining historical images and real annotated samples in the historical images as a training dataset, wherein the training dataset includes categories of real annotated samples divided by target size, including extremely small targets eS, relatively small targets rS, ordinary small targets gS, and normal-sized targets N; Step S2, constructing a small target detection model, the model including a convolutional neural network, an improved thick-thin pipeline network (CRPN), a mapping module, a classification head, a detection head, and a loss function module; Step S3: inputting the historical image into a convolutional neural network to extract a feature map of the image, wherein the feature map contains real labeled samples and category information; Step S4: input the feature map into the improved CRPN, generate candidate samples through the coarse pipeline, optimize the candidate samples through the fine pipeline, and calculate the classification score of each candidate sample; Step S5, mapping the optimized candidate samples on the feature map to obtain the region of interest, pooling and aligning the original features of the region of interest to obtain the features of the region of interest; Step S6: Input the features of the region of interest into the classification head and the detection head to obtain the predicted categories and regression positions of all real labeled samples; at the same time, screen out positive samples and typical negative samples from the features of the region of interest to establish a positive sample teacher set and a typical negative sample teacher set; and the screening of positive samples and typical negative samples from the features of the region of interest specifically includes the following steps: Get the IoU value and classification score of the candidate samples corresponding to the features of the region of interest; Define IQ_H = IoU × classification score, and preset the positive sample threshold T 正 , IQ_H>T 正 The features of are taken as positive samples; Define IQ_L = (1-IoU) × classification score, and preset the negative sample threshold T 负 , IQ_L>T 负 The features of are taken as typical negative samples; Step S7, based on the predicted category, regression position, positive sample teacher set and typical negative sample teacher set, calculate the classification loss, regression loss, positive sample contrast loss of feature imitation and negative sample contrast loss, calculate the comprehensive loss through the four losses, and complete the training of the small target detection model based on the comprehensive loss; The positive sample contrast loss uses the following loss function: In formula (2), ρ represents the candidate sample set, represents the current candidate sample, υ p represents the current positive sample, represents the sample in the positive sample set, τ represents the temperature coefficient of control contrastive learning; The typical negative sample contrast loss uses the following loss function: In formula (3), ρ represents the candidate sample set, represents the current candidate sample, υ n represents the current negative sample, represents the sample in the negative sample set, τ represents the temperature coefficient of control contrastive learning; Step S8: input the current image to be detected into the trained small target detection model, and output all target detection results, including small target detection results.

2. The small target detection method according to claim 1, characterized in that: Step S4 generates candidate samples through the rough pipeline, specifically including: Step S41, determine the categories of all true labeled samples in the feature map; for extremely small targets eS, execute step S42; for relatively small targets rS and ordinary small targets gS, execute step S43; for normal-sized targets N, execute step S44; Step S42: Calculate the IoU values ​​of a predetermined number of M anchor boxes corresponding to each true labeled sample belonging to eS, sort the anchor boxes in descending order of IoU values, and select the top Q anchor boxes as candidate samples, with Q ≤ M. Step S43: Calculate the IoU values ​​of a predetermined number of anchor boxes M corresponding to each true annotated sample belonging to rS and gS; construct a nonlinear function of the area of ​​each true annotated sample to calculate the IoU threshold; select the anchor boxes with IoU values ​​greater than the IoU threshold T as candidate samples to implement the screening of samples in the rS and gS categories; Step S44, calculate the IoU values ​​of a predetermined number M of anchor boxes corresponding to each true labeled sample belonging to N; preset an IoU threshold, and select anchor boxes with IoU values ​​greater than the IoU threshold as candidate samples to achieve screening of samples in N categories.

3. The small target detection method according to claim 2, characterized in that: The nonlinear function of the area of ​​each true labeled sample constructed is as follows: T=min(T max , max(αC, βC+δγ×f(wh))) (1) In formula (1), T max Represents the upper limit of the IoU threshold, C represents the IoU standard value, α represents the lower limit coefficient of the IoU threshold, β represents the starting coefficient of the nonlinear function, w and h represent the width and height of the real labeled sample, f(wh) represents the nonlinear function of the product of the width and height of the real sample, γ represents the slope of the nonlinear function, and δ represents the slope coefficient.

4. The small target detection method according to claim 1, characterized in that: The convolutional neural network described in step S2 adopts a backbone network and a fast regional convolutional neural network Faster RCNN model.

5. The small target detection method according to claim 1, characterized in that: The improved coarse and fine pipeline network described in step S2 is constructed based on the dynamic threshold method, including a coarse pipeline network and a fine pipeline network, which respectively generate and optimize candidate regions for the input image. By optimizing the threshold calculation process of the coarse pipeline and the fine pipeline, the CRPN is improved.

6. A small target detection system applied to the small target detection method according to any one of claims 1 to 5, characterized in that: The system includes: a data acquisition module, a model construction module, a feature map extraction module, a CRPN processing module, a mapping feature map module, a classification and detection module, a positive and negative sample set construction module, a model training module and a result output module; wherein, The data acquisition module is used to acquire historical images and real annotated samples in the historical images as a training data set and input them into the feature map extraction module; the training data set includes categories of real annotated samples divided by target size, including extremely small targets eS, relatively small targets rS, ordinary small targets gS, and normal-sized targets N; it is also used to acquire the current image to be detected and input it into the model construction module after model training is completed; The model construction module is used to build a small target detection model, which includes a convolutional neural network, an improved thick-thin pipe network (CRPN), a mapping module, a classification head, a detection head, and a loss function module. It is also used to input the current image to be detected into the trained small target detection model after the model training is completed, and send the detection result to the result output module; The feature map extraction module is used to input the historical image into a convolutional neural network to extract the feature map of the image, where the feature map contains real labeled samples and category information; The CRPN processing module is used to input the feature map into the improved CRPN, generate candidate samples through the coarse pipeline, optimize the candidate samples through the fine pipeline, and calculate the classification score of each candidate sample; The mapping feature map module is used to map the optimized candidate samples on the feature map to obtain the region of interest, and to pool and align the original features of the region of interest to obtain the features of the region of interest; The classification and detection module is used to input the features of the region of interest into the classification head and the detection head to obtain the predicted categories and regression positions of all real labeled samples; The positive and negative sample set construction module is used to screen positive samples and typical negative samples from the features of the region of interest, and establish a positive sample teacher set and a typical negative sample teacher set; The model training module is used to calculate the classification loss, regression loss, positive sample contrast loss of feature imitation and negative sample contrast loss based on the predicted category, regression position, positive sample teacher set and typical negative sample teacher set, and obtain the comprehensive loss through the calculation of the four losses, and complete the training of the small target detection model based on the comprehensive loss; The result output module is used to output all target detection results, including small target detection results.

Images