A small target detection method and device

Abstract

The application provides a small target detection device and method. The method comprises the following steps: acquiring a to-be-detected image and a blank image; determining a full target region image and a small target region image in the to-be-detected image; wherein the small target region image is covered by the full target region image; scaling the to-be-detected image to a preset size to obtain a scaled image; and obtaining a spliced image based on the blank image, the full target region image, the small target region image and the scaled image; inputting the spliced image into a detection model to obtain an output parameter; and determining a region of the small target in the to-be-detected image based on the output parameter. The application embodiment improves the small target detection rate.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a small target detection method and device. Background Art

[0002] Object detection generally refers to the classification and location of objects in images captured by cameras or other visual sensors. Small object detection, also known as small-scale object detection, involves detecting objects with fewer pixels. This is widely used in applications such as automotive and facial recognition.

[0003] However, small object detection remains a significant challenge. Due to the low pixel count, limited features, and easy confusion with the background, mainstream object detection algorithms still perform far below expectations. Even deep learning-based object detection still presents significant challenges for small objects and faces, primarily due to low image resolution, blurry images, and high background noise, which hinder effective feature extraction. Therefore, a small object detection method is urgently needed to improve detection rates. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects of the prior art and provide a small target detection method and device to improve the small target detection rate.

[0005] The present invention is achieved in that:

[0006] In a first aspect, a small target detection method is provided, the method comprising:

[0007] Obtain the image to be detected and the blank image;

[0008] Determining a full target area image and a small target area image in the image to be detected; wherein the small target area image is covered by the full target area image; full target area image;

[0009] Scaling the image to be detected to a preset size to obtain a scaled image; and obtaining a spliced ​​image based on the blank image, the full target area image, the small target area image, and the scaled image;

[0010] The spliced ​​image is input into a detection model to obtain output parameters; and based on the output parameters, an area of ​​the small target in the image to be detected is determined.

[0011] Optionally, the full target area image and the small target area image are obtained by dividing the image to be detected based on a pre-set first fixed area and a second fixed area; the first fixed area is an area where the frequency of occurrence of full targets exceeds a first threshold; the second fixed area is an area where the frequency of occurrence of small targets exceeds a second threshold.

[0012] Optionally, obtaining a spliced ​​image based on the blank image, the full target area image, the small target area image, and the zoomed image includes:

[0013] The full target area image, the small target area image and the zoomed image are arranged in a preset manner to cover the blank image to obtain a spliced ​​image.

[0014] Optionally, the spliced ​​image is input into a detection model to obtain output parameters, wherein the output parameters include one or more of target category, target category confidence, target position, and target position confidence;

[0015] Among them: the target category is used to characterize the category to which the target belongs; the target category confidence is used to characterize the degree of credibility that the target belongs to the target category; the target position is used to characterize the position of the target in the image to be detected; the target position confidence is used to characterize the degree of credibility that the target is at the target position.

[0016] Optionally, determining the area of ​​the small target in the image to be detected based on the output parameter includes:

[0017] If the area corresponding to the target position is the area where the small target area image is located, then determine whether the confidence of the target position is greater than the preset first position confidence; if it is, determine that the area where the target position is located is the initial small target area; determine whether the target category confidence is greater than the preset first category confidence; if it is, determine that the target category prediction is correct; restore the initial small target area to the corresponding area of ​​the image to be detected, and obtain the area of ​​the small target corresponding to the target category in the image to be detected.

[0018] Optionally, the method further includes:

[0019] If the area corresponding to the target position is a preset false detection area, determine whether the target position confidence is greater than the preset second position confidence; if greater, determine that the area where the target position is located is a false detection area, and remove the target position and the target category.

[0020] Optionally, if there are multiple small targets, the method further includes:

[0021] After obtaining the area of ​​each small target in the image to be detected, the intersection-and-union ratio of each small target area is calculated; the small target areas whose intersection-and-union ratio is greater than a preset intersection-and-union ratio threshold are weightedly fused to obtain a fusion area; and the target category corresponding to the target category with the maximum confidence in the fusion area is used as the fusion target category of the fusion area, and the fusion area is used as the area of ​​the small target corresponding to the fusion target category in the image to be detected.

[0022] Optionally, the testing method of the detection model includes:

[0023] A sample image test set is obtained, and the detection model is tested using the sample image test set to obtain a small target detection threshold, a potential false detection area, and a false detection removal confidence level.

[0024] Optionally, determining the area of ​​the small target in the image to be detected based on the target category, the target category confidence, the target position, and the target position confidence includes:

[0025] If the area corresponding to the target position is the area where the small target area image is located, then determining whether the confidence of the target position is greater than the small target detection threshold; if so, determining that the area where the target position is located is the initial small target area;

[0026] If the area corresponding to the target position is the potential false detection area, then determine whether the target position confidence is greater than the false detection removal confidence; if so, determine that the area where the target position is located is a false detection area, and remove the target position and the target category;

[0027] After determining that the area where the target position is located is the initial small target area, determine whether the target category confidence is greater than the preset category confidence; if it is greater, determine that the target category prediction is correct; restore the initial small target area to the corresponding area of ​​the image to be detected, and obtain the area of ​​the small target corresponding to the target category in the image to be detected.

[0028] In a second aspect, the present invention provides a small target detection device, the device comprising:

[0029] A first acquisition unit, configured to acquire an image to be detected and a blank image;

[0030] a determining unit, configured to determine a full target area image and a small target area image in the image to be detected; wherein the small target area image is covered by the full target area image;

[0031] a second acquisition unit, configured to scale the image to be detected to a preset size to obtain a scaled image; and to obtain a spliced ​​image based on the blank image, the full target area image, the small target area image, and the scaled image;

[0032] The detection unit is configured to input the spliced ​​image into a detection model to obtain output parameters; and based on the output parameters, determine the area of ​​the small target in the image to be detected.

[0033] The present invention has the following beneficial effects: By applying embodiments of the present invention, by reducing the image to be detected and splicing it with the full target area image and the small target area image, the features of the small target area are enhanced, thereby improving the small target detection rate. Furthermore, since no additional image features are added, this method improves the small target detection rate while substantially increasing the existing deep learning detection computation, demonstrating promising results in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 A schematic diagram of a flow chart of a small target detection method provided by an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of the positional relationship between the first area and the second area provided in an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of the positional relationship between regions of a spliced ​​image provided by an embodiment of the present invention;

[0038] Figure 4 A schematic structural diagram of a small target detection device provided by an embodiment of the present invention;

[0039] Figure 5 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0041] In the field of target detection, in order to accurately detect small targets and improve the detection rate of small targets, the inventors of this case conducted research and found that in the existing technology, there are several methods for detecting small targets in infrared images:

[0042] 1. Add more complex network structures to make the extracted small target features more prominent, such as the common use of FPN network, adding CABM structure, and using transformer structure.

[0043] 2. Use traditional methods to process the image to be detected (such as infrared images). For example, use median, homomorphic Gaussian and other filtering methods to remove noise, and then combine the covariance of the target, the distribution of infrared radiation energy, etc. to obtain the situation of the target to be detected, thereby improving the detection rate of small targets.

[0044] After analysis and research, the inventors of this case found that the current methods for detecting small targets in images to be detected have the following main problems:

[0045] 1) Time-consuming: Using a more complex deep learning structure can indeed improve the detection rate of small targets, but a more complex feature extraction structure requires more computing power. In the actual project, the use of computing power is limited, so this method is not applicable in many actual projects.

[0046] 2) Traditional methods are not very robust: As we all know, the advantage of traditional algorithms is their fast computing speed, which can produce good detection results for targets in specific scenes. However, once the scene changes, the algorithm generally needs to be adjusted accordingly, or even a new set of algorithms needs to be redeveloped. This is unacceptable for situations where some scenes change frequently. For example, since the appearance of small car targets changes with changes in the road environment, it is unacceptable to detect small targets in images taken by on-board infrared cameras.

[0047] After research, the inventors of this case also found that since there are large targets that are relatively close in a picture (occupying more pixels in the image to be detected) and small targets that are relatively far away (occupying fewer pixels in the image), in the process of target detection based on deep learning models, since the extraction of target features in the picture mainly relies on the target's outline, pixel value and other information to determine whether it belongs to the target to be detected, the more features the target can provide in the image, the easier it is to be detected.

[0048] Based on this, the present invention first uses an image reassembly and splicing method to combine the information in vehicle-mounted infrared images when using a deep learning model for target detection. This makes it easier to detect small targets in the image while not affecting the detection rate of large targets, and the calculation remains unchanged.

[0049] The small target detection method provided by the present invention can be applied to any scenario requiring target detection, for example, pedestrian or vehicle detection scenarios.

[0050] After continuous research, the inventors of this case finally provided a small target detection method. The basic concept of this solution is: obtaining an image to be detected and a blank image; determining a full target area image and a small target area image in the image to be detected; wherein the full target area image and the small target area image are images located in a first area and a second area respectively; the first area is the area with the highest probability of all targets appearing; the second area is the area with the highest probability of small targets appearing; scaling the image to be detected to a preset ratio to obtain a scaled image; obtaining a spliced ​​image based on the blank image, the full target area image, the small target area image, and the scaled image; inputting the spliced ​​image into a pre-trained detection model to obtain output parameters;

[0051] In one implementation, the output parameters of the detection model may include one or more of a target category, a target category confidence, a target location, and a target location confidence; wherein: the target category is used to characterize the category to which the target belongs; the target category confidence is used to characterize the degree of confidence that the target belongs to the target category; the target location is used to characterize the location of the target in the image to be detected; and the target location confidence is used to characterize the degree of confidence that the target is at the target location. Furthermore, based on the target category, target category confidence, target location, and target location confidence, the area of ​​the small target in the image to be detected can be determined, thereby completing small target detection. It is understood that the output result of the detection model may be correct or incorrect. The target category confidence can be used to characterize the degree of confidence of the target category output by the model. For example, whether the target category confidence is greater than a preset value can be used to determine whether the target category is predicted correctly. Similarly, the target location confidence can be used to characterize the degree of confidence of the target location output by the model. For example, whether the target location confidence is greater than a preset value can be used to determine whether the target location is predicted correctly.

[0052] By applying the embodiment of the present invention, based on the spliced ​​image and adopting the detection model, small target areas in the image to be detected are detected, thereby improving the small target detection rate and realizing the detection of small targets.

[0053] The small target detection method provided by the present application can be applied to electronic devices with data processing capabilities. The electronic device can be a server on the network side or a terminal used on the user side, such as a PC, a notebook, a smart phone, etc. The server on the network side or the terminal used on the user side can process vehicle images or other types of images of targets to be detected according to the small target detection method provided by the present application. In addition, the functional software for implementing the small target detection method provided by the embodiment of the present invention can be a dedicated software with information processing capabilities, or it can be a plug-in in software with information processing capabilities. The small target detection method provided by the present application is introduced below through the following embodiments.

[0054] See also Figure 1 , which shows a flow chart of a small target detection method provided by an embodiment of the present application, which may include:

[0055] S101, obtaining an image to be detected and a blank image;

[0056] The image to be detected can be an on-board infrared image, an image captured by an on-board infrared camera, or an image captured by another type of camera, and the present invention does not limit this. The image to be detected may contain targets and small targets. The targets can be objects that meet the preset target type, such as trucks, cars, buses, pedestrians, etc.; small targets can be targets that occupy fewer pixels in the image to be detected, for example, targets with fewer than 15 pixels in the image to be detected. Of course, in other implementations, targets with fewer than other pixel values ​​can also be targets, and the present invention does not limit this.

[0057] The blank image may be an image without any content; the sizes of the blank image and the image to be detected may be the same or different, which is not limited in the present invention.

[0058] S102, determining a full target area image and a small target area image in the image to be detected;

[0059] The small target area image is covered by the full target area image.

[0060] It is understood that the full target area image can be the area in the image to be detected where all targets appear most frequently, while the small target area image can be the area in the image to be detected where small targets appear most frequently. Small targets are targets with a relatively small pixel ratio in the target, so the target contains small targets, and the small target area image is covered by the full target area image.

[0061] For example, the full target area image and the small target area image are images located in the first area and the second area respectively; the first area is the area with the highest probability of all targets appearing; the second area is the area with the highest probability of small targets appearing;

[0062] The first area may include the second area, that is, the second area may be included in the first area; for example, Figure 2 As shown in the figure, assuming that the central area B of the image to be detected is the area with the highest probability of all targets appearing, and the central area C of area B is the area with the highest probability of small targets appearing; the image of area B can be considered as the full target area image; the image of area C is the small target area image.

[0063] S103, scaling the image to be detected to a preset size to obtain a scaled image; and obtaining a spliced ​​image based on the blank image, the full target area image, the small target area image, and the scaled image;

[0064] The preset size can be set according to needs. For example, it can be one-quarter the size of the image to be detected, or it can be other sizes. The present invention does not limit this.

[0065] It can be understood that the scaled image is an image that is proportionally reduced from the image to be detected (i.e., the original image). After reduction, the features of the small target will be further weakened, so that the overall summary of the image can be globally reflected.

[0066] S104: Input the spliced ​​image into the detection model to obtain output parameters; and based on the output parameters, determine the area of ​​the small target in the image to be detected.

[0067] The present invention does not limit the specific stitching method. The size of the stitched image and the image to be detected can be the same or different. If the size of the stitched image and the image to be detected are different, the stitched image can be cropped, expanded, or scaled after being obtained so that the stitched image input into the detection model has the same size as the image to be detected.

[0068] The detection model can be a deep learning model, such as CNN, RNN (LSTM), transformer, VAE, GAN, etc. The present invention does not limit the specific model structure. Any model that can achieve target detection can be used as the detection model of this application.

[0069] The target category refers to the category to which the target belongs, such as public transportation, pedestrians, cars, etc. The target category confidence is used to reflect the probability that the detection model believes that the target category prediction is correct; the target position refers to the position or area box of the target in the image to be detected. The target position confidence is used to reflect the probability that the detection model believes that the target position prediction is correct.

[0070] For example, whether the target category prediction is correct can be determined based on whether the target category confidence is greater than a preset category threshold, and whether the target category prediction is correct can be determined based on whether the target position confidence is greater than a preset position threshold; when it is determined that both the position and category predictions are correct, the area of ​​the small target in the image to be detected can be determined.

[0071] As can be seen, the present invention proposes a method for detecting small targets using stitched images. By reducing the image to be detected and stitching it with an image of the full target area and an image of the small target area, the features of the small target area are enhanced, as the first area is the area with the highest probability of all targets appearing, while the second area is the area with the highest probability of small targets appearing. This improves the small target detection rate. Furthermore, since no additional image features are added, this method improves the small target detection rate with essentially no increase in the original deep learning detection computation, demonstrating its effectiveness in practical applications.

[0072] In one implementation, the full target area image and the small target area image are obtained by dividing the image to be detected based on a pre-set first fixed area and a pre-set second fixed area, respectively; the first fixed area is an area where the frequency of occurrence of all targets exceeds a first threshold; the second fixed area is an area where the frequency of occurrence of small targets exceeds a second threshold.

[0073] It can be understood that when predetermining the first fixed area and the second fixed area, the frequencies of all targets (including small targets and non-small targets) appearing in the area and the frequencies of small targets appearing in the area are taken into consideration respectively. If a small target a and a non-small target B appear in a certain area, the area will be counted twice when the target appearance frequency is calculated as the initial first fixed area; the area will only be counted once when the target appearance frequency is calculated as the initial second fixed area. Finally, among all the initial first fixed areas whose frequencies have been calculated, the area whose frequency exceeds the first threshold is selected as the final first fixed area; among all the initial second fixed areas whose frequencies have been calculated, the area whose frequency exceeds the second threshold is selected as the final second fixed area. Alternatively, it can be that among all the initial first fixed areas whose frequencies have been calculated, the area with the largest frequency is selected as the final first fixed area; and among all the initial second fixed areas whose frequencies have been calculated, the area with the largest frequency is selected as the final second fixed area.

[0074] The first and second thresholds can be pre-set based on specific needs and are not limited in this disclosure. For example, they can be set between 80-95% and 85-95%, respectively. The first and second thresholds can be the same; a higher threshold increases the area requirement and improves detection accuracy to a certain extent. Alternatively, they can be different. To improve the accuracy of small target detection, the first threshold can be smaller than the second threshold. This disclosure does not limit their specific values ​​or their relationship.

[0075] In one implementation method, the original training data (such as the sample image training set data used to train the detection model) can be statistically analyzed. After statistically analyzing the areas where each target appears in the original training data, the target area with the highest probability can be calculated as the first fixed area; after statistically analyzing the areas where each small target appears in the original training data, the small target area with the highest probability can be calculated as the second fixed area.

[0076] After determining the first fixed area and the second fixed area, the first area and the second fixed area can be directly divided into corresponding positions in the image to be detected according to the positions where the first fixed area and the second fixed area appear in the training image, thereby obtaining the first area and the second area in the image to be detected, and obtaining the full target area image and the small target area image.

[0077] In one implementation, obtaining a spliced ​​image based on the blank image, the full target area image, the small target area image, and the zoomed image includes:

[0078] The full target area image, the small target area image and the zoomed image are arranged in a preset manner to cover the blank image to obtain a spliced ​​image.

[0079] The preset arrangement can be set in advance according to needs, and the present invention does not limit this.

[0080] For example, in one implementation, the full target area image may cover the lower half of the blank image; and the small target area image and the zoomed image may cover the left and right halves of the upper half of the blank image to obtain a spliced ​​image.

[0081] For example, the full target area image can be used to cover the lower half of the blank image; and the small target area image and the zoomed image can be used to cover the left half and right half of the upper half of the blank image, respectively; or, the small target area image and the zoomed image can be used to cover the right half and left half of the upper half of the blank image, respectively, to obtain a spliced ​​image.

[0082] Of course, in other implementations, the small target area image and the zoomed image may be overlaid on the lower half of the blank image, and the full target area image may be overlaid on the upper half of the blank image to obtain a spliced ​​image. The present invention does not limit the specific splicing method.

[0083] For example, the specific implementation process can be:

[0084] 1) Obtain a blank image A, which has the same size as the original image to be detected.

[0085] 2) Obtain the full target area image (e.g. Figure 2 The image in area B is used to cover the lower half of the blank image A with the image in area B;

[0086] 3) Get the image of the small target area (for example: Figure 2 The image in area C shown in the figure) can be subjected to histogram equalization and gamma correction in sequence, and the image size can be expanded to 1 / 4 of the original image area, so that the small target can be further highlighted, and then the image of the small target area can be pasted on the upper right corner of A.

[0087] 4) The image to be detected (i.e. the original image) is resized to 1 / 4 of the original image size to obtain a scaled image (which can be called image D), and image D is pasted on the upper left corner of A to obtain a stitched image of the same size as the original image (e.g. Figure 3 ), and perform small target detection based on the re-stitched image.

[0088] By applying the detection method based on image stitching in the embodiment of the present invention, a specific area is selected, the area is image processed, the area is partially enlarged, and the area is stitched onto the original image to perform detection in this way, thereby improving the small target detection rate.

[0089] In one implementation, determining the area of ​​the small target in the image to be detected based on the target category, the target category confidence, the target position, and the target position confidence includes:

[0090] If the area corresponding to the target position is the area where the small target area image is located, then determine whether the confidence of the target position is greater than the preset first position confidence; if it is, determine that the area where the target position is located is the initial small target area; if the target category belongs to the small target category, determine whether the target category confidence is greater than the preset first category confidence; if it is, determine that the target category is a small target category; restore the initial small target area to the corresponding area of ​​the image to be detected, and obtain the area of ​​the small target corresponding to the small target category in the image to be detected.

[0091] By feeding the spliced ​​image into a detection network (such as a neural network) for target detection, the detection results output by the detection model may include target category, target category confidence, target position, and target position confidence; for each target position, the target position may first be mapped to the area in which it is located, such as the first area, the second area, or other areas; if it is the second area, it indicates that the target position may be in a small target area, and further, it may be screened according to the preset first position confidence to further determine whether it is in a small target area; if it is in the first area, it indicates that the target position may be in a target area, and it may be screened according to the preset third position confidence to further determine whether it is in a target area. The preset first position confidence may be higher or lower than the preset third position confidence, and different degrees of screening are performed with different position confidences to achieve refined partition detection. The specific values ​​of the preset first position confidence and the preset third position confidence can be set in advance according to demand, and the present invention does not limit this.

[0092] After determining the area where the target position is located, the target category corresponding to the target position can be determined; first, if the area where the target position is located is the area where the small target area image is located, determine whether the target category output by the model for the target position is predicted correctly. If the target category confidence of the target category is greater than the preset first category confidence, it can be determined that the target category prediction is correct; if it is not greater than, it can be considered that the model prediction is inaccurate and the target category of the target position cannot be determined; if the area where the target position is located is the area where the full target area image is located or other areas, it can be determined whether the target category confidence of the target category is greater than the preset second category confidence; if it is greater, it is determined that the target category prediction is correct; if it is not greater than, it can be considered that the model prediction is inaccurate and the target category of the target position cannot be determined. The preset first category confidence and the preset second category confidence can be different to achieve refined detection of small target categories. The specific values ​​of the preset first category confidence and the preset second category confidence can be set in advance according to needs, and the present invention does not limit this.

[0093] It can be understood that the preset first position confidence is used to characterize the credibility threshold for determining that the area where the target position is located is a small target area; the preset second position confidence is used to characterize the credibility threshold for determining that the area where the target position is located is a preset false detection area; when the target position confidence is greater than the preset first position confidence, it can be considered that the target position is in the small target area; if the target position confidence is greater than the preset second position confidence, it can be considered that the target position is in the false detection area.

[0094] Similarly, the preset first category confidence is used to characterize the confidence threshold for determining whether the target category prediction of the target (i.e., small target) in the initial small target area is correct; the preset second category confidence is used to characterize the confidence threshold for determining whether the target category prediction of the target (i.e., non-small target) in the non-initial small target area is correct; when the target category confidence is greater than the preset first category confidence, it can be considered that the small target category judgment is correct; if the target category confidence is greater than the preset second category confidence, it can be considered that the target category prediction is correct.

[0095] By applying the embodiment of the present invention, after a small target area is detected, a category threshold is set separately to confirm the detection result, thereby further improving the accuracy of small target detection.

[0096] In one implementation, the method further includes:

[0097] If the area corresponding to the target position is a preset false detection area, determine whether the target position confidence is greater than the preset second position confidence; if greater, determine that the area where the target position is located is a false detection area, and remove the false detection area and the target category.

[0098] The preset false detection area can be a pre-demarcated area prone to false detection. A sample image test set can be fed into the detection model in advance, and the output of the detection model can be statistically analyzed to determine the area prone to false detection. This area can then be used as the preset false detection area.

[0099] If the area corresponding to the target position is a preset false detection area, it can be screened according to the preset second position confidence. If the target position confidence is greater than the preset second position confidence, the area where the target position is located is determined to be a false detection area, and the false detection area and the target category are removed; if it is not greater than, it is considered that the model prediction may be incorrect, and it is considered that the target position is not in the false detection area.

[0100] Exemplarily, the preset false detection area may include multiple areas. For example, the first area, the second area, and the zoomed image may correspond to preset false detection areas respectively. Each preset false detection area may be confirmed by:

[0101] The first step is to count the targets whose pixels are lower than the preset value according to the number of target pixels, as small targets, and obtain the coordinates of the center point of the small target box;

[0102] The second step is to use statistical analysis to find the area where small targets appear most frequently, which generally does not exceed 1 / 4 of the original image area.

[0103] The third step is to mark the area with the highest frequency of small targets to obtain the second area;

[0104] The fourth step is to repeat the above three steps to obtain the area where all targets appear as the first area, which generally does not exceed 1 / 2 of the original image area.

[0105] Step 5: The original image is reduced to obtain a scaled image;

[0106] Step 6: Project the preset false detection area on the first area and the second area, and zoom the position in the image to obtain the false detection area E. C , E B , E D , as the first area, the second area, and the preset false detection areas corresponding to the zoomed image respectively.

[0107] By cropping out some preset false detection areas where targets are unlikely to appear, and shrinking the original image and re-stitching it onto the detection image, we can determine as many valid detection areas as possible without missing any targets.

[0108] In one implementation, if there are multiple small goals, the method further includes:

[0109] After obtaining the area of ​​each small target in the image to be detected, the intersection-and-union ratio of each small target area is calculated; the small target areas whose intersection-and-union ratio is greater than a preset intersection-and-union ratio threshold are weightedly fused to obtain a fusion area; and the target category corresponding to the target category with the maximum confidence in the fusion area is used as the fusion target category of the fusion area, and the fusion area is used as the area of ​​the small target corresponding to the fusion target category in the image to be detected.

[0110] The position of the initial small target area detected is restored. According to the IOU of the detected target on the original image being greater than the preset intersection-union ratio threshold, it can be considered that the same target is detected in multiple areas. In order to avoid repeated calculation of the target area frame, the final target frame can be obtained by weighting. For example, since the scaling factors of the areas B>C>D where the targets are detected are different, the weights are based on C>B>D, that is, the weight of the small target detected in the second area, the weight of the small target detected in the first area, and the weight of the small target detected in the zoomed image area decrease in sequence. The preset intersection-union ratio threshold can be set in advance according to demand, and the present invention does not limit this.

[0111] It is understandable that when the same target is detected in multiple detection areas at the same time, a weighted fusion method is used to obtain the final detection result, and the detection target category with the highest confidence is selected, thereby improving the accuracy of small target detection.

[0112] In one implementation, the detection model is pre-trained in the following manner:

[0113] Obtaining a sample image training set, and marking the target category and target position of the sample image training set; wherein, targets with a pixel number lower than a preset pixel number are marked as small targets;

[0114] The initial detection model is trained using the labeled sample image training set to obtain a trained detection model.

[0115] For example, the sample image training set may be vehicle-mounted infrared images; and the model training process may include:

[0116] 1) Vehicle-mounted infrared target image acquisition

[0117] An infrared camera is mounted on a car, and then images are collected while driving on various roads.

[0118] 2) Training set annotation

[0119] Targets are marked on the captured images, including their location and category. Vehicles and pedestrians smaller than 15 pixels are considered small targets, while targets larger than 15 pixels are considered normal targets. Of course, other methods can be used to define small targets in other implementations, and this invention is not limited to this.

[0120] 3) Training of the detection model. Use a sample image training set consisting of labeled images to train the initial detection model to obtain a trained detection model. The present invention does not limit the training algorithm and training process, and can be set according to needs during actual application. During the training process, a single captured image is input into the detection model, and the model outputs the category, target position, target category confidence, and target position confidence of the target in the image. According to the difference with the label, the model is continuously adjusted to obtain the model that meets the training convergence conditions, thereby completing the model training.

[0121] In one implementation, the method further includes:

[0122] A sample image test set is obtained, and the trained detection model is tested using the sample image test set to obtain a small target detection threshold, a potential false detection area, and a false detection removal confidence level.

[0123] After obtaining the original captured images, 70% of them can be used as a sample image training set, and the remaining 30% as a sample image validation set. The sample image test set can be used to test the detection model. Based on the output of the detection model, the number of cases that meet the correct prediction is counted. The threshold for predicting the position of small targets, the area prone to false detection, and the threshold for removing false detections are used as the target detection threshold, potential false detection area, and confidence level for removing false detections, respectively. In the subsequent target detection process, these three factors can be used to remove false detection areas and further determine small target areas, thereby improving the small target detection rate.

[0124] In one implementation, determining the area of ​​the small target in the image to be detected based on the output parameter includes:

[0125] If the area corresponding to the target position is the area where the small target area image is located, then determine whether the confidence of the target position is greater than the small target detection threshold; if so, determine that the area where the target position is located is the initial small target area;

[0126] If the area corresponding to the target position is the potential false detection area, then determine whether the target position confidence is greater than the false detection removal confidence; if so, determine that the area where the target position is located is a false detection area, and remove the target position and the target category;

[0127] After determining that the area where the target position is located is the initial small target area, determine whether the target category confidence is greater than the preset category confidence; if it is greater, determine that the target category prediction is correct; restore the initial small target area to the corresponding area of ​​the image to be detected, and obtain the area of ​​the small target corresponding to the target category in the image to be detected.

[0128] By applying the embodiments of the present invention, small target detection is achieved by determining the small target detection threshold, potential false detection area and false detection removal confidence based on the model, thereby improving the model detection rate.

[0129] An embodiment of the present application further provides a small target detection device. The small target detection device provided in the embodiment of the present application is described below. The small target detection device described below and the small target detection method described above can be referenced to each other.

[0130] See also Figure 4 , shows a schematic structural diagram of a small target detection device provided in an embodiment of the present application, which may include: a first acquisition unit 201, a determination unit 202, a second acquisition unit 203 and a detection unit 204.

[0131] A first acquisition unit, configured to acquire an image to be detected and a blank image;

[0132] a determining unit, configured to determine a full target area image and a small target area image in the image to be detected; wherein the small target area image is covered by the full target area image;

[0133] a second acquisition unit, configured to scale the image to be detected to a preset size to obtain a scaled image; and to obtain a spliced ​​image based on the blank image, the full target area image, the small target area image, and the scaled image;

[0134] The detection unit is configured to input the spliced ​​image into a detection model to obtain output parameters; and based on the output parameters, determine the area of ​​the small target in the image to be detected.

[0135] By applying the embodiments of the present invention, by reducing the image to be detected and splicing it with the full target area image and the small target area image, the features of the small target area are enhanced, thereby improving the small target detection rate. Furthermore, since no additional image features are added, this method improves the small target detection rate while substantially increasing the existing deep learning detection computation, demonstrating promising results in practical applications.

[0136] In one possible implementation, the full target area image and the small target second area image are obtained by dividing the image to be detected based on a pre-set first fixed area and a second fixed area; the first fixed area is an area where the frequency of occurrence of full targets exceeds a first threshold; the second fixed area is an area where the frequency of small targets exceeds a second threshold.

[0137] In a possible implementation, the second acquisition unit obtains a spliced ​​image based on the blank image, the full target area image, the small target area image, and the zoomed image, specifically by:

[0138] The full target area image, the small target area image and the zoomed image are arranged in a preset manner to cover the blank image to obtain a spliced ​​image.

[0139] In a possible implementation, the spliced ​​image is input into a detection model to obtain output parameters, where the output parameters include one or more of target category, target category confidence, target position, and target position confidence;

[0140] Among them: the target category is used to characterize the category to which the target belongs; the target category confidence is used to characterize the degree of credibility that the target belongs to the target category; the target position is used to characterize the position of the target in the image to be detected; the target position confidence is used to characterize the degree of credibility that the target is at the target position.

[0141] In one possible implementation, the detection unit determines the area of ​​the small target in the image to be detected based on the target category, the target category confidence, the target position, and the target position confidence, specifically as follows:

[0142] If the area corresponding to the target position is the area where the small target area image is located, then determine whether the confidence of the target position is greater than the preset first position confidence; if it is, determine that the area where the target position is located is the initial small target area; determine whether the target category confidence is greater than the preset first category confidence; if it is, determine that the target category prediction is correct; restore the initial small target area to the corresponding area of ​​the image to be detected, and obtain the area of ​​the small target corresponding to the target category in the image to be detected.

[0143] In a possible implementation, the apparatus further includes:

[0144] The second determination unit is used to determine whether the target position confidence is greater than a preset second position confidence if the area corresponding to the target position is a preset false detection area; if it is greater, determine that the area where the target position is located is a false detection area, and remove the target position and the target category.

[0145] In a possible implementation, if there are multiple small targets, the device further includes a fusion unit configured to:

[0146] After obtaining the area of ​​each small target in the image to be detected, the intersection-and-union ratio of each small target area is calculated; the small target areas whose intersection-and-union ratio is greater than a preset intersection-and-union ratio threshold are weightedly fused to obtain a fusion area; and the target category corresponding to the target category with the maximum confidence in the fusion area is used as the fusion target category of the fusion area, and the fusion area is used as the area of ​​the small target corresponding to the fusion target category in the image to be detected.

[0147] In one possible implementation, the apparatus further includes a verification unit configured to:

[0148] A sample image test set is obtained, and the trained detection model is tested using the sample image test set to obtain a small target detection threshold, a potential false detection area, and a false detection removal confidence level.

[0149] In one possible implementation, the detection unit determines the area of ​​the small target in the image to be detected based on the target category, the target category confidence, the target position, and the target position confidence, specifically as follows:

[0150] If the area corresponding to the target position is the area where the small target area image is located, then determine whether the confidence of the target position is greater than the small target detection threshold; if so, determine that the area where the target position is located is the initial small target area;

[0151] If the area corresponding to the target position is the potential false detection area, then determine whether the target position confidence is greater than the false detection removal confidence; if so, determine that the area where the target position is located is a false detection area, and remove the target position and the target category;

[0152] After determining that the area where the target position is located is the initial small target area, determine whether the target category confidence is greater than the preset category confidence; if it is greater, determine that the target category prediction is correct; restore the initial small target area to the corresponding area of ​​the image to be detected, and obtain the area of ​​the small target corresponding to the target category in the image to be detected.

[0153] The present application also provides an electronic device. Figure 5 , shows a schematic structural diagram of the electronic device, which may include: at least one processor 301, at least one communication interface 302, at least one memory 303 and at least one communication bus 304;

[0154] In the embodiment of the present application, the number of the processor 301, the communication interface 302, the memory 303, and the communication bus 304 is at least one, and the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304;

[0155] The processor 301 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0156] The memory 303 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0157] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0158] A small target detection method, the method comprising:

[0159] Obtain the image to be detected and the blank image;

[0160] Determining a full target area image and a small target area image in the image to be detected; wherein the small target area image is covered by the full target area image;

[0161] Scaling the image to be detected to a preset size to obtain a scaled image; and obtaining a spliced ​​image based on the blank image, the full target area image, the small target area image, and the scaled image;

[0162] The spliced ​​image is input into a detection model to obtain output parameters; and based on the output parameters, an area of ​​the small target in the image to be detected is determined.

[0163] By applying the embodiments of the present invention, by reducing the image to be detected and splicing it with the full target area image and the small target area image, the features of the small target area are enhanced, thereby improving the small target detection rate. Furthermore, since no additional image features are added, this method improves the small target detection rate while substantially increasing the existing deep learning detection computation, demonstrating promising results in practical applications.

[0164] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0165] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0166] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0167] The above description of the disclosed embodiments will enable one of ordinary skill in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one of ordinary skill in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is to be construed in the widest possible manner consistent with the principles and novel features disclosed herein.

Claims

1. A small target detection method, characterized in that: The method comprises: Obtain the image to be detected and the blank image; Determining a full target area image and a small target area image in the image to be detected; wherein the small target area image is covered by the full target area image; Scaling the image to be detected to a preset size to obtain a scaled image; and obtaining a spliced ​​image based on the blank image, the full target area image, the small target area image, and the scaled image; Inputting the spliced ​​image into a detection model to obtain output parameters; and determining the area of ​​the small target in the image to be detected based on the output parameters; The full target area image and the small target area image are respectively obtained by dividing the image to be detected based on a preset first fixed area and a second fixed area; the first fixed area is an area where the frequency of all targets exceeding a first threshold; the second fixed area is an area where the frequency of small targets exceeding a second threshold; The step of obtaining a spliced ​​image based on the blank image, the full target area image, the small target area image, and the zoomed image includes: The full target area image, the small target area image and the zoomed image are arranged in a preset manner to cover the blank image to obtain a spliced ​​image.

2. The method according to claim 1, characterized in that The spliced ​​image is input into the detection model to obtain output parameters, wherein the output parameters include one or more of target category, target category confidence, target position, and target position confidence; Among them: the target category is used to characterize the category to which the target belongs; the target category confidence is used to characterize the degree of credibility that the target belongs to the target category; the target position is used to characterize the position of the target in the image to be detected; the target position confidence is used to characterize the degree of credibility that the target is at the target position.

3. The method according to claim 2, characterized in that Determining the area of ​​the small target in the image to be detected based on the output parameters includes: If the area corresponding to the target position is the area where the small target area image is located, then determine whether the target position confidence is greater than the preset first position confidence; if it is, determine that the area where the target position is located is the initial small target area; determine whether the target category confidence is greater than the preset first category confidence; if it is, determine that the target category prediction is correct; restore the initial small target area to the corresponding area of ​​the image to be detected, and obtain the area of ​​the small target corresponding to the target category in the image to be detected.

4. The method according to claim 3, characterized in that The method further comprises: If the area corresponding to the target position is a preset false detection area, determine whether the target position confidence is greater than the preset second position confidence; if greater, determine that the area where the target position is located is a false detection area, and remove the target position and the target category.

5. The method according to claim 3, characterized in that If there are multiple small targets, the method further includes: After obtaining the area of ​​each small target in the image to be detected, the intersection-and-union ratio of each small target area is calculated; the small target areas whose intersection-and-union ratio is greater than a preset intersection-and-union ratio threshold are weightedly fused to obtain a fusion area; and the target category corresponding to the target category with the maximum confidence in the fusion area is used as the fusion target category of the fusion area, and the fusion area is used as the area of ​​the small target corresponding to the fusion target category in the image to be detected.

6. The method according to claim 1, characterized in that The testing method of the detection model includes: A sample image test set is obtained, and the detection model is tested using the sample image test set to obtain a small target detection threshold, a potential false detection area, and a false detection removal confidence level.

7. The method according to claim 2, characterized in that Determining a region of the small target in the image to be detected based on the output parameters includes: If the area corresponding to the target position is the area where the small target area image is located, then determining whether the target position confidence is greater than the small target detection threshold; if so, determining that the area where the target position is located is the initial small target area; If the area corresponding to the target position is a potential false detection area, determine whether the target position confidence is greater than the false detection removal confidence; if so, determine that the area where the target position is located is a false detection area, and remove the target position and the target category; After determining that the area where the target position is located is the initial small target area, determine whether the target category confidence is greater than the preset first category confidence; if greater, determine that the target category prediction is correct; restore the initial small target area to the corresponding area of ​​the image to be detected, and obtain the area of ​​the small target corresponding to the target category in the image to be detected.

8. A small target detection device, characterized in that: The device comprises: A first acquisition unit, configured to acquire an image to be detected and a blank image; a determining unit, configured to determine a full target area image and a small target area image in the image to be detected; wherein the small target area image is covered by the full target area image; a second acquisition unit, configured to scale the image to be detected to a preset size to obtain a scaled image; and to obtain a spliced ​​image based on the blank image, the full target area image, the small target area image, and the scaled image; A detection unit, configured to input the spliced ​​image into a detection model to obtain output parameters; and determine an area of ​​the small target in the image to be detected based on the output parameters; The full target area image and the small target area image are respectively obtained by dividing the image to be detected based on a preset first fixed area and a second fixed area; the first fixed area is an area where the frequency of all targets exceeding a first threshold; the second fixed area is an area where the frequency of small targets exceeding a second threshold; The step of obtaining a spliced ​​image based on the blank image, the full target area image, the small target area image, and the zoomed image includes: The full target area image, the small target area image and the zoomed image are arranged in a preset manner to cover the blank image to obtain a spliced ​​image.

Images