A target detection method and apparatus based on an improved ATSS

Abstract

This invention provides a target detection method based on an improved ATSS (Attention Detection System). The method includes: collecting and labeling sample images to obtain labeled sample images; preprocessing the labeled sample images to obtain training data; inputting the training data into a target detection network based on the improved ATSS for repeated training to obtain a trained target detection model; and inputting the image to be detected into the target detection model to obtain and output the target detection result. Compared with existing technologies, this invention can accurately and quickly detect targets and solves the problems of missed detection of occluded targets and unstable detection bounding boxes.

Description

Technical Field

[0001] This invention relates to image processing, smart parking, and target detection, and particularly to a target detection method and apparatus based on an improved ATSS. Background Technology

[0002] Object detection refers to the ability of computer and software systems to locate and identify each object in an image / scene. It has been widely used in fields such as face detection, vehicle detection, pedestrian counting, network images, security systems, and autonomous vehicles.

[0003] In practical applications, target detection requires real-time performance while also achieving high detection rates and accuracy for small targets and occluded targets in videos.

[0004] Existing target detection methods based on ATSS (Adaptive Training Sample Selection) suffer from problems such as uneven sample allocation and overlapping allocation of sample points for different targets during the model training and sample allocation phase. This leads to missed detection of occluded targets and unstable detection boxes.

[0005] In summary, there is an urgent need to propose an improved ATSS-based target detection method to solve the above problems. Summary of the Invention

[0006] In view of this, the main objective of the present invention is to provide a target detection method based on an improved ATSS, which can achieve both accuracy and speed.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a target detection method based on an improved ATSS is provided, the method comprising:

[0008] The first step is to collect sample images and label them to obtain labeled sample images;

[0009] The second step is to perform image preprocessing on the labeled sample images to obtain training data;

[0010] The third step is to input the training data into the object detection network based on the improved ATSS and train it repeatedly to obtain a trained object detection model.

[0011] The fourth step is to input the image to be detected into the target detection model, obtain the target detection results, and output them.

[0012] Furthermore, the image preprocessing includes one or more of the following: brightness and color adjustment, image scaling, image flipping, and random cropping of occluded targets. The preprocessed images of the labeled sample images are used as training data.

[0013] The random cropping of the occluded target includes the following steps:

[0014] The random cropping step randomly selects the upper left and lower right coordinates on the labeled sample image as the starting and ending points of the cropped small image, and updates the target coordinate values ​​of the cropped small image at the same time.

[0015] The cropping rate determination steps involve calculating the area A of the target in the labeled sample image and the area CA of the target in the cropped image; then calculating the cropping rate. If the cutting rate R c If the cropping rate exceeds the threshold, cropping the small image will be invalid;

[0016] The occlusion rate determination process involves calculating the target's area (CA) and the overlap area (COA) with other targets for each target on the cropped small image, thereby calculating the target's occlusion rate. If the target's occlusion rate R o If the image size exceeds the occlusion threshold, cropping the smaller image will be ineffective.

[0017] In the random cropping output step, if cropping the small image is invalid, output the labeled sample image as training data; otherwise, consider the cropping of the small image valid and output the cropped small image as training data.

[0018] Furthermore, the third step includes:

[0019] The feature pyramid acquisition step involves inputting training data, which is then processed through a backbone network and a feature fusion network based on an improved ATSS object detection network to obtain three feature layers with different dimensions.

[0020] The steps for obtaining classification and localization prediction results are as follows: after the feature layer passes through the classification and localization network based on the improved ATSS target detection network, the classification prediction results and localization prediction results of the anchor points of the corresponding feature layer are obtained respectively.

[0021] The steps for obtaining the true classification and localization results are as follows: For each feature layer, during the training process of the target detection network based on the improved ATSS, positive and negative sample points are laid on the feature layer for the target to be detected; after the sample points are laid, true classification and localization box information is assigned to each anchor point.

[0022] The steps for calculating the network loss of the detection network are as follows: For each feature layer, calculate the difference between the feature layer classification prediction result, localization prediction result and the true classification result, and the true localization result to obtain the network loss of the detection network.

[0023] The backpropagation step involves backpropagating the network loss to continuously optimize the object detection network based on the improved ATSS until the improved ATSS object detection network converges, resulting in a well-trained object detection model.

[0024] The three different feature layers are: the low-level feature layer pyramid, the middle-level feature layer pyramid, and the high-level feature layer pyramid.

[0025] Furthermore, the step of obtaining the true classification and localization results includes:

[0026] The pre-selected positive sample point acquisition step involves selecting the n sample points closest to the target center point on each feature layer as pre-selected positive sample points. The set of pre-selected sample points for all feature layers is labeled as U0.

[0027] The positive sample threshold calculation step is to calculate the average IoU of all pre-selected positive sample points in the pre-selected positive sample point set U0, and use it as the positive sample threshold PTR.

[0028] The steps for obtaining positive and negative sample points are as follows: For the pre-selected positive sample points in the pre-selected positive sample point set U0, the pre-selected positive sample points whose IoU value with the target is greater than the positive sample threshold PTR are defined as positive sample points; if the number of positive sample points of the target is less than n, then the n sample points with the highest IoU value are selected from all the pre-selected positive sample points and defined as positive sample points; all sample points other than positive sample points in all anchor points of the feature layer are defined as negative sample points.

[0029] In the multi-target positive sample point allocation step, if positive sample points are simultaneously allocated to two or more targets on the same feature layer or different feature layers, the corresponding positive sample points are allocated to the targets that are closer to the target center point.

[0030] The steps for real classification and bounding box acquisition are as follows: real bounding boxes are obtained based on the anchor boxes of anchor points; when the anchor point is detected as a positive sample point, the real bounding box corresponding to the anchor point is classified as the target class; when the anchor point is detected as a negative sample point, the real bounding box corresponding to the anchor point is classified as the background class.

[0031] Wherein, the IoU of the preselected positive samples is the intersection-union ratio between the anchor boxes of the preselected positive samples and the target boxes; n represents the threshold number of sample points.

[0032] Furthermore, the step of obtaining the true classification and localization box includes obtaining the true localization box based on the anchor box of the anchor point as follows: assuming the horizontal and vertical coordinates of the anchor point are x and y respectively, the width and height of the anchor box are w and h respectively, and the target box deviation values ​​predicted by the improved ATSS target detection network are dx, dy, dw, and dh respectively, then the width of the true localization box corresponding to the anchor point is pw = w * exp(dw), the height is ph = h * exp(dh), the horizontal coordinate of the center point of the true localization box is px = x + dx * pw, the vertical coordinate is py = y + dy * pw, the horizontal coordinate of the upper left vertex of the true localization box is px1 = px - 0.5 * pw, the vertical coordinate is py1 = py - 0.5 * ph, and the horizontal coordinate of the lower right vertex of the true localization box is px2 = px + 0.5 * pw, the vertical coordinate is py2 = py + 0.5 * ph.

[0033] According to another aspect of the present invention, a target detection device based on an improved ATSS is provided, the device comprising:

[0034] The sample image collection and annotation module is used to collect sample images and annotate them to obtain annotated sample images;

[0035] The image preprocessing module is used to preprocess the labeled sample images to obtain training data;

[0036] The target detection network training module based on the improved ATSS is used to input training data into the target detection network based on the improved ATSS, and perform repeated training to obtain a trained target detection model.

[0037] The target detection model detection module is used to input the image to be detected into the target detection model, obtain the target detection result, and output it.

[0038] Furthermore, the image preprocessing module includes one or more of the following modules: a brightness and color adjustment module, an image scaling module, an image flipping module, and a random cropping module for occluded targets. The preprocessed images of the labeled sample images are used as training data.

[0039] Furthermore, the occlusion target randomization module includes:

[0040] The random cropping module is used to randomly select the top-left and bottom-right coordinates on the labeled sample image as the starting and ending points for cropping the small image, while updating the target coordinate values ​​of the cropped small image.

[0041] The cropping rate determination module is used to calculate the area A of the target in the labeled sample image and the area CA of the target in the cropped image, respectively; and to calculate the cropping rate. If the cutting rate R cIf the cropping rate exceeds the threshold, cropping the small image will be invalid;

[0042] The occlusion rate determination module is used to calculate the occlusion rate of each target on the cropped small image by calculating the target's area (CA) and the overlap area (COA) between the target and other targets. If the target's occlusion rate R o If the image size exceeds the occlusion threshold, cropping the smaller image will be ineffective.

[0043] The random cropping output module is used to output labeled sample images as training data if cropping is invalid; otherwise, it considers cropping to be valid and outputs the cropped images as training data.

[0044] Furthermore, the target detection network training module based on the improved ATSS includes:

[0045] The feature pyramid acquisition module is used to input training data. After passing through the backbone network and feature fusion network of the object detection network based on the improved ATSS, three feature layers with different dimensions are obtained.

[0046] The classification and localization prediction result acquisition module is used to obtain the classification and localization prediction results of the anchor points of the corresponding feature layers after the feature layer passes through the classification and localization network based on the improved ATSS target detection network.

[0047] The classification and localization result acquisition module is used to lay positive and negative sample points for the target detection in each feature layer during the training process of the target detection network based on the improved ATSS. After laying the sample points, the real classification and localization box information is assigned to each anchor point.

[0048] The network loss calculation module of the detection network is used to calculate the difference between the feature layer classification prediction result and localization prediction result and the true classification result and true localization result for each feature layer, so as to obtain the network loss of the detection network.

[0049] The backpropagation module is used to backpropagate the network loss and continuously optimize the object detection network based on the improved ATSS until the object detection network based on the improved ATSS converges, thus obtaining a trained object detection model.

[0050] The three different feature layers are: the low-level feature layer pyramid, the middle-level feature layer pyramid, and the high-level feature layer pyramid.

[0051] Furthermore, the module for obtaining the true classification and localization results includes:

[0052] The pre-selected positive sample point acquisition module is used to select the n sample points that are closest to the target center point on each feature layer as pre-selected positive sample points. The set of pre-selected sample points of all feature layers is labeled as U0.

[0053] The positive sample threshold calculation module is used to calculate the average IoU of all pre-selected positive sample points in the pre-selected positive sample point set U0, which is used as the positive sample threshold PTR.

[0054] The positive and negative sample point acquisition module is used to define positive sample points in the pre-selected positive sample point set U0 that have an IoU value with the target greater than the positive sample threshold PTR as positive sample points; if the number of positive sample points of the target is less than n, then the n sample points with the highest IoU value are selected from all the pre-selected positive sample points and defined as positive sample points; and all anchor points of the feature layer except for positive sample points are defined as negative sample points.

[0055] The multi-target positive sample point allocation module is used to allocate the corresponding positive sample point to the target that is closer to the target center point if positive sample points are allocated to two or more targets at the same feature layer or different feature layers.

[0056] The module for real classification and bounding box acquisition is used to obtain real bounding boxes based on the anchor points. When the anchor point is detected as a positive sample point, the real bounding box corresponding to the anchor point is classified as the target class. When the anchor point is detected as a negative sample point, the real bounding box corresponding to the anchor point is classified as the background class.

[0057] Wherein, the IoU of the preselected positive samples is the intersection-union ratio between the anchor boxes of the preselected positive samples and the target boxes; n represents the threshold number of sample points.

[0058] Compared with existing target detection technologies, the target detection method based on improved ATSS of the present invention has the advantage of high detection rate and accuracy for small targets and occluded targets in videos. Attached Figure Description

[0059] Figure 1 A flowchart of a target detection method based on an improved ATSS according to the present invention is shown.

[0060] Figure 2 A framework diagram of a target detection device based on an improved ATSS according to the present invention is shown. Detailed Implementation

[0061] To enable those skilled in the art to further understand the structure, features and other objectives of the present invention, the following detailed description is provided in conjunction with the accompanying preferred embodiments. The preferred embodiments described are only for illustrating the technical solutions of the present invention and are not intended to limit the present invention.

[0062] Figure 1 A flowchart of a target detection method based on an improved ATSS according to the present invention is provided. Figure 1 As shown, a target detection method based on an improved ATSS according to the present invention includes:

[0063] The first step, S1, is to collect sample images and label them to obtain labeled sample images.

[0064] The second step, S2, involves preprocessing the labeled sample images to obtain training data.

[0065] The third step, S3, involves inputting the training data into the object detection network based on the improved ATSS and repeatedly training it to obtain a well-trained object detection model.

[0066] In the fourth step S4, the image to be detected is input into the target detection model to obtain the target detection result and output it.

[0067] The first step S1 includes: collecting images containing targets under different scenes and lighting conditions as sample images; and labeling the targets in the sample images, including their location and information, to obtain labeled sample images. The targets include one or more of the following: vehicles, license plates, pedestrians, faces, animals, etc.

[0068] Furthermore, the image preprocessing includes one or more of the following: brightness and color adjustment, image scaling, image flipping, and random cropping of occluded targets. The preprocessed images of the labeled sample images are used as training data.

[0069] For example, the image preprocessing employs four image processing methods: brightness and color adjustment, image scaling, image flipping, and random cropping of occluded targets. The training data includes: labeled sample images after brightness and color adjustment, labeled sample images after image scaling, labeled sample images after image flipping, and labeled sample images after random cropping of occluded targets.

[0070] The brightness and color adjustment, image scaling, and image flipping can be achieved using existing image processing software.

[0071] The random cropping of the occluded target includes the following steps:

[0072] In the random cropping step S21, the upper left and lower right coordinates are randomly selected on the labeled sample image as the starting and ending points of the cropped small image, and the target coordinate values ​​of the cropped small image are updated at the same time.

[0073] In step S22, the cropping rate is determined by calculating the area A of the target in the labeled sample image and the area CA of the target in the cropped image; the cropping rate is then calculated. If the cutting rate R c If the cropping rate exceeds the threshold, cropping the small image will be invalid;

[0074] In step S23, for each target in the cropped small image, the occlusion rate is calculated by determining the target's area CA and the overlap area COA between the target and other targets. If the target's occlusion rate R o If the image size exceeds the occlusion threshold, cropping the smaller image will be ineffective.

[0075] In the random cropping output step S24, if the cropped image is invalid, the labeled sample image is output as training data; otherwise, the cropped image is considered valid and is output as training data.

[0076] In the second step S2, random cropping of occluded targets involves obtaining training data after random cropping of occluded targets, thereby ensuring that the targets in the training data maintain appropriate size and occlusion rate.

[0077] The cropping rate determination step S22 is to ensure that the cropped target retains basic semantic information, so as to avoid false detection of the target due to the small size of the cropped image.

[0078] The occlusion rate determination step S23 is to ensure that no target is extremely occluded after cropping, thereby avoiding false detection of the target due to extreme occlusion of the target in the cropped image.

[0079] Further, the cropping rate threshold ranges from 0.1 to 0.6, and the occlusion threshold ranges from 0.5 to 0.9. For example, the cropping rate threshold is selected as 0.5, and the occlusion threshold is selected as 0.8.

[0080] The object detection network based on ATSS (Adaptive Training Sample Selection) can automatically select appropriate anchor boxes as positive samples based on the relevant statistical characteristics of the target, significantly improving the performance of the object detection model without incurring additional computational costs and parameters. For example, see the reference "Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection. S Zhang, C Chi, Y Yao, Z Lei, SZ Li. 2020 IEEE / CVFConference on Computer Vision and Pattern Recognition (CVPR)".

[0081] Furthermore, the third step S3 includes:

[0082] In the feature pyramid acquisition step S31, the input training data is processed through the backbone network and feature fusion network of the object detection network based on the improved ATSS to obtain three feature layers with different dimensions.

[0083] In step S32, after the feature layer passes through the classification and localization network based on the improved ATSS target detection network, the classification prediction results and localization prediction results of the anchor points of the corresponding feature layer are obtained respectively.

[0084] In step S33, for each feature layer, during the training process of the target detection network based on the improved ATSS, positive and negative sample points are laid on the feature layer for the target to be detected; after the sample points are laid, the true classification and localization box information is assigned to each anchor point.

[0085] In step S34 of the detection network loss calculation, for each feature layer, the difference between the feature layer classification prediction result, localization prediction result and the true classification result, and the true localization result is calculated to obtain the network loss of the detection network.

[0086] In backpropagation step S35, the target detection network based on the improved ATSS is continuously optimized based on backpropagation according to the network loss until the target detection network based on the improved ATSS converges, and a well-trained target detection model is obtained.

[0087] The three feature layers with different dimensions are: a low-level feature layer pyramid, a mid-level feature layer pyramid, and a high-level feature layer pyramid. Each feature layer has different semantic information and receptive field; the low-level feature layer is responsible for detecting small targets, the mid-level feature layer is responsible for detecting medium targets, and the high-level feature layer is responsible for detecting large targets.

[0088] Furthermore, step S33, which involves obtaining the true classification and localization results, includes:

[0089] In step S331 of obtaining pre-selected positive sample points, on each feature layer, the n sample points that are closest to the target center point are selected as pre-selected positive sample points, and the set of pre-selected sample points of all feature layers is labeled as U0;

[0090] Positive sample threshold calculation step S332: Calculate the average IoU of all pre-selected positive sample points in the pre-selected positive sample point set U0, and use it as the positive sample threshold PTR;

[0091] In step S333, for the pre-selected positive sample points in the pre-selected positive sample point set U0, the pre-selected positive sample points whose IoU value with the target is greater than the positive sample threshold PTR are defined as positive sample points; if the number of positive sample points of the target is less than n, then the n sample points with the highest IoU value are selected from all the pre-selected positive sample points and defined as positive sample points; all sample points other than positive sample points in all anchor points of the feature layer are defined as negative sample points.

[0092] In the multi-target positive sample point allocation step S334, if positive sample points are simultaneously allocated to two or more targets on the same feature layer or different feature layers, the corresponding positive sample points are allocated to the targets that are closer to the target center point.

[0093] Step S335 involves obtaining the true classification and localization bounding box based on the anchor bounding box of the anchor point. When the anchor point is detected as a positive sample point, the true localization bounding box corresponding to the anchor point is classified as the target class. When the anchor point is detected as a negative sample point, the true localization bounding box corresponding to the anchor point is classified as the background class.

[0094] Wherein, the IoU of the preselected positive samples is the intersection-union ratio between the anchor boxes of the preselected positive samples and the target boxes; n represents the threshold number of sample points.

[0095] Furthermore, the value range of the sample point number threshold n is 9 to 16. For example, the sample point number threshold n is selected as 12.

[0096] The difference between the target detection network based on the improved ATSS and the target detection network based on ATSS is: (1) the average IoU of the pre-selected positive sample points is used as the positive sample threshold PTR; (2) a judgment mechanism for the number of positive sample points is added in the positive sample point and negative sample point acquisition step S333; (3) the multi-target positive sample point allocation step S334 is added.

[0097] The positive sample threshold (PTR) of the ATSS-based target detection network is defined as the sum of the mean IoU and the root mean square error. Since the root mean square error only describes the dispersion of the sample IoU from the mean, if samples below the mean have a larger deviation, setting this threshold too high will cause all pre-selected positive sample points to be discarded, resulting in missed target detections. To address this issue, the positive sample threshold calculation step S332 uses the mean IoU of the pre-selected positive samples as the positive sample threshold (PTR).

[0098] The difference between step S333, which obtains positive and negative sample points, and the target detection network based on ATSS is that a mechanism for judging the number of positive sample points is added. When the number of positive sample points of the target is less than n, the n anchor points with the highest IOU value are automatically selected as positive sample points to improve the target detection rate.

[0099] In ATSS-based object detection networks, only positive sample allocation for the same anchor point in the same feature layer assigned to multiple targets is handled, but this situation on different feature layers is not addressed. Therefore, if two anchor points in different feature layers at the same location in the image are simultaneously assigned to different targets, the bounding box annotation information for that anchor point will be unclear, resulting in overlapping bounding boxes of mutually occluding targets during actual detection. The multi-target positive sample point allocation step S334 solves this problem.

[0100] The real classification and localization box acquisition step S335, which involves obtaining the real localization box based on the anchor box of the anchor point, includes: assuming the horizontal and vertical coordinates of the anchor point are x and y respectively, the width and height of the anchor box are w and h respectively, and the target box deviation values ​​predicted by the improved ATSS target detection network are dx, dy, dw, and dh respectively, then the width of the real localization box corresponding to the anchor point is pw = w * exp(dw), the height is ph = h * exp(dh), the horizontal coordinate of the center point of the real localization box is px = x + dx * pw, the vertical coordinate is py = y + dy * pw, the horizontal coordinate of the upper left vertex of the real localization box is px1 = px - 0.5 * pw, the vertical coordinate is py1 = py - 0.5 * ph, and the horizontal and vertical coordinates of the lower right vertex of the real localization box are px2 = px + 0.5 * pw and py2 = py + 0.5 * ph respectively.

[0101] Figure 2A framework diagram of a target detection device based on an improved ATSS according to the present invention is given. Figure 2 As shown, a target detection device based on an improved ATSS according to the present invention includes:

[0102] Sample image collection and annotation module 1 is used to collect sample images and annotate them to obtain annotated sample images;

[0103] Image preprocessing module 2 is used to preprocess the labeled sample images to obtain training data;

[0104] The object detection network training module 3 based on the improved ATSS is used to input training data into the object detection network based on the improved ATSS, and perform repeated training to obtain a trained object detection model.

[0105] The target detection module 4 is used to input the image to be detected into the target detection model, obtain the target detection result, and output it.

[0106] The sample image collection and annotation module 1 includes: collecting images containing targets under different scenes and lighting conditions as sample images; and annotating the targets in the sample images, including their location and information, to obtain an annotated sample image. The targets include one or more of the following: vehicles, license plates, pedestrians, faces, animals, etc.

[0107] Furthermore, the image preprocessing module 2 includes one or more of the following modules: a brightness and color adjustment module, an image scaling module, an image flipping module, and a random cropping module for occluded targets. The preprocessed images of the labeled sample images are used as training data.

[0108] Furthermore, the occlusion target randomization module includes:

[0109] The random cropping module 21 is used to randomly select the upper left and lower right coordinates on the labeled sample image as the starting and ending points of the cropped small image, and at the same time update the target coordinate values ​​of the cropped small image.

[0110] The cropping rate determination module 22 is used to calculate the area A of the target in the labeled sample image and the area CA of the target in the cropped image, respectively; and to calculate the cropping rate. If the cutting rate R c If the cropping rate exceeds the threshold, cropping the small image will be invalid;

[0111] The occlusion rate determination module 23 is used to calculate the occlusion rate of each target on the cropped small image by calculating the target's area CA and the overlap area COA between the target and other targets. If the target's occlusion rate R oIf the image size exceeds the occlusion threshold, cropping the smaller image will be ineffective.

[0112] The random cropping output module 24 is used to output the labeled sample image as training data if the cropped small image is invalid, otherwise it is considered that the cropped small image is valid and the cropped small image is output as training data.

[0113] Further, the cropping rate threshold ranges from 0.1 to 0.6, and the occlusion threshold ranges from 0.5 to 0.9. For example, the cropping rate threshold is selected as 0.5, and the occlusion threshold is selected as 0.8.

[0114] The object detection network based on ATSS (Adaptive Training Sample Selection) can automatically select appropriate anchor boxes as positive samples based on the relevant statistical characteristics of the target, significantly improving the performance of the object detection model without incurring additional computational costs and parameters. For example, see the reference "Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection. S Zhang, C Chi, Y Yao, Z Lei, SZ Li. 2020 IEEE / CVFConference on Computer Vision and Pattern Recognition (CVPR)".

[0115] Furthermore, the target detection network training module 3 based on the improved ATSS includes:

[0116] The feature pyramid acquisition module 31 is used to input training data. After passing through the backbone network and feature fusion network of the object detection network based on the improved ATSS, three feature layers with different dimensions are obtained.

[0117] The classification and localization prediction result acquisition module 32 is used to obtain the classification prediction result and localization prediction result of the anchor point of the corresponding feature layer after the feature layer passes through the classification and localization network based on the improved ATSS target detection network.

[0118] The classification and localization real result acquisition module 33 is used to lay positive and negative sample points for the detection target on the feature layer during the training process of the target detection network based on the improved ATSS target detection network for each feature layer; after laying the sample points, the real classification and localization box information is assigned to each anchor point.

[0119] The network loss calculation module 34 of the detection network is used to calculate the difference between the feature layer classification prediction result and localization prediction result and the true classification result and true localization result for each feature layer, so as to obtain the network loss of the detection network.

[0120] The backpropagation module 35 is used to backpropagate according to the network loss and continuously optimize the target detection network based on the improved ATSS until the target detection network based on the improved ATSS converges, thus obtaining a trained target detection model.

[0121] The three feature layers with different dimensions are: a low-level feature layer pyramid, a mid-level feature layer pyramid, and a high-level feature layer pyramid. Each feature layer has different semantic information and receptive field; the low-level feature layer is responsible for detecting small targets, the mid-level feature layer is responsible for detecting medium targets, and the high-level feature layer is responsible for detecting large targets.

[0122] Furthermore, the classification and localization real result acquisition module 33 includes:

[0123] The pre-selected positive sample point acquisition module 331 is used to select the n sample points that are closest to the target center point on each feature layer as pre-selected positive sample points. The set of pre-selected sample points of all feature layers is labeled as U0.

[0124] The positive sample threshold calculation module 332 is used to calculate the average IoU of all pre-selected positive sample points in the pre-selected positive sample point set U0, which is used as the positive sample threshold PTR.

[0125] The positive and negative sample point acquisition module 333 is used to define the pre-selected positive sample points in the pre-selected positive sample point set U0 as positive sample points if the IoU value with the target is greater than the positive sample threshold PTR; if the number of positive sample points of the target is less than n, then the n sample points with the highest IoU value are selected from all the pre-selected positive sample points and defined as positive sample points; and all sample points other than positive sample points in all anchor points of the feature layer are defined as negative sample points.

[0126] The multi-target positive sample point allocation module 334 is used to allocate the corresponding positive sample point to the target that is closer to the target center point if positive sample points are allocated to two or more targets at the same feature layer or different feature layers.

[0127] The real classification and localization box acquisition module 335 is used to acquire the real localization box based on the anchor box of the anchor point; when the anchor point is detected as a positive sample point, the real localization box corresponding to the anchor point is classified as the target class; when the anchor point is detected as a negative sample point, the real localization box corresponding to the anchor point is classified as the background class.

[0128] Wherein, the IoU of the preselected positive samples is the intersection-union ratio between the anchor boxes of the preselected positive samples and the target boxes; n represents the threshold number of sample points.

[0129] Furthermore, the value range of the sample point number threshold n is 9 to 16.

[0130] The difference between the target detection network based on the improved ATSS and the target detection network based on ATSS is that: (1) the average IoU of the pre-selected positive sample points is used as the positive sample threshold PTR; (2) a judgment mechanism for the number of positive sample points is added to the positive sample point and negative sample point acquisition module 333; and (3) the multi-target positive sample point allocation module 334 is added.

[0131] Furthermore, the actual classification and localization box acquisition module 335 acquires the actual localization box based on the anchor box of the anchor point as follows: assuming the horizontal and vertical coordinates of the anchor point are x and y respectively, the width and height of the anchor box are w and h respectively, and the target box deviation values ​​predicted by the improved ATSS target detection network are dx, dy, dw, and dh respectively, then the width of the actual localization box corresponding to the anchor point is pw = w * exp(dw), the height is ph = h * exp(dh), the horizontal coordinate of the center point of the actual localization box is px = x + dx * pw, the vertical coordinate is py = y + dy * pw, the horizontal coordinate of the upper left vertex of the actual localization box is px1 = px - 0.5 * pw, the vertical coordinate is py1 = py - 0.5 * ph, and the horizontal coordinate of the lower right vertex of the actual localization box is px2 = px + 0.5 * pw, the vertical coordinate is py2 = py + 0.5 * ph.

[0132] Compared with existing target detection technologies, the target detection method based on improved ATSS of the present invention has the advantage of high detection rate and accuracy for small targets and occluded targets in videos.

[0133] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It should be understood that the present invention is not limited to the implementations described herein. These implementations are described to help those skilled in the art practice the present invention. Any person skilled in the art can easily make further improvements and modifications without departing from the spirit and scope of the present invention. Therefore, the present invention is only limited by the content and scope of the claims, and is intended to cover all alternatives and equivalents included within the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A target detection method based on an improved ATSS, characterized in that, The method includes: The first step is to collect sample images and label them to obtain labeled sample images; The second step is to perform image preprocessing on the labeled sample images to obtain training data; The third step is to input the training data into the object detection network based on the improved ATSS and train it repeatedly to obtain a trained object detection model. The fourth step is to input the image to be detected into the target detection model, obtain the target detection results, and output them. Furthermore, the third step includes: The feature pyramid acquisition step involves inputting training data, which is then processed through a backbone network and a feature fusion network based on an improved ATSS object detection network to obtain three feature layers with different dimensions. The steps for obtaining classification and localization prediction results are as follows: after the feature layer passes through the classification and localization network based on the improved ATSS target detection network, the classification prediction results and localization prediction results of the anchor points of the corresponding feature layer are obtained respectively. The steps for obtaining the true classification and localization results are as follows: For each feature layer, during the training process of the target detection network based on the improved ATSS, positive and negative sample points are laid on the feature layer for the target to be detected; after the sample points are laid, true classification and localization box information is assigned to each anchor point. The steps for calculating the network loss of the detection network are as follows: For each feature layer, calculate the difference between the feature layer classification prediction result, localization prediction result and the true classification result, and the true localization result to obtain the network loss of the detection network. The backpropagation step involves backpropagating the network loss to continuously optimize the target detection network based on the improved ATSS until the target detection network based on the improved ATSS converges, resulting in a well-trained target detection model. The three different feature layers are: low-level feature layer pyramid, mid-level feature layer pyramid, and high-level feature layer pyramid. The steps for obtaining the true classification and localization results include: The pre-selected positive sample point acquisition step involves selecting the n sample points closest to the target center point on each feature layer as pre-selected positive sample points. The set of pre-selected sample points for all feature layers is labeled as U0. The positive sample threshold calculation step is to calculate the average IoU of all pre-selected positive sample points in the pre-selected positive sample point set U0, and use it as the positive sample threshold PTR. The steps for obtaining positive and negative sample points are as follows: For the pre-selected positive sample points in the pre-selected positive sample point set U0, the pre-selected positive sample points whose IoU value with the target is greater than the positive sample threshold PTR are defined as positive sample points; if the number of positive sample points of the target is less than n, then the n sample points with the highest IoU value are selected from all the pre-selected positive sample points and defined as positive sample points; all sample points other than positive sample points in all anchor points of the feature layer are defined as negative sample points. In the multi-target positive sample point allocation step, if positive sample points are simultaneously allocated to two or more targets on the same feature layer or different feature layers, the corresponding positive sample points are allocated to the targets that are closer to the target center point. The steps for real classification and bounding box acquisition are as follows: real bounding boxes are obtained based on the anchor boxes of anchor points; when the anchor point is detected as a positive sample point, the real bounding box corresponding to the anchor point is classified as the target class; when the anchor point is detected as a negative sample point, the real bounding box corresponding to the anchor point is classified as the background class. Wherein, the IoU of the preselected positive samples is the intersection-union ratio between the anchor boxes of the preselected positive samples and the target boxes; n represents the threshold number of sample points.

2. The method as described in claim 1, characterized in that, The image preprocessing includes one or more of the following: brightness and color adjustment, image scaling, image flipping, and random cropping of occluded targets; the preprocessed images of the labeled sample images are used as training data. Furthermore, the random cropping of the occluded target includes the following steps: The random cropping step randomly selects the upper left and lower right coordinates on the labeled sample image as the starting and ending points of the cropped small image, and updates the target coordinate values ​​of the cropped small image at the same time. The cropping rate determination steps involve calculating the area A of the target in the labeled sample image and the area CA of the target in the cropped image; then calculating the cropping rate. If the cutting rate R c If the cropping rate exceeds the threshold, cropping the small image will be invalid; The occlusion rate determination process involves calculating the target's area (CA) and the overlap area (COA) with other targets for each target on the cropped small image, thereby calculating the target's occlusion rate. If the target's occlusion rate R o If the image size exceeds the occlusion threshold, cropping the smaller image will be ineffective. In the random cropping output step, if cropping the small image is invalid, output the labeled sample image as training data; otherwise, consider the cropping of the small image valid and output the cropped small image as training data.

3. The method as described in claim 2, characterized in that, The cropping rate threshold ranges from 0.1 to 0.6, and the occlusion threshold ranges from 0.5 to 0.

9.

4. The method as described in claim 1, characterized in that, The threshold value n for the number of sample points ranges from 9 to 16.

5. A target detection device based on an improved ATSS, characterized in that, The device includes: The sample image collection and annotation module is used to collect sample images and annotate them to obtain annotated sample images; The image preprocessing module is used to preprocess the labeled sample images to obtain training data; The target detection network training module based on the improved ATSS is used to input training data into the target detection network based on the improved ATSS, and perform repeated training to obtain a trained target detection model. The target detection model detection module is used to input the image to be detected into the target detection model, obtain the target detection result, and output it. Furthermore, the target detection network training module based on the improved ATSS includes: The feature pyramid acquisition module is used to input training data. After passing through the backbone network and feature fusion network of the object detection network based on the improved ATSS, three feature layers with different dimensions are obtained. The classification and localization prediction result acquisition module is used to obtain the classification and localization prediction results of the anchor points of the corresponding feature layers after the feature layer passes through the classification and localization network based on the improved ATSS target detection network. The classification and localization result acquisition module is used to lay positive and negative sample points for the target detection in each feature layer during the training process of the target detection network based on the improved ATSS. After laying the sample points, the real classification and localization box information is assigned to each anchor point. The network loss calculation module of the detection network is used to calculate the difference between the feature layer classification prediction result and localization prediction result and the true classification result and true localization result for each feature layer, so as to obtain the network loss of the detection network. The backpropagation module is used to backpropagate the network loss and continuously optimize the target detection network based on the improved ATSS until the target detection network based on the improved ATSS converges, thus obtaining a trained target detection model. The three different feature layers are: low-level feature layer pyramid, mid-level feature layer pyramid, and high-level feature layer pyramid. The module for obtaining the true classification and localization results includes: The pre-selected positive sample point acquisition module is used to select the n sample points that are closest to the target center point on each feature layer as pre-selected positive sample points. The set of pre-selected sample points of all feature layers is labeled as U0. The positive sample threshold calculation module is used to calculate the average IoU of all pre-selected positive sample points in the pre-selected positive sample point set U0, which is used as the positive sample threshold PTR. The positive and negative sample point acquisition module is used to define positive sample points in the pre-selected positive sample point set U0 that have an IoU value with the target greater than the positive sample threshold PTR as positive sample points; if the number of positive sample points of the target is less than n, then the n sample points with the highest IoU value are selected from all the pre-selected positive sample points and defined as positive sample points; and all anchor points of the feature layer except for positive sample points are defined as negative sample points. The multi-target positive sample point allocation module is used to allocate the corresponding positive sample point to the target that is closer to the target center point if positive sample points are allocated to two or more targets at the same feature layer or different feature layers. The real classification and localization box acquisition module is used to obtain the real localization box based on the anchor box of the anchor point; when the anchor point is detected as a positive sample point, the real localization box corresponding to the anchor point is classified as the target class; when the anchor point is detected as a negative sample point, the real localization box corresponding to the anchor point is classified as the background class. Wherein, the IoU of the preselected positive samples is the intersection-union ratio between the anchor boxes of the preselected positive samples and the target boxes; n represents the threshold number of sample points.

6. The apparatus as claimed in claim 5, characterized in that, The image preprocessing module includes one or more of the following modules: brightness and color adjustment module, image scaling module, image flipping module, and random cropping module for occluded targets; the preprocessed images of the labeled sample images are used as training data. Furthermore, the occlusion target randomization module includes: The random cropping module is used to randomly select the top-left and bottom-right coordinates on the labeled sample image as the starting and ending points for cropping the small image, while updating the target coordinate values ​​of the cropped small image. The cropping rate determination module is used to calculate the area A of the target in the labeled sample image and the area CA of the target in the cropped image, respectively; and to calculate the cropping rate. If the cutting rate R c If the cropping rate exceeds the threshold, cropping the small image will be invalid; The occlusion rate determination module is used to calculate the occlusion rate of each target on the cropped small image by calculating the target's area (CA) and the overlap area (COA) between the target and other targets. If the target's occlusion rate R o If the image size exceeds the occlusion threshold, cropping the smaller image will be ineffective. The random cropping output module is used to output labeled sample images as training data if cropping is invalid; otherwise, it considers cropping to be valid and outputs the cropped images as training data.

Images