An anchor self-learning based target detection method

Abstract

The application discloses an anchor self-learning-based target detection method, and belongs to the technical field of target detection. The target detection network structure is achieved by design, and is composed of a main network and two fully connected branches. On the basis of the faster-rcnn algorithm, an anchor scale and aspect ratio generation network, namely an anchor scale and aspect ratio learner, is additionally designed. The learner inputs four-channel data into the designed network to generate optimized anchor scales and aspect ratios. The basic scales and aspect ratios of anchor boxes are adaptively learned by using the network to meet the design requirements of different data sets. The second part is an RPN network, which is trained by using the RPN. The network assigns the anchor scales and aspect ratios generated by the designed anchor box learner to an anchor generator of the RPN to generate optimal anchor sizes to better detect targets. The application enables the algorithm to have data adaptive capability to realize full coverage of targets, and has good accuracy and robustness.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and more specifically to a target detection method based on anchor self-learning. Background Technology

[0002] Object detection has been widely applied in various fields, including autonomous driving, disaster search and rescue, remote sensing object detection, medical lesion detection, and industrial inspection. It is an important branch of image processing and computer vision, a core component of intelligent monitoring systems, and a fundamental algorithm in the broader field of identity recognition, playing a crucial role in subsequent tasks such as face recognition, gait recognition, crowd counting, and instance segmentation. This method reduces the consumption of human capital through computer vision, which has significant practical implications. In recent years, due to the widespread use of deep learning, object detection algorithms have developed rapidly, but problems such as low reliability and low detection accuracy still exist in some areas. This invention discloses a deep learning object detection method based on an anchor mechanism, which not only enables end-to-end object detection but also improves the accuracy and robustness of object detection.

[0003] Deep learning is a new research direction in the field of machine learning, its depth being reflected in the multiple transformations of features. A commonly used deep learning model is a multi-layered neural network. Each layer of the neural network performs a non-linear mapping of the input. Through the stacking of multiple non-linear mappings, abstract features are calculated in the deep neural network to aid in classification. It contains multi-layered network models, and the learning process maps from low to high layers to new feature spaces, exhibiting hierarchical and distributed abstraction characteristics. This allows it to fit complex non-linear functions and handle higher-dimensional non-linear input data. Deep learning network structures have three key characteristics: unsupervised learning is used to pre-train each layer; only one layer is trained at a time using unsupervised learning, and the training result is used as the input to the next layer; and a top-down supervised algorithm is used to adjust all layers. In recent years, deep learning has attracted widespread attention in fields such as computer vision, object detection, natural language processing, and speech recognition, and has yielded many achievements.

[0004] Due to the widespread application of deep learning, object detection algorithms have developed rapidly. There are various classification methods for object detection algorithms, primarily based on the presence or absence of anchors: anchor-based algorithms and anchor-free algorithms. Anchor-free algorithms do not use anchor mechanisms, relying instead on other methods for object detection. However, both keypoint-based and pixel-wise prediction methods are essentially dense prediction methods. The resulting solution space makes simple anchor-free methods prone to generating too many negative samples predicted as positive by the model, leading to high recall but low precision. Therefore, anchor-free object detection algorithms tend to generate a large number of negative samples, resulting in generally low detection accuracy, unstable results, and extensive post-processing. Compared to anchor-based algorithms, anchor-free algorithms, due to keypoint prediction, suffer from imbalanced positive and negative samples, inaccurate localization, and generally lower detection accuracy. Anchor-based object detection methods utilize anchor mechanisms to generate a large number of candidate regions, then classify and regress these regions. This method can be divided into single-stage and two-stage object detection algorithms. Numerous studies have shown that two-stage algorithms generally achieve higher accuracy than one-stage algorithms. Two-stage algorithms primarily utilize anchor mechanisms to generate a large number of candidate regions, which are highly likely to contain the target. Deep learning is then used to perform regression localization and classification operations on these candidate regions, ultimately obtaining the target's location and type. In contrast, one-stage algorithms do not filter the generated candidate regions; instead, they directly perform regression and localization operations on all candidate regions of the feature map. This approach results in a large number of negative samples, leading to lower accuracy and reliability.

[0005] This shows that most object detection algorithms, whether one-stage or two-stage, are anchor-based, using anchor mechanisms to generate candidate regions for subsequent detection. Therefore, the improvement in detection rate of anchor-based algorithms is closely related to the selection of candidate regions; the quality of the selected candidate regions is mainly related to the anchor, and the quality of the anchor depends on its basic scale and aspect ratio. Currently, the design of anchors in object detection algorithms mainly relies on human design. Different datasets require different anchor sizes and shapes, and the human-designed anchor scales and aspect ratios require strong experience from the designer. The designed anchor sizes are often highly subjective, and this design is mainly for general targets, often ignoring non-general targets. This leads to detectors primarily targeting general targets while neglecting non-general targets. Some methods directly use basic clustering algorithms to cluster several fixed-size clusters, such as YOLOv2. Clustering algorithms can solve the randomness brought about by human design to a certain extent. However, clustering algorithms are sensitive to outliers and are prone to bias, resulting in inaccurate clustering and failing to achieve optimal anchor scales and aspect ratios.

[0006] In summary, in order to overcome the shortcomings of the above two methods, this patent mainly improves the anchor-based detection algorithm to enhance the accuracy and reliability of target detection. Summary of the Invention

[0007] The main objective of this invention is to disclose a target detection method based on anchor self-learning, addressing the technical problem that anchor-based target detection algorithms lack adaptability to data. The designed method overcomes the limitations of manually designing anchor sizes based on experience or directly clustering anchor sizes using clustering methods, enabling the algorithm to adapt to data and allowing it to be embedded into any target detection algorithm to achieve full coverage of targets, exhibiting good accuracy and robustness.

[0008] To achieve the above objectives and solve the above technical problems, the technical solution of the present invention is as follows:

[0009] An anchor-based self-learning object detection method is implemented by designing an anchor-based object detection network structure.

[0010] The designed object detection network structure consists of two parts: a generative network for predicting anchor scales and aspect ratios, and the RPN network of Faster R-CNN. The specific design process is as follows:

[0011] The generative network that predicts anchor scales and aspect ratios refers to adding a network module to the anchor generation structure of the Faster R-CNN algorithm to learn anchor scales and aspect ratios, in order to learn the optimal shape of the anchor.

[0012] The input data for the generative network that predicts anchor scales and aspect ratios consists of four-channel data composed of RGB and ground-truth mask image data, while the input data for the RPN network of Faster R-CNN consists of three-channel RGB image data.

[0013] To learn the optimal shape of the anchor, the feature map of the last layer of the backbone network is used to predict the anchor scales. The feature map of the last layer of the backbone network outputs a total of 35*1 vector values, which are converted into a 5*7 matrix. The softmax function is used to operate on each row so that the output data is between 0 and 1. Finally, each row of data is multiplied by the designed initial anchor scales. By multiplying by the initial anchor scales, the values ​​of the anchor scales are between 8 and 512. Here, the initial anchor scales are set to (8, 16, 32, 64, 128, 256, 512).

[0014] To learn the optimal anchor shape, the feature map of the last layer of the backbone network is used to predict aspect ratios. First, the feature map of the last layer of the backbone network is processed by a fully connected layer to generate 18*1 vector values, which are then converted into a 3*6 matrix. The softmax function is then applied to each row to make the output data between 0 and 1. Finally, each row of data is multiplied by the designed initial aspect ratios. By multiplying by the initial aspect ratios, the aspect ratios are between 0.1 and 2.5. Here, the initial aspect ratios are set to (0.1, 0.5, 1.0, 1.5, 2.0, 2.5).

[0015] The generation representation of anchor scales and aspect ratios in the designed network module is as follows:

[0016] AS =F1(x1) (1)

[0017] A a =F2(x1) (2)

[0018] Where x1 represents the input data, F1 and F2 represent the designed network functions, and A S Indicates the output anchorscales, A a This indicates the aspect ratios of the output;

[0019] The learned optimal anchor scales and aspect ratios are directly assigned to the anchor module of Faster R-CNN to obtain the optimal anchor size;

[0020] The generated optimal anchors are classified and regressed. The classification is a binary classifier used to determine whether pixel features belong to the foreground or background, and the position offset is predicted through regression. Finally, non-maximum suppression is used to generate the required candidate regions, and the generated candidate regions are further classified and detected. The mmdetection framework is used, and the feature pyramid network is automatically embedded into Faster R-CNN, so the candidate regions are generated not only on the feature map of the last layer, but also on the feature maps of the previous layers. After the anchors are generated, binary classification and regression are performed again to determine the candidate regions, and finally, fine target detection and classification are performed.

[0021] Anchor-based object detection methods include the following steps:

[0022] First, the training dataset is divided into two parts, P1 and P2. P1 can be any 4 / 5 of the original training set, and P2 is the remaining 1 / 5.

[0023] Second, a portion of the data P1 is used to train the anchor learner network designed in this invention.

[0024] Third, input another part of the data P2 into the anchor learning network trained in the previous part to perform inference operations and average the results;

[0025] Fourth, after averaging the final output data from the previous step, the optimal anchor scales and aspect ratios are obtained;

[0026] Fifth, the obtained optimal value is directly assigned to the proposed anchor-based algorithm;

[0027] Sixth, the algorithm based on Faster R-CNN is trained on the entire training set using L1 loss and binary classification loss functions, as shown in the following form:

[0028] Loss = L cls +L reg (3)

[0029] Where L cls Let L be the binary classification loss function. reg This is the L1 loss function.

[0030] The advantages of this invention compared to existing technologies are:

[0031] (1) The self-learning anchor scale and ratio network structure designed in this invention can avoid the need to manually design the relevant parameters of the anchor, allowing the algorithm to automatically learn the anchor size from the data, thereby avoiding the difficulty of designing different anchors for specific tasks and improving the recall and accuracy of the target detection algorithm.

[0032] (2) The method proposed in this invention can adapt to various datasets. Based on the dataset, it can learn the optimal anchor scales and aspect ratios for that dataset, making the learner universal. The method proposed in this invention can be combined with various object detection algorithms, as long as the object detection algorithm utilizes the anchor mechanism.

[0033] (3) The method proposed in this invention can effectively improve the accuracy of small target detection. Attached Figure Description

[0034] Figure 1 Flowchart of the target detection method based on anchor self-learning in this invention;

[0035] Figure 2 This invention presents a schematic diagram of the overall network structure for target detection based on anchor self-learning. Detailed Implementation

[0036] Currently, the design of anchors in object detection algorithms largely relies on human design. Different datasets require different anchor sizes and shapes, and the human-designed basic scales and aspect ratios demand extensive experience from the designer. The resulting anchor sizes are often highly subjective, primarily targeting general objects while neglecting non-general ones. This leads to detectors focusing mainly on general object detection while ignoring non-general object detection. Some methods, such as YOLOv2, directly use basic clustering algorithms to cluster objects into a few fixed sizes. While clustering can mitigate the randomness introduced by human design to some extent, it is sensitive to outliers and prone to bias, resulting in inaccurate clustering and failing to achieve optimal anchor scales and aspect ratios.

[0037] The design concept of this invention is described as follows: To overcome the shortcomings of the two existing design methods, this invention designs a new object detection network structure. This structure is mainly based on an improvement of the Faster R-CNN algorithm and consists of a backbone network and two fully connected branches. The improvement in detection accuracy in the Faster R-CNN algorithm is mainly related to the quality of the candidate regions generated by the Region Proposal Network (RPN). Therefore, to improve the quality of candidate regions, this invention designs an additional anchor scales and aspect ratio generation network, i.e., an anchor scales and aspect ratio learner, based on the Faster R-CNN algorithm. This learner inputs four-channel data into the designed network to generate optimized anchor scales and aspect ratios. By using the network to adaptively learn the basic scale and aspect ratio of the anchor boxes, the design requirements of different datasets can be met. The anchor boxes best suited for the dataset can be obtained without manual design or clustering, improving the recall and performance of the object detection algorithm. The second part is the RPN network, which is used to train the network. The network assigns the anchor scales and aspect ratio generated by the anchor box learner designed in the previous part to the anchor generator of the RPN in order to generate the optimal anchor size, which can better detect the target.

[0038] The present invention will now be explained and described in detail with reference to the accompanying drawings. The implementation process of the present invention is as follows: Figure 1 As shown, an anchor-based self-learning object detection method is implemented by designing an anchor-based object detection network structure, such as... Figure 2 As shown, the specific steps include the following:

[0039] Step 1: Prepare the training set

[0040] The training dataset is divided into two parts, P1 and P2. P1 is any 4 / 5 of the original training set, and P2 is the remaining 1 / 5.

[0041] Step 2: Design an anchor-based object detection network structure and use P1 as input data to train the network.

[0042] The object detection network structure designed in this invention is mainly an improvement on the Faster R-CNN algorithm, which is the earliest classic object detection method to use the anchor mechanism and achieve end-to-end detection. An anchor scales and aspect ratios generation network, i.e., an anchor scales and aspect ratios learner, is added to the anchor generation structure of the Faster R-CNN algorithm. This learner inputs four-channel data (RGB and ground-truthmask image data) into the designed network. Here, this invention uses a ResNet50 network, meaning the four-channel data is input into the ResNet50 network.

[0043] Step 3: Input P2 into the trained ResNet50 network for inference operations and average the results.

[0044] This process involves prediction operations performed on the feature maps of the last layer of the backbone network. Since the initial anchor scales of Faster R-CNN are (32, 64, 128, 256, 512), the network structure designed in this invention learns 5 anchor scales, making the learned anchor scales closer to the object scale of the real data. Therefore, the last layer of the backbone network outputs a total of 35 data points, which are converted into a 5*7 matrix through data processing. Then, the softmax function is used to operate on each row, making the data range between 0 and 1. Finally, each row of data is multiplied by the designed initial anchor scales; the initial anchor scales are set to (8, 16, 32, 64, 128, 256, 512). By multiplying by the initial anchor scales, the range becomes 8 to 512. The reason for designing the initial anchor scales within this range is that the mainstream detection range for object detection algorithms is between 8 and 512. If the pixel size of the target is smaller than 8 pixels, mainstream algorithms have difficulty accurately detecting the target. Similarly, to learn the optimal shape, the feature map of the last layer of the backbone network is used to predict aspect ratios, the process being the same as anchor scale prediction. First, the feature map of the last layer generates 18 data points through a fully connected layer. This data is then preprocessed into a 3x6 matrix, and a softmax function is applied to each row to ensure the data ranges from 0 to 1. Finally, each row is multiplied by the designed initial aspect ratios. Here, the initial aspect ratios are set to (0.1, 0.5, 1.0, 1.5, 2.0, 2.5). This design references the aspect ratios of Faster R-CNN, i.e.,

[0045] (0.5, 1.0, 2.0). After multiplication, the result is between 0.1 and 2.5.

[0046] Step 4: Obtain the optimal values ​​for anchor scales and aspect ratios.

[0047] After averaging the final output data from step three, the optimal anchor scales and aspect ratios can be obtained. The generation of anchor scales and aspect ratios in the designed network structure can be expressed as:

[0048] A S =F1(x1) (1)

[0049] A a =F2(x1) (2)

[0050] Where x1 represents the input data, F1 and F2 represent the designed network functions, and A S Indicates the output anchorscales, A a This indicates the aspect ratios of the output.

[0051] Step 5: Assign values ​​to the anchor-based algorithm

[0052] After learning the optimal anchor scales and aspect ratios, the optimal values ​​obtained in step four are directly assigned to replace the original data. This design structure can meet the design requirements of different datasets, and the anchor size most suitable for the dataset can be obtained without manual design or clustering, thus improving the recall and performance of the object detection algorithm.

[0053] Step Six: Perform training and testing on the Faster R-CNN-based algorithm across the entire training set.

[0054] In this invention, the proposed network structure consists of two parts: a network for predicting anchor scales and aspect ratios, and a Faster R-CNN RPN network. The improved detection accuracy in the Faster R-CNN algorithm is mainly related to the quality of the candidate regions generated by the RPN (Region Proposal Network). The RPN extracts feature maps by sharing a backbone network and uniformly slides a window across the last layer of the feature map to generate nine anchors. This invention first trains the network using the RPN, directly assigning the anchor scales and aspect ratios generated by the anchor learner designed in step two to the RPN's anchor generator to generate optimal anchor sizes for better target detection. Then, the generated anchors are classified and regressed. This classification uses a binary classifier to determine whether an anchor is foreground or background. Position offset prediction is performed through regression. Finally, non-maximum suppression is used to generate the required candidate regions, which are then further classified and detected. The mmdetection framework is adopted, and the Feature Pyramid Network (FPN) is automatically embedded into Faster R-CNN. Therefore, candidate region generation can occur not only on the feature map of the last layer but also on the feature maps of the previous layers. After anchor generation, binary classification and regression are performed to determine candidate regions, and finally, fine-grained target detection and classification are performed. The network input for predicting anchor scales and aspect ratios consists of four-channel data composed of RGB and ground-truth mask image data, while the Faster R-CNN network input consists of three-channel RGB image data. This invention uses the L1 loss function and a binary classification loss function to train the network, which can be expressed as follows:

[0055] Loss = L cls +L reg (3)

[0056] Where L cls Let L be the binary classification loss function. reg This is the L1 loss function.

[0057] In summary, the entire algorithm flowchart is as follows: Figure 1 As shown.

[0058] First, the training dataset is divided into two parts, P1 and P2 (P1 can be any 4 / 5 of the original training set, and P2 is the remaining 1 / 5).

[0059] Second, a portion of the data P1 is used to train the ResNet50 network designed in this invention.

[0060] Third, input another part of the data P2 into the ResNet50 network trained in the previous part for inference operations and average the results;

[0061] Fourth, after averaging the final output data from the previous step, the optimal anchor scales and aspect ratios are obtained;

[0062] Fifth, the obtained optimal value is directly assigned to the proposed anchor-based algorithm;

[0063] Sixth, the algorithm based on Faster R-CNN was trained and tested across the entire training set.

[0064] In summary, this invention designs a network architecture capable of simultaneously learning the scale and shape information of anchors, achieving full coverage of the target, and exhibiting good robustness. The overall network structure of this invention is as follows: Figure 2 As shown.

[0065] Example 1

[0066] Dataset Preparation: This invention primarily conducts validation experiments on three datasets: Visdrone2019, UAVDT, and DOTA. The Visdrone2019 dataset contains 263 video data points, 179,264 image frames, and 10,209 still images. These images were obtained through drone aerial photography, resulting in significant variations in target size within a single image, facilitating the validation of the proposed algorithm's effectiveness. The 10,209 still images represent the target detection task, comprising 6,471 images for the training set, 548 images for the validation set, and 1,610 images for the test set. The DOTA dataset is a satellite remote sensing image dataset with 16 categories. The training set contains 1,411 images, and the validation set contains 458 images. Image resolutions range from 800*800 to 4000*4000, and the dataset predominantly features small targets. The UAVDT dataset contains 24,441 images in the training set and 16,294 images in the validation set. The image resolution is approximately 1080*540. The dataset was obtained from drone platforms at multiple locations in the city, and the labeled object categories include cars, buses, and trucks.

[0067] Implementation Process: For all experiments, this invention uses ResNet50 and FPN as the backbone network. Since this invention uses three different datasets—Visdrone2019 and DOTA—which are remote sensing images with large fields of view and high pixel counts, while targets occupy a smaller proportion of the image, have lower pixel counts, and are smaller in size, image cropping was performed on the remote sensing image data to improve target detection accuracy. Uniform sliding window sampling was used to uniformly sample the images to 640*640 pixels; if the image was less than 640 pixels, zeros were used to pad the image. For each dataset, training was performed for 24 epochs, with an initial learning rate of 0.000001. The learning rate was reduced by 0.1 at epochs 18 and 22, respectively. The network was trained using a batch size of 5 images. The entire network architecture was based on and improved upon using the mmdetection and PyTorch frameworks. All these experiments were conducted on an NVIDIA 3090 graphics card.

[0068] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A target detection method based on anchor self-learning, characterized in that, The target detection method described above is implemented by designing an anchor-based target detection network structure. The designed object detection network structure consists of two parts: two fully connected branches and a backbone network. The two fully connected branches are learner networks used to predict anchor scales and aspect ratios, and the backbone network is the RPN network of Faster R-CNN. The specific design process is as follows: The learner network that predicts anchor scales and aspect ratios refers to adding a network module to the anchor generation structure of the Faster R-CNN algorithm to learn anchor scales and aspect ratios, which is used to automatically learn the optimal shape of the anchor. The learner network that predicts anchor scales and aspect ratios inputs four-channel data consisting of RGB and ground-truth mask image data, while the RPN network of Faster R-CNN inputs three-channel RGB image data. To learn the optimal shape of the anchor, the feature map of the last layer of the backbone network is used to predict the anchor scales. The feature map of the last layer of the backbone network outputs a total of 35*1 vector values, which are converted into a 5*7 matrix. The softmax function is used to operate on each row so that the output data is between 0 and 1. Finally, each row of data is multiplied by the designed initial anchor scales. By multiplying by the initial anchor scales, the values ​​of the anchor scales are between 8 and 512. Here, the initial anchor scales are set to (8, 16, 32, 64, 128, 256, 512). To learn the optimal shape of the anchor, the feature map of the last layer of the backbone network is used to predict aspect ratios. First, the feature map of the last layer of the backbone network is passed through a fully connected layer to generate 18*1 vector values, which are then converted into a 3*6 matrix. The softmax function is then applied to each row to make the output data between 0 and 1. Finally, each row of data is multiplied by the designed initial aspect ratios. By multiplying by the initial aspect ratios, the aspect ratios take values ​​between 0.1 and 2.

5. Here, the initial aspect ratios are set to (0.1, 0.5, 1.0, 1.5, 2.0, 2.5); The generation representation of anchor scales and aspect ratios in the designed network module is as follows: A S =F1(x1) (1)A a =F2(x1) (2) where x1 represents the input data, F1 and F2 represent the designed network functions, A S Indicates the output anchor scales, A a This indicates the aspect ratios of the output; The learned optimal anchor scales and aspect ratios are directly assigned to the anchor module of Faster R-CNN to obtain the optimal anchor size. The generated optimal anchors are classified and regressed. The classification is a binary classifier used to determine whether pixel features belong to the foreground or background, and the position offset is predicted through regression. Finally, the required candidate regions are generated through non-maximum suppression, and the generated candidate regions are further classified and detected. The mmdetection framework is adopted, and the feature pyramid network is automatically embedded into Faster R-CNN, so the candidate regions are generated not only on the feature map of the last layer, but also on the feature maps of the first few layers. After the anchors are generated, binary classification and regression are performed again to determine the candidate regions, and finally, fine target detection and classification are performed. The target detection method based on anchor self-learning includes the following steps: First, the training dataset is divided into two parts, P1 and P2. P1 is any 4 / 5 of the original training set, and P2 is the remaining 1 / 5. Second, a portion of the data P1 is used to train the designed anchor-based learner network; Third, input another part of the data P2 into the anchor-based learner network trained in the previous part to perform inference operations and average the results; Fourth, after averaging the data from the final output of the third step, the optimal anchor scales and aspect ratios are obtained; Fifth, the obtained optimal value is directly assigned to the proposed anchor-based algorithm; Sixth, the algorithm based on Faster R-CNN is trained on the entire training set using L1 loss and binary classification loss functions, as shown in the following form: Loss=L cls +L reg (3) Where L cls Let L be the binary classification loss function. reg This is the L1 loss function.

Images