Anchor-free target detection method and system based on diagonal network

Through diagonal network, the Focal Loss and Fisher criteria are used to sort and pair key points, and the diagonal regression loss is designed, solving the problems of anchor frame design difficulties, sample imbalance and low detection accuracy. , realizes high-precision object detection.

CN114863136BActive Publication Date: 2024-11-19CHINESE PEOPLES LIBERATION ARMY ARMY ARTILLERY & AIR DEFENSE ACAD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210576106.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-25
Publication Date
2024-11-19
Estimated Expiration
2042-05-25

AI Technical Summary

Technical Problem

In the prior art, there are problems such as difficulty in designing anchor frames, sample imbalance and low detection accuracy. Especially in the anchor frame object detection method, the key point pairing is inaccurate and the regression accuracy is low, resulting in high false detection rate and low target detection accuracy.

Method used

Adopting an anchor-free frame object detection method based on diagonal network, extracting depth features through hourglass network, corner point pooling and center pooling, obtaining key point heat maps and embedded feature vectors, and using Focal Loss and Fisher criteria to perform key points Sorting and pairing, designing diagonal regression losses for fine positioning of target prediction boxes, reducing error detection rates and improving detection accuracy.

Benefits of technology

The accuracy and accuracy of target detection are improved, the sample imbalance problem is solved, the fine positioning of the target box is achieved, the error detection rate is reduced, and the accuracy and regression accuracy of key point pairing are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114863136B_ABST
    Figure CN114863136B_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for anchor-free target detection based on a diagonal network, including: using an hourglass network to extract input image depth features and perform pooling operations, thereby obtaining a key point heat map, an embedded feature vector feature map and an offset feature map; using a diagonal network to process the key point heat map, sorting the key points according to confidence, selecting the top k key points of confidence as target key points, and obtaining the key point positions and key point category information of the upper left corner, lower right corner and center point; based on the Fisher criterion, designing an embedded vector learning loss function to perform key point combination pairing training, and further identifying the preliminary pairing results using the center key point; using the distance, length and slope measurement loss of the diagonal center points of the predicted rectangle and the labeled rectangular box to design a diagonal loss for target prediction box regression training. The technical problems of difficult anchor frame design, sample imbalance and low detection accuracy are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method and system for detecting anchor-free objects based on a diagonal network. Background Art

[0002] Object detection is an important research direction in the field of computer vision. Its purpose is to predict the bounding box position and category of each object of interest in an image. Object detection has a wide range of applications in autonomous driving, video surveillance, visual guidance, and air defense warning. At present, deep learning-based object detection mainly includes two categories: anchor-based object detection methods and anchor-free object detection methods.

[0003] In the early stages of research, the anchor-frame-based object detection method received widespread attention and significantly improved the accuracy and recall of object detection. However, it still has two problems: first, the layout of the anchor frame requires setting parameters such as size and aspect ratio, which requires strong prior knowledge inspiration, so it is difficult to design a suitable set of anchor frames; second, in order to ensure detection accuracy and recall, a large number of anchor frames need to be deployed, which leads to a significant imbalance between positive and negative samples.

[0004] The invention patent "A method for remote sensing image target detection based on anchor-free frame" with application number CN202011357230.5 includes the following: establishing an anchor-free frame remote sensing image target detection model, including: feature extraction network, feature pyramid and anchor-free frame detector; obtaining remote sensing images, and performing sliding partitioning to obtain small-sized images; inputting small-sized images into feature extraction networks, and using feature pyramid structures to obtain feature maps of three different scales; inputting feature maps into anchor-free frame detectors for prediction to obtain target prediction results; designing a multi-index fusion loss function, using target prediction results and multi-index fusion loss functions to complete the training of target detection models, and obtaining trained target detection models; using trained target detection models to perform remote sensing image target detection. The multi-index fusion loss function used in the existing patent integrates the category loss function Lclass, the position loss function Lloc, the bias loss function Loff and the angle loss function Lang, which are different from the diagonal regression loss used in the present invention, and the existing patent does not disclose the technical solution of Fisher criterion pairing used in the present application, so the existing patent is significantly different from the present application. In traditional technology, the anchor-free object detection method based on key points first detects several key points of the target box, and then pairs them as candidate target boxes to achieve target detection. This type of method uses push and pull losses to learn an embedding vector for each key point for combined pairing. In the existing training process, the push and pull losses are simply summed, and the similarities of the same target and the differences of different targets are not considered as a whole. This makes the learned embedding vectors not very discriminative, key point matching errors, and false detections. On the other hand, when regressing the key point position, the existing anchor-free object detection method only considers the deviation caused by downsampling, and does not regress the target box as a whole, which also leads to low target detection accuracy.

[0005] In summary, the existing technology has technical problems such as difficulty in anchor frame design, sample imbalance and low detection accuracy. Summary of the invention

[0006] The technical problem to be solved by the present invention is how to solve the technical problems of difficulty in anchor frame design, sample imbalance and low detection accuracy in the prior art.

[0007] The present invention adopts the following technical solutions to solve the above technical problems: A method for detecting anchor-free objects based on a diagonal network includes:

[0008] S1. Collect an input image, use the hourglass network Hourglass Net as the backbone network to extract the deep features of the input image, perform corner pooling and center pooling operations on the deep features, and obtain a key point heat map, an embedded feature vector feature map, and an offset feature map;

[0009] S2, using the diagonal network to process the key point heat map according to the Focal Loss loss, sorting the key points according to the confidence of each key point heat map, selecting the key points with the top k confidences as the target key points, and obtaining the key point positions and key point category information of the upper left corner, the lower right corner and the center point;

[0010] S3, calculating the distance between the embedded feature vectors of each upper left corner and k lower right corner key points according to the key point positions and the key point category information, and pairing the upper left corner key points with the lower right corner key points to obtain a key point combination pairing to construct a target prediction frame, and when there is a similar central key point falling into the central area of ​​the candidate target frame, determining that the current target prediction frame is true;

[0011] S4. Using Fisher linear discriminant analysis method, the grouping pairing loss is obtained by the following logical processing:

[0012]

[0013] in, According to the key point combination pairing, network training is performed using the group pairing loss to obtain a suitable target detection network;

[0014] S5, using the target detection network to learn the upper left corner offset (Δx tl ,Δy tl ) and the offset of the lower right corner key point (Δx br ,Δy br ), the diagonal regression loss is obtained by the following logic:

[0015] Loss L =αL dis +βL size +γL shape

[0016] , and then regress the target prediction box to finely locate the target box (x tl +Δx tl ,y tl +Δy tl ,x br +Δx br ,y br +Δy br ), where L dis , L size and L shape They are the diagonal center point distance, diagonal length difference and diagonal inclination angle difference.

[0017] Based on the Fisher criterion, the present invention designs a loss function for embedding vector learning to train key point combination pairing, and further confirms the preliminary pairing results using the central key point to reduce the target false detection rate. At the same time, a diagonal loss for target prediction box regression training is designed, which uses the distance, length, and slope measurement loss of the diagonal center points of the predicted rectangle and the labeled rectangle to improve the accuracy of target detection.

[0018] The present invention uses the offset output by the network to regress the target prediction frame and improve the positioning accuracy of the target frame. In order to improve the target positioning accuracy, the present invention enables the target detection network to learn the offset caused by downsampling in the network and the offset caused by inaccurate key point positions, so as to achieve fine positioning of the target frame.

[0019] In a more specific technical solution, the key point heat map in step S1 includes key points at the upper left corner, the lower right corner and the center point.

[0020] In a more specific technical solution, the Focal Loss is obtained by using the following logical processing in step S2:

[0021]

[0022] , where p cij is the confidence that the position (i, j) on the key point heat map is the c-th type of target, N is the number of targets in the image, and α and β are hyperparameters that control the contribution of each point.

[0023] In a more specific technical solution, the operation of sorting the key points in step S2 includes: sorting in descending order.

[0024] In a more specific technical solution, step S3 includes:

[0025] S31, pairing the key points of the upper left corner, the lower right corner and the center point, grouping the key points of the same target into a group, and constructing the target prediction box accordingly;

[0026] S32, processing the key point positions of each key point at the upper left corner and k lower right corners and the key point category information to obtain the distance of the embedded feature vector;

[0027] S33, determining whether the distance of the embedded feature vector is less than or equal to a preset threshold, and determining whether the key point category information is consistent;

[0028] S34, if yes, pair the key points of the upper left corner, the lower right corner and the center point and add the key point combination pairing to the candidate target set;

[0029] S35: If not, no pairing operation is performed.

[0030] In a more specific technical solution, the step S3 further includes:

[0031] S31', judging whether the current center point falls into the geometric center area of ​​the target prediction box, and whether the embedded feature of the center point is similar to the embedded features of the upper left corner and the lower right corner of the target prediction box;

[0032] S32', if yes, then determine that the current target prediction box is true;

[0033] S33', if not, determine that the current target prediction box is false.

[0034] The present invention uses the target center key point to judge the authenticity of the candidate target. The key point of the same category as a candidate target is selected from the center key point, and the false detection target that may exist in the candidate target frame obtained in the previous step is further identified and removed, which solves the problem of sample imbalance in the prior art and improves the accuracy of the target detection algorithm.

[0035] In a more specific technical solution, the following logic is used to determine the central area of ​​the candidate target frame in step S3:

[0036] γ=|x tl -x br |×|y tl -y br |*ε

[0037] , where ε is a hyperparameter, γ is the radius, (x tl ,y tl ) is the coordinate of the key point in the upper left corner, (x br ,y br ) are the coordinates of the key point in the lower right corner.

[0038] The present invention sets a geometric center area based on scale sensitivity, and sets the geometric center area by defining a radius that is positively correlated with the target size, so as to eliminate incorrectly matched candidate target frames in a candidate target set.

[0039] In a more specific technical solution, step S5 includes:

[0040] S51, obtain the diagonal center point distance of the target prediction box by the following logic processing:

[0041]

[0042] S52, obtain the diagonal length difference of the target prediction box by the following logic processing:

[0043]

[0044] S53, obtain the diagonal tilt angle difference of the target prediction frame by the following logic processing:

[0045] ,

[0046] Among them, α, β, and γ are used to control L dis , L size and L shape The weights of the three components in network training.

[0047] In order to solve the problem that the positioning of the target prediction box is not accurate enough, the present invention designs a target prediction box regression module, determines the diagonal regression loss for network learning according to the diagonal center point distance, diagonal length difference, and diagonal inclination angle difference, calculates the diagonal information (length, center point and slope) of the target rectangular box using the coordinate information of the upper left corner and the lower right corner, and uses the diagonal loss function to measure the similarity between the prediction box and the annotation box to realize the frame regression, further improve the positioning accuracy, and realize the fine positioning of the target box.

[0048] In a more specific technical solution, the loss target detection network in step S5 includes: a diagonal network DiagonalNet.

[0049] Aiming at the defects of inaccurate key point pairing and low regression accuracy in the prior art, the present invention designs a diagonal network (DiagonalNet) for anchor-free target detection for target detection, so as to improve the accuracy of key point pairing and regression accuracy.

[0050] In a more specific technical solution, a method for detecting an anchor-free object based on a diagonal network includes:

[0051] The backbone network and detection head module are used to collect input images, use the hourglass network Hourglass Net as the backbone network to extract the deep features of the input images, perform corner pooling and center pooling operations on the deep features, and obtain key point heat maps, embedded feature vector feature maps, and offset feature maps;

[0052] A key point detection module is used to process the key point heat map according to the Focal Loss loss using a diagonal network, sort the key points according to the confidence of each key point heat map, select the key points with the top k confidences as target key points, obtain the key point positions and key point category information of the upper left corner, the lower right corner and the center point, and the key point detection module is connected to the backbone network and the detection head module;

[0053] A key point pairing module is used to calculate the distance between the embedded feature vectors of each upper left corner and k lower right corner key points in the embedded feature vector feature map according to the key point positions and the key point category information, and to pair the upper left corner key points with the lower right corner key points to obtain a key point combination pairing to construct a target prediction frame. When there is a similar central key point falling into the central area of ​​the candidate target frame, the current target prediction frame is determined to be true. The key point pairing module is connected to the key point detection module;

[0054] The network training module is used to use the Fisher linear discriminant analysis method and the following logic processing to obtain the group pairing loss:

[0055]

[0056] in, According to the key point combination pairing, network training is performed using the group pairing loss to obtain a suitable target detection network, and the network training module is connected to the key point pairing module;

[0057] The target prediction box regression module is used to learn the upper left corner offset (Δx tl ,Δy tl ) and the offset of the lower right corner key point (Δx br ,Δy br ), the diagonal regression loss is obtained by the following logic:

[0058] Loss L =αL dis +βL size +γL shape ,

[0059] The target prediction box is processed by regression to finely locate the target box (x tl +Δx tl ,y tl +Δy tl ,x br +Δx br ,y br +Δy br ), where L dis , L size and L shape They are respectively the diagonal center point distance, the diagonal length difference and the diagonal tilt angle difference. The target prediction box regression module is connected to the network training module.

[0060] Compared with the prior art, the present invention has the following advantages: Based on the Fisher criterion, the present invention designs a loss function for embedding vector learning to train key point combination pairing, and further confirms the preliminary pairing results using the central key point pair, thereby reducing the target false detection rate. At the same time, a diagonal loss for target prediction box regression training is designed, which uses the distance, length, and slope measurement loss of the diagonal center points of the predicted rectangle and the labeled rectangle, thereby improving the accuracy of target detection.

[0061] The present invention uses the offset output by the network to regress the target prediction frame and improve the positioning accuracy of the target frame. In order to improve the target positioning accuracy, the present invention enables the target detection network to learn the offset caused by downsampling in the network and the offset caused by inaccurate key point positions, so as to achieve fine positioning of the target frame.

[0062] The present invention uses the target center key point to judge the authenticity of the candidate target. The key point of the same category as a candidate target is selected from the center key point, and the false detection target that may exist in the candidate target frame obtained in the previous step is further identified and removed, which solves the problem of sample imbalance in the prior art and improves the accuracy of the target detection algorithm.

[0063] The present invention sets a geometric center area based on scale sensitivity, and sets the geometric center area by defining a radius that is positively correlated with the target size, so as to eliminate incorrectly matched candidate target frames in a candidate target set.

[0064] In order to solve the problem that the positioning of the target prediction box is not accurate enough, the present invention designs a target prediction box regression module, determines the diagonal regression loss for network learning according to the diagonal center point distance, diagonal length difference, and diagonal inclination angle difference, calculates the diagonal information (length, center point and slope) of the target rectangular box using the coordinate information of the upper left corner and the lower right corner, and uses the diagonal loss function to measure the similarity between the prediction box and the annotation box to realize the frame regression, further improve the positioning accuracy, and realize the fine positioning of the target box.

[0065] Aiming at the defects of inaccurate key point pairing and low regression accuracy in the prior art, the present invention designs a diagonal network (DiagonalNet) for anchor-free target detection for target detection to improve the accuracy of key point pairing and regression accuracy. The present invention solves the technical problems of difficult anchor frame design, sample imbalance and low detection accuracy in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 Schematic diagram of the basic modules of the anchor-free object detection system based on the diagonal network;

[0067] Figure 2Schematic diagram of the design scheme for anchor-free target detection method based on diagonal network;

[0068] Figure 3 A schematic diagram of a correctly paired target prediction frame in further identification of a candidate target frame;

[0069] Figure 4 A schematic diagram of a target prediction frame that is incorrectly paired in further identification of a candidate target frame;

[0070] Figure 5 Schematic diagram of diagonal regression loss. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] Example 1

[0073] like Figure 1 and Figure 2 As shown, the target detection framework of the present invention mainly includes the following modules:

[0074] Backbone network and detection head module 1:

[0075] The present invention uses the hourglass network as the backbone network to extract the deep features of the input image. The deep features are then subjected to corner pooling and center pooling operations to obtain the following information:

[0076] Key point heatmaps: including top-left, bottom-right and center. The size of each heatmap is 4 times the input image downsampled, denoted as W×H. Each key point has a class label c∈{1,2,…,C}, and the heatmap dimension is 3×C×W×H. Each value on the heatmap represents the confidence that the key point appears at the corresponding position.

[0077] Embeddings: Each key point corresponds to an embedding vector, and each feature map is of size W×H×l, where l is the dimension of the embedding vector. The present invention sets l=1, i.e., the embedding vector is a scalar. The dimension of the embedding feature vector feature map is 3×W×H, and each value on the feature map represents the embedding vector corresponding to the key point, which is used for key point pairing.

[0078] Offset feature map (Offsets): used to learn the offset (Δx, Δy) of the coordinates of the upper left and lower right key points. The dimension of the offset feature map is 2×2×W×H. This value is used to fine-tune the positioning of the target box.

[0079] Key point detection module 2:

[0080] This module will output three types of key point heat maps based on the diagonal network: the upper left corner, the lower right corner, and the center point. For each type of heat map, the first k key points with the largest confidence are selected as the key points of the target, and a total of 3*k key point positions and category information of three categories are obtained.

[0081] Key Point Pairing Module 3:

[0082] The key point pairing module pairs the three types of key points detected in the previous step, groups the key points of the same target into a group, and constructs a target prediction box. The accuracy of the pairing directly determines the performance of target detection. The process is divided into the following two steps: (1) Generation of candidate targets: Calculate the distance between the embedded feature vectors of each upper left corner and the k lower right corner key points. If the distance is greater than the set threshold, or the category information of the two key points is inconsistent, then no pairing is performed. Otherwise, the pairing is successful and it is added to the candidate target set. (2) Selection of candidate targets: There may be falsely detected targets in step (1). The target center key points are used to judge the authenticity of the candidate targets again. If a similar center key point falls into the center area of ​​the candidate target box, then the candidate target box is considered to be the true target box. Otherwise, the candidate target box is a falsely detected target box.

[0083] Target prediction box regression module 4:

[0084] Finally, the offset output by the network is used to regress the target prediction box to improve the positioning accuracy of the target box. The rectangular box formed by the two key points in the upper left corner and the lower right corner of the target is defined as the predicted target box (Bounding Box). Due to the positioning deviation caused by the downsampling operation in the target detection network and the positioning deviation of the key point detection itself, the prediction box positioning is inaccurate. In order to improve the target positioning accuracy, the network learns the offset caused by the downsampling in the network and the offset caused by the inaccurate position of the key points, which leads to the inaccurate positioning of the target prediction box obtained by the key point pairing module. For this reason, the target prediction box regression module is designed to finely locate the target box. The module outputs the upper left corner offset (Δx tl ,Δy tl ) and the offset of the lower right corner key point (Δx br ,Δy br ), then the target prediction box after regression is: (x tl +Δx tl ,y tl +Δy tl ,x br +Δx br ,y br +Δy br ).

[0085] Example 2

[0086] The present invention designs a diagonal network (DiagonalNet) for anchor-free object detection for object detection. The scheme designs a group pairing loss for supervising the learning of embedding vectors; the scheme designs a diagonal loss for the overall regression of the prediction box. The proposed object detection method is experimentally verified on the MS COCO dataset, and compared with the object detection methods with anchor boxes (such as Faster R-CNN, SSD, etc.) and without anchor boxes (such as CenterNet, CornerNet, etc.). The results show that the object detection method of the diagonal network proposed in the present invention has higher detection accuracy.

[0087] Table 1 Comparison of experimental results

[0088]

[0089] Among them, the Chinese names corresponding to the target detection algorithms in Table 1 are as follows: Faster R-CNN, TridentNet, SSD513, RetinaNet, CornerNet, CenterNet, ExtremeNet, CentripetalNet, and DiagonalNet.

[0090] The present invention has made innovative designs in key point pairing, loss function definition of prediction box regression, etc. The working principles of the innovative designs in this aspect are introduced below.

[0091] 1. Working principle of key point detection module

[0092] Key point detection uses Focal Loss, which is defined as follows:

[0093]

[0094] Among them, p cij For a key point (such as the upper left corner), the position (i, j) on the heat map is the confidence of the c-th type of target, N is the number of targets in the image, and α and β are hyperparameters that control the contribution of each point.

[0095] 2. How keypoint pairing works

[0096] The key point pairing module pairs the three types of key points detected in the previous step, groups the key points of the same target into a group, and constructs the target prediction box. The accuracy of the pairing directly determines the performance of target detection. The present invention generates an embedding vector for each key point. Assume that the embedding vectors of the upper left corner, lower right corner, and center key points of the kth target are e lk , e rk ,e ck The working process of the key point pairing module of the present invention is as follows:

[0097] (1) Select the top k key points in the upper left corner, lower right corner and center from the heat map according to the confidence level, where each key point has two types of information: position and category.

[0098] (2) Calculate the distance between the embedded feature vectors of each upper left corner and the k lower right corner key points. If the distance is greater than the set threshold, or the category information of the two key points is inconsistent, no pairing is performed. Otherwise, the pairing is successful and it is added to the candidate target set. Each candidate target is represented by a 5-dimensional vector (x tl ,y tl ,x br ,y br,c) represents the upper left corner coordinates, lower right corner coordinates and target category c of the target box.

[0099] (3) The candidate target frame obtained in the previous step may contain false detection targets, which affects the accuracy of the target detection algorithm. Therefore, it is necessary to further identify the candidate targets. The present invention uses the target center key point to judge the authenticity of the candidate target. Select the key point with the same category as a candidate target from the center key point, and further judge whether the center point meets the following two conditions:

[0100] ①Whether the center point falls into the geometric center area of ​​the target box;

[0101] ②Whether the embedded feature of the center is similar to the embedded features of the upper left corner and lower right corner of the target box.

[0102] If the center point cannot meet the above two conditions at the same time, the candidate target is considered to be a false detection target and is deleted.

[0103] The definition of the target geometric center area in the present invention is a circular area with the center point of the line between the upper left corner and the lower right corner as the center and γ as the radius. In addition, considering that the geometric center areas of targets of different sizes should also be different, the present invention proposes a method for setting the geometric center area based on scale sensitivity, which realizes the setting of the geometric center area by defining a radius that is positively correlated with the target size, that is, the larger the target, the larger the radius of its center area, and the smaller the target, the smaller the radius of its center area. The radius is defined as follows:

[0104] γ=|x tl -x br |×|y tl -y br |*ε (7)

[0105] Where ε is a hyperparameter.

[0106] like Figure 3 and Figure 4 As shown in the figure, a schematic diagram of further identifying the candidate target boxes of key point pairing combinations is given. Figure 3 The central key point (x c ,y c ) falls into the upper left corner (x tl ,y tl ) and the lower right corner (x br ,y br ) The central area of ​​the rectangular box composed of two key points is considered to be the correct target prediction box, and Figure 4 The central key point in does not fall into the central area of ​​the rectangular box, and the rectangular box is identified as a wrongly matched candidate target box and needs to be removed from the candidate target set.

[0107] In the network training process, the group pairing loss is designed by referring to the classification idea of ​​Fisher linear discriminant analysis (i.e., similar samples are aggregated as much as possible, and heterogeneous samples are dispersed as much as possible). Since the grouped samples are one-dimensional embedding vectors, the present invention uses variance to measure the similarity between samples. The smaller the variance of samples in the same group, the more similar the samples in the group are, and the larger the variance of samples in different groups, the less similar the samples in different groups are. The group pairing loss is defined as follows:

[0108]

[0109] in,

[0110] 3. How target prediction box regression works

[0111] Due to the offset caused by downsampling in the network and the offset caused by inaccurate key point positions, the target prediction box obtained by the key point pairing module is not accurately positioned. Therefore, a target prediction box regression module is designed to finely locate the target box. This module outputs the upper left corner offset (Δx tl ,Δy tl ) and the offset of the lower right corner key point (Δx br ,Δy br ), then the target prediction box after regression is: (x tl +Δx tl ,y tl +Δy tl ,x br +Δx br ,y br +Δy br ).

[0112] In the network training stage, the present invention designs a diagonal loss for network learning. The diagonal loss uses the coordinate information of the upper left corner and the lower right corner to calculate the diagonal information (length, center point and slope) of the target rectangular box, and uses the diagonal loss function to measure the similarity between the predicted box and the labeled box, realizes the frame regression, and further improves the positioning accuracy.

[0113] like Figure 5 As shown in the figure, the solid rectangle represents the target annotation box, and its upper left corner and lower right corner coordinates are (x tl ,y tl ) and (x br ,y br ), the dotted rectangle represents the target prediction box, and its upper left and lower right corner coordinates are and L pre and L gt are the diagonal lengths of the prediction box and the annotation box, L disis the distance between the center points of the two diagonals, θ pre and θ gt is the inclination angle of the diagonal of the prediction box and the diagonal of the annotation box. The diagonal regression loss designed by the present invention is defined as follows:

[0114] Loss L =αL dis +βL size +γL shape (3)

[0115] Among them, L dis , L size and L shape They are the diagonal center point distance, diagonal length difference and diagonal tilt angle difference, which are defined as follows:

[0116]

[0117]

[0118]

[0119] α, β, γ are used to control L dis , L size and L shape The weights of the three components in network training.

[0120] In summary, based on the Fisher criterion, the present invention designs a loss function for embedding vector learning to train key point combination pairing, and further confirms the preliminary pairing results using the central key point to reduce the target false detection rate. At the same time, a diagonal loss for target prediction box regression training is designed, which uses the distance, length, and slope measurement loss of the diagonal center points of the predicted rectangle and the labeled rectangle to improve the accuracy of target detection.

[0121] The present invention uses the offset output by the network to regress the target prediction frame and improve the positioning accuracy of the target frame. Due to the positioning deviation caused by the downsampling operation in the target detection network and the positioning deviation of the key point detection itself, the prediction frame is not accurately positioned. In order to improve the target positioning accuracy, the network learns the offset caused by the downsampling in the network and the offset caused by the inaccurate position of the key points, so as to achieve fine positioning of the target frame.

[0122] The present invention uses the target center key point to judge the authenticity of the candidate target. The key point of the same category as a candidate target is selected from the center key point, and the false detection target that may exist in the candidate target frame obtained in the previous step is further identified and removed, which solves the problem of sample imbalance in the prior art and improves the accuracy of the target detection algorithm.

[0123] The present invention sets a geometric center area based on scale sensitivity, and sets the geometric center area by defining a radius that is positively correlated with the target size, so as to eliminate incorrectly matched candidate target frames in a candidate target set.

[0124] In view of the inaccurate positioning of the target prediction frame obtained by the key point pairing module due to the offset caused by downsampling in the network and the offset caused by inaccurate key point positions, the present invention designs a target prediction frame regression module, determines the diagonal regression loss for network learning according to the diagonal center point distance, diagonal length difference, and diagonal inclination angle difference, calculates the diagonal information (length, center point and slope) of the target rectangular frame using the diagonal loss and the coordinate information of the upper left corner and the lower right corner, and uses the diagonal loss function to measure the similarity between the prediction frame and the annotation frame to realize frame regression, further improve the positioning accuracy, and achieve fine positioning of the target frame.

[0125] Aiming at the defects of inaccurate key point pairing and low regression accuracy in the prior art, the present invention designs a diagonal network (DiagonalNet) for anchor-free target detection for target detection to improve the accuracy of key point pairing and regression accuracy. The present invention solves the technical problems of difficult anchor frame design, sample imbalance and low detection accuracy in the prior art.

[0126] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for anchor-free object detection based on a diagonal network, characterized in that: The method comprises: S1. Collect an input image, use the hourglass network Hourglass Net as the backbone network to extract the deep features of the input image, perform corner pooling and center pooling operations on the deep features, and obtain a key point heat map, an embedded feature vector feature map, and an offset feature map; S2, using the diagonal network to process the key point heat map according to the Focal Loss loss, sorting the key points according to the confidence of each key point heat map, selecting the key points with the top k confidences as the target key points, and obtaining the key point positions and key point category information of the upper left corner, the lower right corner and the center point; S3, calculating the distance between the embedded feature vectors of each upper left corner and k lower right corner key points according to the key point positions and the key point category information, and pairing the upper left corner key points with the lower right corner key points to obtain a key point combination pairing to construct a target prediction box, and when there is a similar central key point falling into the central area of ​​the candidate target box, the current target prediction box is determined to be true; S4. Using Fisher linear discriminant analysis method, the following logic processing is used to obtain the group pairing loss: in, According to the key point combination pairing, network training is performed using the group pairing loss to obtain a suitable target detection network; S5, using the target detection network to learn the upper left corner offset (Δx tl ,Δy tl ) and the offset of the lower right corner key point (Δx br ,Δy br ), the diagonal regression loss is obtained by the following logic: Loss L =αL dis +βL size +γL shape , The target prediction box is processed by regression, and the target prediction box (x tl +Δx tl ,y tl +Δy tl ,x br +Δx br ,y br +Δy br ), where L dis , L size and L shape They are the diagonal center point distance, diagonal length difference and diagonal tilt angle difference; Wherein, the step S5 comprises: S51, obtain the diagonal center point distance of the target prediction box by the following logic processing: S52, obtain the diagonal length difference of the target prediction box by the following logic processing: S53, obtain the diagonal tilt angle difference of the target prediction frame by the following logic processing: Among them, α, β, and γ are used to control L dis , L size and L shape The weights of the three components in network training.

2. The method for anchor-free object detection based on a diagonal network according to claim 1, characterized in that: The key point heat map in step S1 includes key points at the upper left corner, the lower right corner and the center point.

3. The method for anchor-free object detection based on a diagonal network according to claim 1, characterized in that: In step S2, the Focal Loss is obtained by using the following logic processing: , Among them, p cij is the confidence that the position (i, j) on the key point heat map is the c-th type of target, N is the number of targets in the image, and α and β are hyperparameters that control the contribution of each point.

4. The method for anchor-free object detection based on a diagonal network according to claim 1, characterized in that: The operation of sorting the key points in step S2 includes: sorting in descending order.

5. The method for anchor-free object detection based on a diagonal network according to claim 1, characterized in that: The step S3 comprises: S31, pairing the key points of the upper left corner, the lower right corner and the center point, grouping the key points of the same target into a group, and constructing the target prediction box accordingly; S32, processing the key point positions of each key point at the upper left corner and k lower right corners and the key point category information to obtain the distance of the embedded feature vector; S33, determining whether the distance of the embedded feature vector is less than or equal to a preset threshold, and determining whether the key point category information is consistent; S34, if yes, pair the key points of the upper left corner, the lower right corner and the center point and add the key point combination pairing to the candidate target set; S35: If not, no pairing operation is performed.

6. The method for anchor-free object detection based on a diagonal network according to claim 1, characterized in that: The step S3 further comprises: S31', judging whether the current center point falls into the geometric center area of ​​the target prediction box, and whether the embedded feature of the center point is similar to the embedded features of the upper left corner and the lower right corner of the target prediction box; S32', if yes, then determine that the current target prediction box is true; S33', if not, determine that the current target prediction box is false.

7. The method for anchor-free object detection based on a diagonal network according to claim 1, characterized in that: In step S3, the central area of ​​the candidate target frame is determined using the following logic: γ=|x tl -x br |×|y tl -y br |*e, Among them, ε is a hyperparameter, γ is the radius, (x tl ,y tl ) is the coordinate of the key point in the upper left corner, (x br ,y br ) are the coordinates of the key point in the lower right corner.

8. The method for anchor-free object detection based on a diagonal network according to claim 1, characterized in that: The loss target detection network in step S5 includes: a diagonal network DiagonalNet.

9. A diagonal network-based anchor-free object detection system, used to execute the diagonal network-based anchor-free object detection method according to any one of claims 1 to 8, characterized in that: The system comprises: The backbone network and detection head module are used to collect input images, use the hourglass network Hourglass Net as the backbone network to extract the deep features of the input images, perform corner pooling and center pooling operations on the deep features, and obtain key point heat maps, embedded feature vector feature maps, and offset feature maps; A key point detection module is used to process the key point heat map according to the Focal Loss loss using a diagonal network, sort the key points according to the confidence of each key point heat map, select the key points with the top k confidences as target key points, obtain the key point positions and key point category information of the upper left corner, the lower right corner and the center point, and the key point detection module is connected to the backbone network and the detection head module; A key point pairing module is used to calculate the distance between the embedded feature vectors of each upper left corner and k lower right corner key points in the embedded feature vector feature map according to the key point positions and the key point category information, and to pair the upper left corner key points with the lower right corner key points to obtain a key point combination pairing to construct a target prediction frame. When there is a similar central key point falling into the central area of ​​the candidate target frame, the current target prediction frame is determined to be true. The key point pairing module is connected to the key point detection module; The network training module is used to use the Fisher linear discriminant analysis method and the following logic to obtain the group pair loss: in, According to the key point combination pairing, network training is performed using the group pairing loss to obtain a suitable target detection network, and the network training module is connected to the key point pairing module; The target prediction box regression module is used to learn the upper left corner offset (Δx tl ,Δy tl ) and the offset of the lower right corner key point (Δx br ,Δy br ), the diagonal regression loss is obtained by the following logic: Loss L =αL dis +βL size +γL shape , The target prediction box is processed by regression to finely locate the target box (x tl +Δx tl ,y tl +Δy tl ,x br +Δx br ,y br +Δy br ), where L dis , L size and L shape They are respectively the diagonal center point distance, the diagonal length difference and the diagonal tilt angle difference. The target prediction box regression module is connected to the network training module.