A multi-point target detection method based on concentric circle intersection-over-union ratio

Abstract

The application provides a multi-point target detection method based on a concentric circle intersection ratio, belongs to the field of computer vision, and particularly relates to a supervised learning regression method for a 24-point target representation method under a target detection task. The method first converts a data set format, so as to construct a data set true value format matched with the regression method; secondly, a concentric circle expression is constructed for model output data, so as to constitute a plurality of loss functions; next, the concentric circle expression is constructed for the data set after the conversion, so as to constitute a plurality of loss functions; next, the concentric circle expression of the model output data and the concentric circle expression constituted by the true value data set are used for loss function calculation, and a total of 25 loss items are constituted; finally, a multi-task regression method is adopted to balance the learning complexity of each loss, and the regression method is realized. The method avoids the calculation of the area of a random polygon and the intersection area calculation process in the traditional multi-point target representation regression method, greatly simplifies the complexity of the regression strategy, can greatly improve the convergence speed of the training learning process, and enables the method to be put into practical engineering application.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to a supervised learning detection method for multi-point target representation under a target detection task. BACKGROUND

[0002] As an important branch of computer field in the era of artificial intelligence, the target detection task has important application value in the fields of unmanned driving, assisted driving, intelligent navigation, security monitoring, etc. At the current stage, the target detection task mostly takes standard images as data input sources, extracts features from the part of the semantic information region of interest, and labels the target of interest with a rectangular box to realize target detection function.

[0003] With the use of fisheye cameras, panoramic cameras and other large field of view visual sensors, target detection methods for distorted images have emerged. However, due to the constraints of imaging principles, the traditional rectangular box target representation method cannot meet the target detection needs of distorted images. This method will introduce a large amount of useless background information when labeling the target of interest, especially in image regions with severe radial distortion, such as the edges of fisheye images. Therefore, there is an urgent need for a new method to adapt to the target detection representation process under distorted images.

[0004] Under distorted images, the multi-point representation method can better represent the contour of the target and has distortion robustness. However, due to the large number of regression parameters and strong coupling of the loss function of this method, it leads to a large convergence difficulty and long convergence time of the supervised learning process based on this method, which has certain limitations in actual use.

[0005] The Chinese patent with patent application number 202111462501.8 and the invention name of "Angle-adaptive elliptical template target detector" uses elliptical representation to realize the expression of the detected target. This method has good expression effect for rotated targets under standard images, but still inevitably contains a large amount of redundant background information area for distorted targets under fisheye and panoramic images, and the target edge contour is not prominent, as shown in Figure 1 In addition, this method has relatively simple regression parameters and fewer numbers, only containing 6 basic parameters in [cx, cy, la, sa, θ, n(i)], which is easier than the N+3 regression parameters in [x c , y c , d1, d2,..., d N , class] in this method, and the coupling and complexity challenge difficulty is smaller. SUMMARY

[0006] In order to improve the above problems, the present application provides a multi-point target detection method based on the intersection ratio of concentric circles, which can reduce the complexity of target detection.

[0007] A multi-point target detection method based on concentric circle intersection ratio, comprising the following steps:

[0008] Step 1: Construct the data set true value information under the multi-point target representation method, including target position information and category information, the number of multi-points is represented by N, specifically including:

[0009] Step 101, label the target region of the given image, mark the labeled region as mask, and calculate the target center coordinates;

[0010] Step 102, taking the target center coordinates as the starting point of the rays, equally spaced clockwise to construct N rays, which produce N intersection points with the mask boundary, in turn recorded as (x1, y1, x2, y2,..., xN, yN); N N );

[0011] Step 103, take the N intersection points in step 102, respectively, to calculate the Euclidean distance relative to the center coordinates (x c ,y c ):

[0012]

[0013] Step 104, taking the target center coordinates in step 101, taking the N Euclidean distances in step 103, and taking the target class label class as the target category information of the N-point target representation method, the total expression form is defined as:

[0014] [x c ,y c ,d1,d2,...,d N ,class];

[0015] Step 2: Express the predicted target center coordinates and N-point contour coordinates output by the deep learning network in the form of concentric circles, a total of N concentric circles, specifically:

[0016] Step 201, select a deep learning network model that has completed the target detection task, modify its output data format to be the same as the definition of the N-point representation method data true value in step 104, a total of N+3 output parameters, mark the deep learning network output parameters as [x pc ,y pc ,d p1 ,d p2 ,...,d pN ,class p ];

[0017] Step 202, construct the numerical conversion relationship between the network output parameters and the true value information, the calculation method is as follows:​

[0018]

[0019] wherein, w img and h img represent the width and height of the input image of the target detection task, i = 1, 2, 3, …, N, and the calculation is performed in turn in a clockwise order, and the concentric circle expression is constructed according to formula (3), a total of N concentric circles, and the calculation is as follows:

[0020]

[0021] wherein, i = 1, 2, 3, …, N, and the recording is performed in turn in a clockwise order, and is recorded as [pd_c1, pd_c2, …, pd_cn]. N

[0022] Step 3: The center coordinate ground truth and N-point contour coordinates in the ground truth information are expressed by concentric circles, a total of N concentric circles, and the specific expression is as follows:

[0023] Step 301: The concentric circle expression of the ground truth data is constructed according to the target category label ground truth information, and the calculation is as follows:

[0024]

[0025] wherein, i = 1, 2, 3, …, N, and the recording is performed in turn in a clockwise order, and is recorded as [gt_c1, gt_c2, …, gt_cn]. N

[0026] Step 4: The intersection over union loss is calculated for the predicted concentric circle and the ground truth concentric circle, and the loss calculation based on cross entropy is performed for the category information, and the specific expression is as follows:

[0027] Step 401: The single loss function is constructed by using the predicted concentric circle and the ground truth concentric circle expression recording results of step 202 and step 301, and the calculation is as follows:

[0028] Loss i = 1-GIOU cir (pd_c i , gt_c i ) (6)

[0029] wherein, GIOU cir is calculated as follows:

[0030]

[0031] wherein, S_c represents the input predicted concentric circle pd_c i and gt_c i ​​The maximum circumscribed circle area of the circle of the same index number in the current frame is denoted as A

[0032] IOU cir The calculation method is as follows:

[0033]

[0034] Wherein, r min and r max respectively represent the smaller radius value and the larger radius value of the prediction concentric circle and the true value concentric circle of the same index number in the current frame, dist cen represents the center distance of the two concentric circles, and t1 and t2 are calculated as follows:

[0035]

[0036] Step 402, the numerical loss of the true value class and the predicted class is calculated by using the standard cross-entropy loss calculation method, denoted as Loss p cls , i = 1, 2, 3,..., N, and the Loss i in step 401 is recorded in turn according to the clockwise order, and there are N position loss terms, denoted as [Loss1, Loss2,..., Loss N ], the class loss and the position loss term are combined to form the final loss term, denoted as [Loss1, Loss2,..., Loss N , Loss cls ], and there are N+1 sub-loss terms;

[0037] Step 5: Introducing the multi-task regression idea, evaluating the task complexity of the N+1 loss terms, using the gradient information in the training process to establish the multi-task regression loss weight, specifically:

[0038] Step 501, the loss weight [w1, w2,..., w N , w cls ] is recorded, which corresponds to the N+1 sub-loss terms one by one, and the global loss function of the regression process is:

[0039]

[0040] Wherein, w i represents the weight coefficient of the position loss term;

[0041] ​Step 502, using formula (10) as the final global loss function, the deep learning network model is back propagated and learned and trained; after completion, the to-be-detected image is calculated according to the method of step 102 to obtain the center point coordinates, and then the N intersection points generated by the boundary are calculated according to the method of step 102, and the N+1 point coordinates are formed into a vector and input into the trained deep learning network model for target detection, and the target category is output.

[0042] Preferably, in step 501, w i The calculation method is:

[0043]

[0044] Wherein, ratio_pos i Indicates the change rate of each item in the position loss function in the training process, ratio_cls indicates the change rate of the category loss function in the training process, and T indicates the temperature coefficient; ratio_pos i And ratio_cls are calculated as follows:

[0045]

[0046] Wherein, t indicates the current time, t-1 indicates the last training time of the current time, w cls Indicates the weight coefficient of the classification loss term, and the calculation method is:

[0047]

[0048] Preferably, T is 100.

[0049] The present application has the following beneficial effects:

[0050] The present application provides a multi-point target detection method based on the concentric circle intersection ratio, introduces the multi-task learning idea into the regression calculation process, and converts the multi-parameter regression calculation problem into a multi-task learning problem. In addition, the present method changes the loss calculation method of parameter regression, avoids the calculation of the area of the random polygon and the intersection area calculation process of the traditional multi-point target representation regression method, greatly reduces the complexity of the regression strategy, ensures the speed and quality of the convergence of the training process, and makes the method applicable in actual engineering tasks. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 It is a comparison chart of the effects of four target representation methods including the method of the present application under fisheye image.

[0052] Figure 2 It is a true value data set construction process of the multi-point representation method.

[0053] Figure 3A diagram showing the physical meaning of concentric circles. DETAILED DESCRIPTION

[0054] The foregoing and other features of the present application will become apparent to those skilled in the art upon consideration of the following text description taken in conjunction with the accompanying drawings. In the description and drawings, specific embodiments of the application are disclosed in detail. These are indicative of the principles of the application, which are to be considered part of the broader principles of the application.

[0055] A multi-point target detection method based on the intersection-over-union ratio of concentric circles, comprising the following steps:

[0056] Step 1: Construct the ground truth information of the data set under the multi-point target representation method, including target position information and category information. This step creatively proposes a total of N+3 ground truth regression parameters for single-body targets, where N is the number of multi-points. Specifically, the present application takes N=24 as an example to calculate the details, which specifically includes:

[0057] Step 101, the regression strategy of the present application takes the Microsoft COCO data set as an example to describe the data set ground truth information construction process, reads the semantic segmentation ground truth information in the Microsoft COCO data set, and performs image labeling on the targets contained in the semantic segmentation ground truth information, denoted as mask, reads the target detection ground truth rectangular frame position information [x l ,y t ,w,h] in the data set, calculates the target center coordinates, and the calculation method is as follows:

[0058]

[0059] where x l , y t represent the horizontal and vertical coordinates of the upper left corner of the ground truth rectangular coordinate frame information respectively, and w, h represent the length and width of the ground truth rectangular coordinate frame information respectively;

[0060] Step 102, taking the target center coordinates as the starting point of the rays, construct N rays clockwise, the first ray is parallel to the x-axis of the image coordinate system, denoted as index number 1, each adjacent ray has an angle interval of 360° / N, and N intersection points are generated with the mask boundary, which are denoted as (x1, y1, x2, y2,..., x N ,y N ) in clockwise order, as shown in Figure 2 ;

[0061] Step 103, take the N intersection points in step 102, respectively, and perform relative center coordinates (x c ,y c) of the target center coordinates in step 103, and the calculation method is as formula (2), wherein i = 1, 2,..., N, and the calculation is sequentially performed in clockwise order;

[0062]

[0063] Step 104: Taking the Euclidean distance of the N intersection points in step 103 and the target center coordinates as the target boundary true value information of the N-point target representation method, taking the target center coordinate information in step 101 as the target center position information of the N-point target representation method, and taking the class label of the current target in the Microsoft COCO data set as the target class information of the N-point target representation method, denoted as class, the total expression method is defined as: [x c ,y c ,d1,d2,...,d N ,class], a total of N+3 numerical parameters, representing the current target.

[0064] Second step: The predicted target center coordinates and N-point contour coordinates output by the deep learning network are expressed in concentric circles, a total of N concentric circles, which creatively establishes a numerical mapping relationship between the network prediction output parameters and the regression true value, and the specific method is as follows:

[0065] Step 201: Without limiting the specific structure of the deep learning network model of the target detection task, only modifying the output data format to be the same as the N-point representation method data true value construction definition in step 104: [x c ,y c ,d1,d2,...,d N ,class], a total of N+3 output parameters, and the network output parameters are denoted as [x pc ,y pc ,d p1 ,d p2 ,...,d pN ,class p ].

[0066] Step 202: Building a numerical conversion relationship between the network output parameters and the true value information, and the calculation method is as follows:

[0067]

[0068] Wherein, w img and h img represent the width and height of the input image of the target detection task, i = 1, 2, 3,..., N, and the calculation is sequentially performed in clockwise order, and the concentric circle expression is constructed according to formula (3), a total of N concentric circles, and the calculation method is as follows:

[0069]

[0070] wherein i = 1, 2, 3, …, N, sequentially recorded in clockwise order, denoted as [pd cl, pd c2, …, pd c N ], concentric circle table is shown as Figure 3 indicated, wherein each prediction distance d pi occupies an individual circle in a group in a concentric circle;

[0071] Step 3: The center coordinate true value and N-point contour coordinates in the true value information are expressed in the concentric circle table, a total of N concentric circles. This step creatively proposes a concentric circle representation model for N-point calculation, specifically:

[0072] Step 301, according to the label true value information, construct the concentric circle expression of the true value data, the calculation method is as follows:

[0073]

[0074] wherein i = 1, 2, 3, …, N, sequentially recorded in clockwise order, denoted as [gt cl, gt c2, …, gt c N ];

[0075] Step 4: Calculate the intersection over union loss for the predicted concentric circle and the true value concentric circle, and calculate the loss based on cross entropy for the category information, a total of N+1 loss terms, specifically:

[0076] Step 401, use the predicted concentric circle and the true value concentric circle expression record results of step 202 and step 301 to construct a single loss function, the calculation method is as follows:

[0077] Loss i = 1-GIOU cir (pd_c i ,gt_c i ) (6)

[0078] wherein i = 1, 2, 3, …, N, specifically, GIOU cir is calculated as follows:

[0079]

[0080] wherein S_c represents the maximum circumscribed circle area of the input predicted concentric circle pd_c i and gt_c i , specifically, IOU cir is calculated as follows:

[0081]

[0082] wherein r min and rmax Dist represents the smaller and larger radius values ​​in the predicted concentric circles and the true concentric circles with the same index number, respectively. cen This represents the Euclidean distance between the centers of two concentric circles. Specifically, t1 and t2 are calculated as follows:

[0083]

[0084] Step 402: Calculate the true class and the predicted class using the standard cross-entropy loss method. p The numerical loss is denoted as Loss. cls Regarding the Loss in step 401 i Let i = 1, 2, 3, ..., N, and record them in clockwise order, for a total of N positional sub-loss terms, denoted as [Loss1, Loss2, ..., Loss]. N The category loss and location loss terms are combined to form the final loss term, denoted as [Loss1, Loss2, ..., Loss]. N Loss cls There are a total of N+1 sub-loss terms;

[0085] Step 5: Introducing the concept of multi-task regression, we evaluate the task complexity of N+1 loss terms, utilize gradient information from the training process to establish multi-task regression loss weights, balance the learning complexity of different types of tasks, construct the loss function, and realize the model training and learning process. This step creatively proposes a method for constructing and calculating the parameters of the loss function, specifically as follows:

[0086] Step 501, record the loss weights [w1, w2, ..., w N ,w cls Each of the N+1 sub-loss terms corresponds one-to-one, and the global loss function for the regression process is:

[0087]

[0088] In formula (10), w i The weighting coefficients representing the location loss term are calculated as follows:

[0089]

[0090] Where, ratio_pos i The ratio_cls represents the rate of change of each term in the position loss function during training, the ratio_cls represents the rate of change of the class loss function during training, and T represents the temperature coefficient with a value of 100. Specifically, the ratio_pos... i The ratio_cls is calculated as follows:

[0091]

[0092] wherein t represents the current time, t-1 represents the last round of training time of the current time, w cls represents the weight coefficient of the classification loss term, and the calculation method is as follows:

[0093]

[0094] Step 502, using formula (10) as the final global loss function, the deep learning network model is back propagated and learned and trained; after completion, the to-be-detected image is calculated according to the method of step 102 to obtain the center point coordinates, and then the N intersection points generated by the boundary are calculated according to the method of step 102, the N+1 point coordinates are formed into a vector, and input into the trained deep learning network model to perform target detection, and output the target category.

[0095] In summary, the above is only a specific embodiment under the Microsoft COCO data and N=24, and is not used to limit the protection scope of the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for multi-point target detection based on concentric circle intersection-over-union ratio, characterized in that, Comprise the following steps: First step: construct the data set true value information under the multi-point target representation method, including target position information and category information, the number of multi-points is represented by N, specifically including: Step 101, label the target region of the given image, mark the labeled region as mask, and calculate the target center coordinates; Step 102, taking the target center coordinate as the starting point of the rays, equally spaced clockwise to build N rays, a total of N intersection points with the mask boundary, recorded as (x1, y1, x2, y2,..., xN, yN) in turn. N N )​ Step 103, take the N intersection points in step 102, respectively, to the center coordinates (x c ,y c ) of the Euclidean distance: Step 104, take the target center coordinates in step 101, take the N Euclidean distances in step 103, and take the target category label class as the target category information of the N-point target representation method, and the total expression form is defined as: [x c ,y c ,d1,d2,...,d N ,class]; Second step: the predicted target center coordinates and N-point contour coordinates output by the deep learning network are expressed by concentric circles, a total of N concentric circles, specifically: Step 201, select a deep learning network model that completes the target detection task, modify its output data format to be the same as the definition of the N-point representation method data true value in step 104, a total of N+3 output parameters, and record the deep learning network output parameters as [x pc ,y pc ,d p1 ,d p2 ,...,d pN ,class p ] Step 202, build the numerical conversion relationship between the network output parameters and the true value information, and the calculation method is as follows: where w img and h img denote the width and height of the input image of the target detection task, i = 1, 2, 3, …, N, and the calculation is sequentially performed in a clockwise order, and the concentric circle expression is constructed according to formula (3), a total of N concentric circles, and the calculation is as follows: wherein i = 1, 2, 3,..., N, are recorded in turn in clockwise order, denoted as [pd_c1, pd_c2,..., pd_cN]. N ] Third step: the center coordinate true value and N-point contour coordinates in the true value information are expressed by concentric circles, a total of N concentric circles, specifically: Step 301, according to the target category label true value information, build the concentric circle expression of the true value data, and the calculation method is as follows: wherein i = 1, 2, 3,..., N, are recorded in turn in clockwise order, denoted as [gt cl, gt c2,..., gt c N ]; Fourth step: calculate the intersection over union loss of the predicted concentric circle and the true value concentric circle, and calculate the loss of the category information based on cross entropy, specifically: Step 401, use the predicted concentric circle and the true value concentric circle expression record results of step 202 and step 301 to build a single loss function, and the calculation method is as follows: Loss i = 1 - GIOU cir (pd_c i , gt_c i ) (6) wherein GIOU cir The calculation is as follows: where S_c denotes the input predicted concentric circle pd_c i with the maximum circumscribed circle area of gt_c i ; IOU cir The calculation is: where r min and r max denote the smaller and larger radius values of the prediction and ground truth concentric circles with the same index number, respectively, dist cen denotes the Euclidean distance between the centers of the prediction and ground truth concentric circles with the same index number, and t1and t2are calculated as follows: Step 402: Calculate the true class and the predicted class using the standard cross-entropy loss method. p The numerical loss is denoted as Loss. cls Regarding the Loss in step 401 i Let i = 1, 2, 3, ..., N, and record them in clockwise order, for a total of N positional sub-loss terms, denoted as [Loss1, Loss2, ..., Loss]. N The category loss and location loss terms are combined to form the final loss term, denoted as [Loss1, Loss2, ..., Loss]. N Loss cls There are a total of N+1 sub-loss terms; Fifth step: introduce the multi-task regression idea, evaluate the task complexity of N+1 loss items, use the gradient information in the training process to establish the multi-task regression loss weight, specifically: Step 501, record loss weights [w1, w2, …, w N ,w cls ] corresponding to N+1 sub-loss terms, and the global loss function of the regression process is: wherein w i is a weight coefficient representing the position loss term; Step 502, use formula (10) as the final global loss function to perform back propagation and learning training on the deep learning network model; After completion, input the image to be detected into the trained deep learning network model to perform target detection, and output the target category; In step 501 w i The calculation method is: wherein ratio_pos i represents the change rate of each term in the position loss function during the training process, ratio cls represents the change rate of the class loss function during the training process, and T represents a temperature coefficient; ratio_pos i and ratio cls are calculated as follows: wherein t represents the current time, t-1 represents the time of the previous round of training at the current time, w cls represents the weight coefficient of the category loss term, and the calculation method is:

2. The method of claim 1, wherein, T is taken as 100.

Images