A target detection method based on non-maximum suppression threshold optimization

Abstract

The application relates to the technical field of target detection, and discloses a target detection method based on non-maximum suppression threshold optimization, which comprises the following steps: obtaining target data to be detected; inputting the target data to be detected into a pre-trained target detection model to obtain a plurality of target detection frames and the confidence of each target detection frame; obtaining the visibility ratio of each target detection frame; inputting the confidence and the visibility ratio of each target detection frame into a pre-trained non-maximum suppression threshold prediction model to obtain corresponding non-maximum suppression thresholds; and performing redundant frame screening on all target detection frames according to the non-maximum suppression thresholds to obtain a target detection result. The application can solve the problem that a fixed threshold set manually cannot cope with dynamically changing target images or video sequences, can significantly improve the problems of missed detection and false detection caused by manual setting of a fixed threshold, and is beneficial to improving the accuracy of target detection in a heavy occlusion scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to a target detection method based on non-maximum suppression threshold optimization. BACKGROUND

[0002] Target detection is to detect target images or video sequences, judge whether there is a target and accurately locate each target instance. However, in open crowded scenes such as schools and stations, and in non-structured scenes such as human cavities, blood vessels and nerve-intensive in laparoscopic surgery scenes, the heavily overlapping occlusions between dense targets (such as crowds, multiple surgical instruments and multiple tissues and organs) make the detector unable to distinguish the accurate detection boundaries between each target, resulting in a large number of redundant and erroneous detection boxes in the detection process, causing false detection, missed detection and low positioning accuracy in the target detection of serious occlusion scenes.

[0003] In the existing target detection method, the traditional non-maximum suppression method or the softening non-maximum suppression method is usually used to eliminate redundant detection boxes. However, both of these two non-maximum suppression methods use a manually set non-maximum suppression threshold, and the setting of the threshold will seriously affect the removal effect of the detection box. When the threshold is set too low, due to the high overlap degree (i.e. intersection over union) between the targets in the heavily occluded scene, many correct detection boxes will be removed as redundant boxes, resulting in serious missed detection; when the threshold is set too high, the false positive detection boxes with high overlap degree with the real boxes cannot be completely correctly eliminated, resulting in a large number of false detections. In addition, since the occlusion degree of the target in each image or each frame of the video sequence is dynamically changing, the manually set fixed threshold cannot meet the dynamic changes.

[0004] Therefore, how to avoid the problems of missed detection, false detection and low positioning accuracy caused by the manually set fixed threshold is still a problem that needs to be solved in the current target detection method. SUMMARY

[0005] Therefore, the present application aims to overcome the problems of missed detection, false detection and low positioning accuracy caused by the manually set fixed threshold in the existing non-maximum suppression method, and to provide a target detection method based on non-maximum suppression threshold optimization.

[0006] In order to solve the above problems, the present application provides a target detection method based on non-maximum suppression threshold optimization, comprising:

[0007] obtaining target data to be detected;

[0008] inputting the target data to be detected into a pre-trained target detection model to obtain a plurality of target detection boxes and the confidence of each target detection box.

[0009] obtaining a visible ratio of each of the target bounding boxes;

[0010] inputting the confidence and the visible ratio of each of the target bounding boxes into a pre-trained non-maximum suppression threshold prediction model to obtain a corresponding non-maximum suppression threshold;

[0011] performing redundant bounding box screening on all of the target bounding boxes according to the non-maximum suppression threshold to obtain a target detection result.

[0012] Optionally, the performing redundant bounding box screening on all of the target bounding boxes according to the non-maximum suppression threshold to obtain a target detection result comprises:

[0013] constructing a candidate bounding box set according to all of the target bounding boxes, and constructing an effective bounding box set; the effective bounding box set is an empty set;

[0014] obtaining a target bounding box with the highest confidence from the candidate bounding box set as an anchor point box, and transferring the anchor point box to the effective bounding box set;

[0015] obtaining an intersection-over-union of the anchor point box and the remaining target bounding boxes in the candidate bounding box set;

[0016] detecting whether the intersection-over-union is less than or equal to the non-maximum suppression threshold corresponding to the anchor point box;

[0017] if yes, retaining the anchor point box in the effective bounding box set, and returning to the step of obtaining a target bounding box with the highest confidence from the candidate bounding box set as an anchor point box, and transferring the anchor point box to the effective bounding box set when the candidate bounding box set is an empty set;

[0018] until it is detected that the candidate bounding box set is an empty set, determining the final effective bounding box set as a target detection result.

[0019] Optionally, before the obtaining of the target data to be detected, the method further comprises:

[0020] obtaining a target data set, and dividing the target data set into a training subset, a validation subset and a test subset;

[0021] constructing a target detection model, and training and optimizing the target detection model by using the training subset;

[0022] constructing a visible ratio evaluation model, the visible ratio evaluation model being defined as an area ratio between a visible bounding box of a target in a target bounding box and all bounding boxes of the target;

[0023] Constructing a non-maximum suppression threshold prediction model, and using the validation subset, the confidence of the target detection box obtained by the optimized target detection model, and the visible ratio of the target detection box obtained by the visible ratio evaluation model to train and optimize the non-maximum suppression threshold prediction model;

[0024] Using the test subset to test the accuracy of the optimized target detection model and the non-maximum suppression threshold prediction model;

[0025] Whether the detection accuracy meets the preset application conditions;

[0026] If satisfied, it enters the application phase.

[0027] Optionally, the target detection model includes a backbone network, a region candidate network and a classification network; the backbone network is used to extract feature maps, the region candidate network is used to generate target detection frames based on the feature maps, and the classification network is used to calculate the confidence of the target detection frames.

[0028] Optionally, the optimizing the target detection model by training the training subset includes:

[0029] Inputting the target data in the training subset into a target detection model, extracting a feature map of the target data through the target detection model, inputting the feature map into a region candidate network to obtain a target detection frame, and simultaneously inputting the feature map into a classification network to obtain a confidence score of the target detection frame;

[0030] The gradient descent algorithm is used to minimize the bounding box regression loss, and the parameters of the target detection model are updated to obtain an optimized target detection model. The parameter update process of the target detection model is as follows:

[0031] ,

[0032] in, 、 are the parameters of the target detection model for the next iteration and the current iteration respectively, For parameters The learning rate during update, The target detection model is used to detect the target data The label prediction value of is the true value of the label, is the bounding box regression loss function.

[0033] Optionally, the non-maximum suppression threshold prediction model includes three fully connected layers and an activation function layer; the fully connected layer is used to obtain the confidence and visibility ratio of the target detection box, and the activation function layer is used to output the non-maximum suppression threshold.

[0034] Optionally, the training optimizes the non-maximum suppression threshold prediction model, comprising:

[0035] According to a target optimization function for constructing a non-maximum suppression threshold prediction model; the target optimization function is defined as searching for parameters that can make the accuracy of the filtered target detection results reach the highest for the preliminary detection results output by the target detection model, denoted as:

[0036] ,

[0037] Wherein, is the accuracy, is the preliminary detection result output by the optimized target detection model, is the parameter of the non-maximum suppression threshold prediction model, is a conventional non-maximum suppression function, is a validation set;

[0038] Using a target optimization algorithm based on a reward mechanism, the parameters of the non-maximum suppression threshold prediction model are updated in a gradient ascent manner to obtain an optimized non-maximum suppression threshold prediction model; the parameter updating process of the non-maximum suppression threshold prediction model is:

[0039] ,

[0040] Wherein, , are the parameters of the non-maximum suppression threshold prediction model in the next iteration and the current iteration respectively, is the parameter updated, is the learning rate, is the accuracy and the gradient between the parameters.

[0041] Optionally, the constructing a visible ratio evaluation model comprises:

[0042] Detecting the target overlap type of the target detection frame;

[0043] When the target overlap type is two target overlaps, determining the main target and the interference target of the overlapping area, and calculating the visible ratio of the main target; the calculation formula of the visible ratio of the main target is:

[0044] ,

[0045] Wherein, is the visible ratio of the main target A, and are the detection frame of the main target A and the detection frame of the interference target B respectively;

[0046] According to the visibility ratio of the main target, a discount factor and a Heaviside function are introduced to construct a visibility ratio evaluation model. The visibility ratio evaluation model is:

[0047] ,

[0048] in, is the discount factor, is the Heaviside function, is an area function; the calculation formula of the discount factor is:

[0049] ,

[0050] in, is the similarity between the overlapping area I and the main target A, is the similarity between the overlapping area I and the interference target B, and ;

[0051] The Heaviside function is:

[0052] .

[0053] Optionally, the constructing of the visibility ratio evaluation model further includes:

[0054] When the target overlap type is three or more targets overlapping, a visibility ratio evaluation model is constructed according to the area ratio of each overlapping area, the discount factor, and the Heaviside function. The visibility ratio evaluation model is:

[0055] ,

[0056] in, For the overlapping areas, For the The similarity between the overlapping area and the interference target.

[0057] Optionally, when the target overlap type is two targets overlapping, determining the main target and the interference target in the overlapping area includes:

[0058] Get the similarity between the overlapping area and each target;

[0059] The target with higher similarity is determined as the main target in the overlapping area, and the target with lower similarity is determined as the interference target in the overlapping area.

[0060] The technical solutions provided by the embodiments of the present invention can have the following beneficial effects:

[0061] 1) The embodiments of the present invention utilize a non-maximum suppression threshold prediction model to adaptively select the optimal non-maximum suppression threshold based on the confidence level and visibility ratio of the target detection box. This avoids the inability of existing target detection methods, which use manually set fixed thresholds, to cope with dynamically changing target images or video sequences. Furthermore, the embodiments of the present invention can significantly improve the problems of missed detection and false detection caused by traditional non-maximum suppression methods, thereby improving the accuracy of target detection in heavily occluded scenes.

[0062] 2) The method for obtaining the target visibility ratio and the optimal non-maximum suppression threshold in the embodiment of the present invention does not require additional manual annotation for training, thus reducing the dependence on data and greatly improving the scope of application. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0064] Figure 1 A flowchart of a target detection method based on non-maximum suppression threshold optimization provided by one embodiment of the present invention is shown;

[0065] Figure 2 A flowchart of the model training and testing phase of a target detection method based on non-maximum suppression threshold optimization provided by one embodiment of the present invention is shown;

[0066] Figure 3 A flowchart of step S80 of a target detection method based on non-maximum suppression threshold optimization provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0067] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0068] like Figure 1 As shown in FIG, a flow chart of a target detection method based on non-maximum suppression threshold optimization provided by an embodiment of the present invention, the method includes the following steps:

[0069] S10, obtaining target data to be detected.

[0070] In step S10, the to-be-detected target data refers to a target image or a video sequence containing continuous multiple frames of images collected from an actual scene and requiring target detection.

[0071] S20, input the to-be-detected target data into a pre-trained target detection model to obtain multiple target detection boxes and the confidence of each target detection box.

[0072] In step S20, the target detection model can be a target detection model obtained by training YOLOV3 or FasterR-CNN.

[0073] Specifically, when the to-be-detected target data is a target image, the target image is recognized by the pre-trained target detection model to obtain multiple (i.e. two or more) target detection boxes and the confidence of each target detection box. When the to-be-detected target data is a video sequence, each frame of image in the video sequence is recognized by the pre-trained target detection model to obtain a preliminary detection result corresponding to each frame of image, which contains multiple target detection boxes and the confidence of each target detection box.

[0074] S30, obtain the visibility ratio of each target detection box.

[0075] In step S30, the visibility ratio of the target detection box refers to the area ratio between the visible bounding box of the target in the target detection box and the total bounding box of the target.

[0076] As a preferred, for each target detection box, the visibility ratio of the target detection box can be obtained by a pre-constructed visibility ratio evaluation model. The visibility ratio evaluation model is a visibility ratio evaluation model constructed by steps S801 to S804.

[0077] S40, input the confidence and the visibility ratio of each target detection box into a pre-trained non-maximum suppression threshold prediction model to obtain the corresponding non-maximum suppression threshold.

[0078] In step S40, the non-maximum suppression threshold prediction model refers to a neural network model taking the confidence and the visibility ratio as input parameters and taking the non-maximum suppression threshold as output parameters.

[0079] That is, for each target detection frame, the confidence obtained in step S10 is combined with the visibility ratio obtained in step S30, and then input into a pre-trained non-maximum suppression threshold prediction model to obtain the non-maximum suppression threshold of each target detection frame, that is, each target detection frame corresponds to a non-maximum suppression threshold. Compared with the non-maximum suppression method in the background art, the non-maximum suppression threshold prediction model used in the embodiment can obtain an adaptive non-maximum suppression threshold, which can avoid the problem of too large or too small non-maximum suppression threshold, thereby effectively filtering out redundant detection frames.

[0080] S50, according to the non-maximum suppression threshold, all the target detection frames are screened for redundant frames to obtain a target detection result.

[0081] As a preferred, the intersection over union of the target detection frame with the highest confidence and other target detection frames is obtained, and when the intersection over union is greater than or equal to the non-maximum suppression threshold output by the non-maximum suppression threshold prediction model, the target detection frame with the highest confidence is removed, and the foregoing steps are repeatedly executed to obtain the final target detection result.

[0082] In summary, the target detection method based on non-maximum suppression threshold optimization provided by the embodiment has the following beneficial effects:

[0083] 1) The non-maximum suppression threshold prediction model in the embodiment can adaptively select the optimal non-maximum suppression threshold according to the confidence and visibility ratio of the target detection frame, which avoids the problem that the existing target detection method cannot cope with dynamically changing target images or video sequences due to the use of manually set fixed thresholds, and can significantly improve the problems of missed detection and false detection caused by the traditional non-maximum suppression method, thereby improving the accuracy of target detection in heavy occlusion scenarios.

[0084] 2) The target visibility ratio and the optimal non-maximum suppression threshold in the embodiment are obtained without additional manual annotation for training, which reduces the dependence on data and greatly improves the scope of application.

[0085] In an optional embodiment, the step S50 specifically comprises the following steps:

[0086] S501, constructing a candidate detection frame set according to all the target detection frames, and constructing an effective detection frame set; the effective detection frame set is an empty set;

[0087] S502, obtaining the target detection frame with the highest confidence from the candidate detection frame set as an anchor frame, and transferring the anchor frame to the effective detection frame set;

[0088] S503, obtaining the intersection-over-union of the anchor box and the rest of the target detection boxes in the candidate detection box set;

[0089] S504, detecting whether the intersection-over-union is less than or equal to the non-maximum suppression threshold corresponding to the anchor box;

[0090] S505, if yes, retaining the anchor box in the valid detection box set, and returning to step S502 when the candidate detection box set is empty, i.e., obtaining the target detection box with the highest confidence as the anchor box from the candidate detection box set, and transferring the anchor box to the valid detection box set;

[0091] S506, determining the final valid detection box set as the target detection result until the candidate detection box set is detected to be empty.

[0092] In the embodiment, first, an empty set (i.e., a valid detection box set) is established to save valid target detection boxes, and a candidate detection box set is established to save target detection boxes recognized by the target detection model, next, the target detection box with the highest confidence is obtained from the candidate detection box set and marked as an anchor box , and transferred to the valid detection box set , then the intersection-over-union of the anchor box and the rest of the target detection boxes in the candidate detection box set is calculated , and the intersection-over-union is compared with the non-maximum suppression threshold output by the non-maximum suppression threshold prediction model of the anchor box , if , the anchor box is removed from the valid detection box set , otherwise, the anchor box is retained in the valid detection box set .

[0093] Then, it is detected whether the candidate detection box set is empty, if the candidate detection box set is not empty, the above steps are repeatedly executed, otherwise, the detection boxes in the valid detection box set are taken as the final valid target detection boxes.

[0094] It can be understood that, in the suppression threshold post-processing stage of the embodiment, for any one target detection frame of the preliminary detection result, the IOU of the target detection frame with other target detection frames and the non-maximum threshold corresponding to the target detection frame are compared, so that the effective target detection frame can be obtained, the problems of missed detection and false detection can be effectively avoided, and the target detection accuracy can be improved.

[0095] In an optional embodiment, as shown in Figure 2 the target detection method based on non-maximum suppression threshold optimization further includes a model training and testing stage, and the model training and testing stage specifically includes the following steps:

[0096] S60, obtaining a target data set and dividing the target data set into a training subset, a verification subset and a test subset.

[0097] In step S60, the target data set contains a preset first number of target images or a preset second number of video sequences, and the preset first number and the preset second number are set as needed. The training subset is used to train the target detection model, the verification subset is used to train the non-maximum suppression threshold prediction model, and the test subset is used to test the target detection model and the non-maximum suppression threshold prediction model.

[0098] Preferably, step S60 specifically includes the following steps:

[0099] First, according to the target data collected in multiple actual scenes, a target data set is constructed , and the target data is a target image or a video sequence.

[0100] Then, the target data set is divided into a training subset , a verification subset and a test subset according to a preset ratio.

[0101] In the embodiment, the actual scenes include but are not limited to open and crowded scenes such as schools and stations, and unstructured scenes such as human cavities, blood vessels and nerve-intensive scenes in laparoscopic surgery scenes. The preset ratio is set as needed.

[0102] More specifically, when the target data is a target image, 750 target images in different application scenarios can be collected to construct a target data set, and the target data set is divided into a training subset, a verification subset and a test subset according to a ratio of 2:2:1. When the target data is a video sequence, the video sequence can be converted into an image sequence by using a video sequence frame extraction method, and then the constructed image data set is processed.

[0103] S70, constructing a target detection model and training and optimizing the target detection model by using the training subset.

[0104] In this embodiment, the target detection model is a target detection model based on Fast R-CNN, which includes a backbone network, a region candidate network and a classification network; the backbone network is used to extract feature maps, the region candidate network is used to generate target detection frames based on the feature maps, and the classification network is used to calculate the confidence of the target detection frames.

[0105] Preferably, in step S70, training and optimizing the target detection model using the training subset includes the following steps:

[0106] First, the target data in the training subset is input into the target detection model, a feature map of the target data is extracted by the target detection model, the feature map is input into the region candidate network to obtain the target detection frame, and the feature map is input into the classification network to obtain the confidence of the target detection frame;

[0107] Then, the gradient descent algorithm is used to minimize the bounding box regression loss, and the parameters of the target detection model are updated to obtain an optimized target detection model. The parameter update process of the target detection model is as follows:

[0108] ,

[0109] in, 、 are the parameters of the target detection model for the next iteration and the current iteration respectively, For parameters The learning rate during update, The target detection model is used to detect the target data The label prediction value of is the true value of the label, is the bounding box regression loss function.

[0110] S80: Construct a visibility ratio evaluation model. The visibility ratio evaluation model is defined as the area ratio between the visible border of the target in the target detection frame and the entire border of the target, which can be expressed as:

[0111] ,

[0112] in, is the visible ratio, is the visible bounding box of the target, is the entire bounding box of the target (including the unobstructed and obscured parts), is the area function.

[0113] S90, constructing a non-maximum suppression threshold prediction model, and training and optimizing the non-maximum suppression threshold prediction model by using the confidence of the target detection frame obtained by the optimized target detection model and the visibility ratio of the target detection frame obtained by the visibility ratio evaluation model.

[0114] In the embodiment, the non-maximum suppression threshold prediction model is a threshold prediction model based on a full connection neural network, and the non-maximum suppression threshold prediction model comprises three full connection layers and an activation function layer; the full connection layers are used to obtain the confidence and the visibility ratio of the target detection frame, and the activation function layer is used to output the non-maximum suppression threshold.

[0115] It can be understood that the non-maximum suppression threshold prediction model can be expressed as , wherein , are the visibility ratio and the confidence corresponding to the target detection frame , respectively, is a parameter of the non-maximum suppression threshold prediction model. In the target detection process, the non-maximum suppression threshold prediction model can be embedded between the output stage of the target detection model and the non-maximum suppression post-processing stage, wherein the output stage of the target detection model comprises the step S20, and the non-maximum suppression post-processing stage comprises the step S50. In addition, the confidence output head of the target detection model and the visibility ratio evaluation model will be embedded before the non-maximum suppression threshold prediction model.

[0116] Further, the activation function adopted by the activation function layer is a ReLU function.

[0117] As a preferred, the step S90 of training and optimizing the non-maximum suppression threshold prediction model specifically comprises the following steps:

[0118] Firstly, according to the target optimization function for constructing the non-maximum suppression threshold prediction model; the target optimization function is defined as searching the parameter capable of making the accuracy of the filtered detection result reach the highest for the preliminary detection result output by the target detection model, and can be expressed as:

[0119] ,

[0120] wherein, is the accuracy, is the preliminary detection result (i.e. the candidate detection frame set comprising multiple target detection frames) output by the optimized target detection model, is the parameter of the non-maximum suppression threshold prediction model, is a conventional non-maximum suppression function, is the verification set.

[0121] Then, the target optimization algorithm based on the reward mechanism is used to update the parameters of the non-maximum suppression threshold prediction model in a gradient ascent manner to obtain an optimized non-maximum suppression threshold prediction model; the parameter updating process of the non-maximum suppression threshold prediction model is as follows:

[0122] ,

[0123] in, 、 are the parameters of the non-maximum suppression threshold prediction model for the next iteration and the current iteration respectively, For parameters The learning rate during update, For accuracy With parameters The gradient between.

[0124] Furthermore, due to the accuracy With parameters There is no analytical function expression between It cannot be directly calculated, so this embodiment uses the chain rule to calculate the gradient The derivation is divided into two steps, and the derivation formula is:

[0125] ,

[0126] Among them, the second Non-maximum suppression threshold prediction model and parameters The gradient of this part is due to the non-maximum suppression threshold prediction model and parameter The functional relationship is explicit and can be automatically obtained using deep learning tools. For accuracy The gradient of the non-maximum suppression threshold prediction model can be estimated using the proximal gradient. The calculation formula is:

[0127] ,

[0128] in, The mean is , the variance is Gaussian distribution under The sampled threshold, is the expected function, is the sampling threshold The probability distribution of is the sampling threshold The accuracy under the sampling threshold rewards.

[0129] Further, since all the sampling thresholds cannot be analytically obtained in practice , the embodiment utilizes Monte Carlo sampling to approximate the calculation of the expected reward, and the calculation formula is as follows:

[0130]

[0131] wherein, p (x) is the probability density function of the sampling threshold , which can be directly used to represent the sampling probability of the sampling threshold .

[0132] Understandably, the target detection model and the non-maximum suppression threshold prediction model of the embodiment can be trained by using the training subset and the verification subset respectively, and when the two models respectively reach the corresponding convergence conditions or reach the corresponding maximum number of updates, it is determined that the model training optimization is completed.

[0133] In other embodiments, the target detection model and the non-maximum suppression threshold prediction model can be trained simultaneously, at which time, after the target data set is divided into a training subset for training the two models and a test subset for testing the two models, the target data in the training subset is input into the target detection model, and then the confidence of each target detection frame output by the target detection model and the visibility ratio of each target detection frame output by the visibility ratio evaluation model are combined and input into the non-maximum suppression threshold prediction model, and when it is detected that the current number of iterations reaches the maximum number of iterations, it is determined that the model training optimization is completed, otherwise, the above steps are repeatedly executed to continue training the model.

[0134] S110, testing the accuracy of the optimized target detection model and the non-maximum suppression threshold prediction model by using the test subset.

[0135] In the embodiment, for the target data in the test subset, the corresponding real target label of the target data is obtained by pre-labeling in an artificial manner. In the testing stage, the target data in the test subset is first input into the optimized target detection model to obtain the preliminary detection result containing multiple target detection frames and the confidence of each target detection frame, and then the confidence of the target detection frame output by the target detection model and the visibility ratio output by the visibility ratio evaluation model are combined and input into the optimized non-maximum suppression threshold prediction model to obtain the non-maximum suppression threshold corresponding to each target detection frame, and then the preliminary detection result is screened for redundant frames by using the non-maximum suppression threshold to obtain the final target detection result, and finally the accuracy of the model is calculated according to the effective target detection frame in the target detection result and the real target label.

[0136] S120, detecting whether the accuracy meets the preset application condition. ​​

[0137] In the embodiment, the preset application condition is an accuracy threshold, which can be set according to the minimum detection error allowed by the actual scene.

[0138] S130, if yes, entering the application stage.

[0139] That is, if the accuracy is greater than or equal to the accuracy threshold, entering the application stage, which includes the steps S10 to S50; otherwise, adjusting the structure and parameters of the target detection model and the non-maximum suppression threshold prediction model according to the accuracy, and returning to step S70 to retrain the target detection model and the non-maximum suppression threshold prediction model until the accuracy is greater than or equal to the accuracy threshold, and entering the application stage.

[0140] It can be understood that, in the model test training stage, the target detection model is trained by using the training set, and the non-maximum suppression threshold prediction model is optimized by using the target optimization algorithm based on the reward mechanism, which can improve the model precision.

[0141] In some optional embodiments, as shown in Figure 3 The step S80, i.e., the constructing the visible ratio evaluation model, includes the following steps:

[0142] S801, detecting the target overlap type of the target detection frame. The target overlap type includes two target overlaps and three or more target overlaps.

[0143] S802, when the target overlap type is two target overlaps, determining the main target and the interference target in the overlap region, and calculating the visible ratio of the main target. The visible ratio calculation formula of the main target is:

[0144] ,

[0145] Wherein, is the visible ratio of the main target A, and are the detection frame of the main target A and the detection frame of the interference target B, respectively.

[0146] As a preferred, in the step S6033, when the target overlap type is two target overlaps, the main target and the interference target in the overlap region are determined, which specifically includes the following steps: obtaining the similarity between the overlap region and each target; determining the target with higher similarity as the main target in the overlap region, and determining the target with lower similarity as the interference target in the overlap region.

[0147] That is, for two targets mutually occluding each other, the similarity of the overlapping region to each target is determined, if the similarity of the overlapping region to the first target is greater than the similarity of the overlapping region to the second target, it is determined that the overlapping region belongs to the first target, and the first target is marked as the main target, otherwise, it is determined that the overlapping region belongs to the second target, and the second target is marked as the main target.

[0148] S803, according to the visible ratio of the main target, a discount factor and a Huber function are introduced to construct a visible ratio evaluation model, which is:

[0149] ,

[0150] wherein, is a discount factor, is a Huber function, is an area function; the calculation formula of the discount factor is:

[0151] ,

[0152] wherein, is the similarity of the overlapping region I to the main target A, is the similarity of the overlapping region I to the interference target B, and ;

[0153] The Huber function is:

[0154] .

[0155] It can be understood that when constructing the visible ratio evaluation model, a discount factor needs to be multiplied by the area ratio in the visible ratio calculation formula , which can eliminate the influence of the part of the overlapping region that does not belong to the target.

[0156] Further, as Figure 3 shown, the step S80 further includes the following steps:

[0157] S804, when the target overlapping type is three or more target overlapping, according to the area ratio, the discount factor and the Huber function of each overlapping region, a visible ratio evaluation model is constructed, which is:

[0158] ,

[0159] wherein, is the i-th overlapping region, is the similarity of the i-th overlapping region to the interference target.

[0160] ​​It should be noted that three or more targets overlap is a general case of practical application scenarios.

[0161] It can be understood that the embodiment uses the area ratio, the discount factor and the Haversine function to construct the visible ratio evaluation model, which can eliminate the influence of the part not belonging to the target in the overlapping area and improve the evaluation accuracy of the visible ratio.

[0162] It should be noted that in the description of the present application, the terms "first", "second" and the like are only used for the purpose of description and should not be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise stated, the meaning of "a plurality of" is at least two.

[0163] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above disclosed technical content without departing from the scope of the technical solution of the present application, and any brief introduction, modification, equivalent change and modification of the above embodiment made according to the technical essence of the present application still belongs to the scope of the technical solution of the present application.

Claims

1. A target detection method based on non-maximum suppression threshold optimization, characterized in that, The method comprises the following steps: acquiring target data to be detected, the target data to be detected being image data; Before acquiring the target data to be detected, the method comprises the following steps: acquiring a target data set and dividing the target data set into a training subset, a verification subset and a test subset; constructing a target detection model and training and optimizing the target detection model by using the training subset; constructing a visible ratio evaluation model, the visible ratio evaluation model being defined as the area ratio between the visible bounding box of a target in a target detection box and the total bounding box of the target; constructing a non-maximum suppression threshold prediction model and training and optimizing the non-maximum suppression threshold prediction model by using the verification subset, the confidence of the target detection box obtained by the optimized target detection model and the visible ratio of the target detection box obtained by the visible ratio evaluation model; testing the accuracy of the optimized target detection model and the non-maximum suppression threshold prediction model by using the test subset; detecting whether the accuracy meets preset application conditions; if yes, entering an application stage; inputting the target data to be detected into the pre-trained target detection model to obtain a plurality of target detection boxes and the confidence of each target detection box; acquiring the visible ratio of each target detection box; inputting the confidence and the visible ratio of each target detection box into the pre-trained non-maximum suppression threshold prediction model to obtain a corresponding non-maximum suppression threshold; performing redundant box screening on all the target detection boxes according to the non-maximum suppression threshold to obtain a target detection result; the redundant box screening according to the non-maximum suppression threshold to obtain the target detection result comprises the following steps: constructing a candidate detection box set according to all the target detection boxes and constructing an effective detection box set; the effective detection box set is an empty set; acquiring a target detection box with the highest confidence from the candidate detection box set as an anchor box and transferring the anchor box to the effective detection box set; acquiring the intersection-over-union of the anchor box and the remaining target detection boxes in the candidate detection box set; detecting whether the intersection-over-union is less than or equal to the non-maximum suppression threshold corresponding to the anchor box; if yes, retaining the anchor box in the effective detection box set and returning to the step of acquiring a target detection box with the highest confidence from the candidate detection box set as an anchor box and transferring the anchor box to the effective detection box set when the candidate detection box set is an empty set; until the candidate detection box set is detected to be an empty set, determining the final effective detection box set as the target detection result.

2. The non-maximum suppression threshold value optimization based object detection method of claim 1, wherein, The target detection model comprises a backbone network, a region candidate network and a classification network; the backbone network is used for extracting a feature map, the region candidate network is used for generating a target detection box according to the feature map and the classification network is used for calculating the confidence of the target detection box.

3. The non-maximum suppression threshold value optimization based object detection method of claim 2, wherein, the training and optimization of the target detection model by using the training subset comprises the following steps: inputting target data in the training subset into a target detection model, extracting a feature map of the target data by the target detection model, inputting the feature map into a region proposal network to obtain a target detection frame, and simultaneously inputting the feature map into a classification network to obtain a confidence of the target detection frame; minimizing the bounding box regression loss by using a gradient descent algorithm, updating parameters of the target detection model, and obtaining an optimized target detection model; the parameter updating process of the target detection model is as follows: ， wherein, , are parameters of the target detection model at the next iteration and the current iteration, respectively, is a parameter is a learning rate at the time of updating, is a label prediction value of the target detection model for target data , is a label true value, is a bounding box regression loss function.

4. The non-maximum suppression threshold value optimization based object detection method of claim 1, wherein, The non-maximum suppression threshold prediction model comprises three full connection layers and an activation function layer; the full connection layers are used to obtain the confidence of the target detection frame and the visibility ratio, and the activation function layer is used to output the non-maximum suppression threshold.

5. The non-maximum suppression threshold value optimization based object detection method of claim 4, wherein, The training optimizes the non-maximum suppression threshold prediction model, including: According to a target optimization function for constructing the non-maximum suppression threshold prediction model; the target optimization function is defined as searching for parameters capable of making the accuracy of the filtered target detection result reach the highest for the preliminary detection result output by the target detection model, and is expressed as: ， wherein, is the accuracy, is the preliminary detection result output by the optimized target detection model, is a parameter of the non-maximum suppression threshold prediction model, is a conventional non-maximum suppression function, is a verification set; The target optimization algorithm based on the reward mechanism is used to update the parameters of the non-maximum suppression threshold prediction model in a gradient ascent manner, and an optimized non-maximum suppression threshold prediction model is obtained; the parameter updating process of the non-maximum suppression threshold prediction model is as follows: ， wherein, , are parameters of the non-maximum suppression threshold prediction model at the next iteration and the current iteration, respectively, is a parameter is a learning rate at the update is an accuracy is a gradient between the parameter and the parameter 6. The non-maximum suppression threshold value optimization based object detection method of claim 1, wherein, The construction of the visibility ratio evaluation model comprises: detecting a target overlap type of the target detection frame; when the target overlap type is two target overlaps, determining a main target and an interference target of an overlap region and calculating a visibility ratio of the main target; the calculation formula of the visibility ratio of the main target is as follows: ， wherein, is the visible ratio of the main target A, and are the detection frame of the main target A and the detection frame of the interference target B, respectively; According to the visibility ratio of the main target, a discount factor and a Huber function are introduced to construct a visibility ratio evaluation model, and the visibility ratio evaluation model is as follows: ， wherein is a discount factor, is a Heaviside function, is an area function; the discount factor being calculated as: ， wherein, is a similarity of the overlap region I to the main target A, is a similarity of the overlap region I to the interfering target B, and ; The Huber function is as follows: 。 7. The non-maximum suppression threshold value optimization based object detection method of claim 6, wherein, The construction of the visibility ratio evaluation model further comprises: when the target overlap type is three or more target overlaps, according to an area ratio of each overlap region, a discount factor and a Huber function, a visibility ratio evaluation model is constructed, and the visibility ratio evaluation model is as follows: ， wherein is the th overlapping region, is the th overlapping region and the similarity of the interference target.

8. The non-maximum suppression threshold value optimization based object detection method of claim 6, wherein, When the target overlap type is two target overlaps, the method comprises: obtaining a similarity between the overlap region and each target; determining a target with a higher similarity as the main target in the overlap region, and determining a target with a lower similarity as the interference target in the overlap region.

Images