GIS equipment inner surface metal particle detection method based on improved Faster RCNN
By improving the bounding box regression loss function of the Faster RCNN network, the problem of low detection accuracy of traditional Faster RCNN in the detection of metal particles on the inner surface of GIS is solved, and high-precision and fast particle detection is achieved.
Patent Information
- Application Number
- CN202311665850.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-06
- Publication Date
- 2026-04-10
AI Technical Summary
Traditional Faster RCNN has low detection accuracy in detecting metal particles on the inner surface of GIS, mainly because the bounding box regression loss function does not consider the overlap between the anchor box and the ground truth box.
We improve the Faster RCNN network by extracting feature maps using ResNet-50 and modifying the bounding box regression loss function of the RPN model to generate candidate boxes. We also utilize the generalized intersection-union loss function (GIoU loss) to improve detection accuracy.
It improves the accuracy and speed of metal particle detection, reduces the false negative rate, and is suitable for real-time detection scenarios.
Smart Images
Figure CN121837685A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of image processing and computer vision, and relates to a GIS inner surface metal particle detection method based on an improved Faster RCNN. BACKGROUND
[0002] Target detection is one of the most important computer vision tasks, which deals with the detection of visual instances of a certain class of objects in a cluttered real-world scene or input image. Due to its wide range of applications, target detection has attracted great attention in recent years. Target detection mainly includes two tasks: target positioning and target classification. Object positioning determines the location and scale by drawing a bounding box around one or more object instances. Classification refers to the process of assigning a class label to the object. In terms of detection, the target detection system builds a model from a set of training data, and in terms of generalization, it needs to provide a large training data set.
[0003] As a specific application of target detection, the specific process of GIS inner surface metal particle detection is as follows: given an input image or video frame, determine whether there are particles in the image, and if so, frame the location of the particles.
[0004] In the past decade, artificial intelligence has had an impact on all aspects of human life, and deep learning is an artificial intelligence field that uses artificial neural networks for representation learning. The application of deep learning has dominated the field of target detection. Currently, the mainstream target detection algorithm based on deep learning can be divided into two categories: Two-Stage algorithm based on candidate frame and One-Stage algorithm based on regression. Two-Stage algorithm based on candidate frame mainly includes RCNN, Fast RCNN, Faster RCNN and Mask RCNN, etc. One-Stage algorithm based on regression mainly includes Yolo series and SSD. Two-Stage target detection algorithm can fully learn the characteristics of the target due to the prior acquisition of candidate frame, and has high detection accuracy and positioning accuracy, but the network structure is complex, the calculation amount is large, and the detection speed is slow, which is not suitable for application scenarios with high real-time requirements. One-Stage target detection algorithm has a simple structure and can directly process the input image, so it has fast detection speed and can be applied to real-time detection, but the detection accuracy of small targets and multiple target objects is low. SUMMARY
[0005] The purpose of the present application is to provide a GIS inner surface metal particle detection method based on an improved Faster RCNN, the bounding box regression loss of the traditional Faster RCNN network RPN model only considers the gap between the anchor box and the true value box coordinate value, without considering the overlap between them, resulting in low target detection accuracy of the Faster RCNN, in order to overcome this shortcoming, the present application provides an inner surface metal particle detection method based on an improved Faster RCNN, which improves the accuracy of particle detection.
[0006] The technical solution adopted by the present application is a GIS inner surface metal particle detection method based on an improved Faster RCNN, which first extracts the feature map of the sample image through the ResNet-50 neural network, then inputs the obtained feature map into the RPN model, and modifies the bounding box regression loss function of the RPN model to generate candidate boxes; finally, the feature map and the candidate boxes are sent to the ROIHead model to obtain the category and positioning of the target; the specific operation steps are as follows:
[0007] Step 1: pre-train the ResNet-50 network to extract the feature map of the metal particle image;
[0008] Step 2: generate candidate boxes on the feature map of the image using the RPN model to obtain 1:1 ratio of positive and negative samples; use the positive and negative samples as labels to train the RPN model to obtain the category and preliminary positioning information of the target in the candidate box, the category including foreground and background;
[0009] Step 3: use the RPN model to obtain the category, preliminary positioning information of the target in the candidate box, and the feature map obtained by the ResNet-50 network, and train the ROI Head model obtained by randomly initializing parameters to obtain the category and positioning of the target.
[0010] Step 1 is as follows:
[0011] The VOC2007 dataset is used to pre-train the ResNet-50 neural network to obtain the network weight, the pre-trained weight is loaded into the ResNet-50 network of the Faster RCNN to obtain the pre-trained ResNet-50 network; the training process after that freezes the ResNet-50 network part, that is, the parameters of the ResNet-50 network do not perform back propagation and gradient update; the image is input into the pre-trained ResNet-50 network to obtain the feature map of the inner surface metal particle image.
[0012] The method for generating candidate boxes in step 2 is as follows:
[0013] Set the RPN model initialization information, which includes the size of the anchor box, the scale; the anchor box size of the RPN model is: 8x8, 16x16, 32x32, wherein 8x8, 16x16 is suitable for small metal particle detection, and 32x32 is suitable for large metal particle detection; each of the three sizes is scaled by 1:1, 1:2, and 2:1 in length and width ratio, a total of 9 sizes as the candidate frame that the RPN model needs to evaluate; the target of training the RPN model is to predict whether the 9 anchor boxes corresponding to each anchor point in the feature map are a frame with a target; if the intersection over union IoU of the frame and the true value frame is greater than 0.7, it is considered that the frame is a candidate frame, otherwise, it is not;
[0014] Input the feature map obtained in step 1 into the RPN model, improve the bounding box regression loss of the RPN model, and finally generate a candidate frame.
[0015] The specific method for training the RPN model is as follows:
[0016] A binary classification label (the binary classification label is {0, 1}) is assigned to each anchor box, wherein 0 represents a negative sample and 1 represents a positive sample; if the intersection over union IoU of an anchor box with one of all true value frames is greater than 0.7, it is called a positive sample; if the intersection over union IoU of an anchor box with all true value frames is less than 0.3, it is called a negative sample; the remaining frames are neither positive samples nor negative samples and are not used for final training; the positioning of the true value frame and the candidate frame output by the regression is compared, and the RPN model is trained by using the gradient descent method;
[0017] Wherein, a training batch mini-batch is composed of all positive and negative samples finally selected from an image, wherein the proportion of positive and negative samples is 1:1; i represents the i-th anchor point in a mini-batch, p i represents the probability that the i-th anchor point corresponds to the anchor box is a target, the value is between [0, 1]; the positive sample is 1 and the negative sample is 0; bi represents the positioning information of the anchor box, which is a classification loss function, and a bounding box regression loss function; only the positive sample is subjected to the bounding box regression operation; the classification and regression operations respectively output the probability pi that the candidate frame is a target and the positioning information bi of the candidate frame, which are respectively normalized by N cls and N reg and the balance weight λ; N cls is the size of the mini-batch, and N reg is the number of anchor points; the anchor point is a pixel point on the feature map output by the ResNet-50 network; the anchor box is a candidate frame corresponding to each anchor point;
[0018] Step 3 is specifically as follows:
[0019] The feature map output by the ResNet-50 network and the candidate box output by the RPN model are sent to the ROI Head network, and the ROI Head network comprises a region of interest (ROI) pooling layer, a fully connected (FC) layer, a bounding box regressor and a Softmax classifier; the ROI pooling layer is used for converting candidate boxes of different sizes into candidate boxes of a fixed size to be input into the FC layer; the bounding box regressor is used for determining the positioning of the candidate box; and the Softmax classifier is used for determining the specific category of the target in the candidate box.
[0020] The GIS inner surface metal particle detection method based on the improved Faster RCNN provided by the application adopts the current popular and efficient Faster RCNN framework and is improved to complete the metal particle detection work, is based on CNN features, can process images of any scale, and has high detection precision. BRIEF DESCRIPTION OF DRAWINGS
[0021] Figure 1 are schematic diagrams of different stages;
[0022] Figure 2 is a flowchart of the application. DETAILED DESCRIPTION
[0023] The GIS inner surface metal particle detection method based on the improved Faster RCNN provided by the application is implemented based on the deep learning framework PyTorch and comprises the following three steps:
[0024] 1. Pre-training the ResNet-50 network to extract the feature map of the metal particle image;
[0025] 2. Generating a candidate box on the feature map of the image by using the RPN model to obtain positive and negative samples in a 1:1 ratio, using the positive and negative samples as labels to train the RPN model to obtain the category and preliminary positioning information of the target in the candidate box, and the category comprising foreground and background;
[0026] 3. Training the ROI Head model obtained by using the RPN model to obtain the category, preliminary positioning and feature map of the target in the candidate box and the ResNet-50 network to obtain the category and positioning of the target.
[0027] Further, the specific steps of step 1 are as follows:
[0028] 1.1, prepare the self-made micro-particle dataset, load the pre-trained network weight of ResNet-50 neural network to the ResNet-50 network of Faster RCNN, and obtain the pre-trained ResNet-50 network. The subsequent training process freezes the ResNet-50 network part, that is, the parameters of the ResNet-50 network are not back propagated and have no gradient update. The image is input into the pre-trained ResNet-50 network to obtain a feature map.
[0029] Further, the specific steps of step 2 are as follows:
[0030] 2.1, set the size, ratio and other initialization information of the RPN model anchor box. The anchor box size of the RPN model is: 8x8, 16x16, 32x32, wherein 8x8 and 16x16 are suitable for small metal particle detection, and 32x32 is suitable for large metal particle detection; each of the three sizes is scaled by a length-width ratio of 1:1, 1:2 and 2:1, and a total of 9 sizes are used as candidate boxes to be evaluated by the RPN model; the purpose of training the RPN model is to predict whether the 9 anchor boxes corresponding to each anchor point in the feature map are target boxes; if the intersection over union IoU of the box and the true value box is greater than 0.7, the box is considered to be a candidate box, otherwise it is not. The feature map obtained in step S11 is input into the RPN model to improve the bounding box regression loss of the RPN model, so as to improve the detection and recognition accuracy, and finally generate a candidate box.
[0031] 2.2, training RPN: in order to train the RPN, a binary classification label (the binary classification label is {0, 1}) is assigned to each anchor box, wherein 0 represents a negative sample and 1 represents a positive sample; if the intersection over union IoU of an anchor box with all true value boxes is greater than 0.7, the anchor box is called a positive sample; if the intersection over union IoU of an anchor box with all true value boxes is less than 0.3, the anchor box is called a negative sample; the remaining boxes are neither positive samples nor negative samples and are not used for final training; the positioning of the true value box and the candidate box output by regression is compared, and the gradient descent method is used to train the RPN model.
[0032] The bounding box regression loss function of the improved Faster RCNN detection network RPN model adopts a generalized intersection over union loss GIoU loss, and the intersection over union IoU is added as a constraint condition for RPN training. Therefore, the goal of training the RPN is to find the candidate box with the maximum intersection over union IoU with the true value box, instead of only considering the difference between the coordinate values, so as to improve the detection accuracy.
[0033] 3.1, set the number of output units of the Faster RCNN detection network according to the number of categories of the training data set, and initialize the weight parameters of the detection network. The feature map output by the ResNet-50 network and the candidate box output by the RPN model are sent to the ROI Head network, which includes a region of interest ROI pooling layer, a fully connected FC layer, a bounding box regressor and a Softmax classifier; the region of interest ROI pooling layer is used to convert candidate boxes of different sizes into candidate boxes of fixed size to pass into the fully connected FC layer; the bounding box regressor is used to determine the positioning of the candidate box; and the Softmax classifier is used to determine the specific category of the target in the candidate box.
[0034] So far, the training phase of the improved Faster RCNN-based internal surface metal particle detection method provided by the application is completed.
[0035] The network structure of the GIS device internal surface metal particle detection method based on the improved Faster RCNN will be described below.
[0036] 1. Pre-training the ResNet-50 network to extract the feature map of the metal particle image: prepare the VOC2007 data set for the pre-trained network weight of the ResNet-50 network, load the pre-trained weight into the ResNet-50 network of the Faster RCNN, and obtain the pre-trained ResNet-50 network. The subsequent training process freezes the ResNet-50 network part, that is, the parameters of the ResNet-50 network do not perform back propagation and do not have gradient update. The image is input into the pre-trained ResNet-50 network to obtain a feature map.
[0037] The RPN model is used to generate a candidate box on the feature map of the image to obtain positive and negative samples in a 1:1 ratio. The positive and negative samples are used as labels to train the RPN model to obtain the category and preliminary positioning information of the target in the candidate box, and the category includes foreground and background;
[0038] 2 Set the size, scale, etc. initialization information of the RPN model anchor box. The anchor box size of the RPN model is: 8x8, 16x16, 32x32, wherein 8x8, 16x16 are suitable for small metal particle detection, and 32x32 is suitable for large metal particle detection; each of the three sizes is scaled by 1:1, 1:2, and 2:1 in length and width ratio, for a total of 9 sizes as candidate boxes that need to be evaluated by the RPN; the goal of the RPN is to predict whether the 9 anchor boxes corresponding to each anchor point in the feature map are target boxes; if the intersection over union IoU of the box and the true value box is greater than 0.7, the box is considered to be a candidate box, otherwise it is not. The feature map obtained in step S11 is input into the RPN model, and the bounding box regression loss of the RPN model is improved to improve the detection accuracy, and finally the candidate box is generated.
[0039] Training the RPN model: In order to train the RPN model, a binary classification label (binary classification label is {0, 1}) is assigned to each anchor box, where 0 represents a negative sample and 1 represents a positive sample; if the intersection over union IoU of an anchor box with any one of the true value boxes is greater than 0.7, it is called a positive sample; if the intersection over union IoU of an anchor box with all true value boxes is less than 0.3, it is called a negative sample; the remaining boxes are neither positive samples nor negative samples and are not used for final training; the positioning of the true value box and the candidate box output by the regression is compared, and the gradient descent method is used to train the RPN model.
[0040] 1 Use the RPN model to obtain the class, preliminary positioning of the target in the candidate box, and the feature map obtained by the ResNet-50 network to train the ROI Head model obtained by randomly initializing the parameters to obtain the class and positioning of the target.
[0041] 2 Set the number of output units of the Faster RCNN detection network according to the number of classes in the training data set, and initialize the weight parameters of the detection network. Send the feature map output by the ResNet-50 network and the candidate box output by the RPN model to the ROI Head network, which includes a region of interest ROI pooling layer, a fully connected FC layer, a bounding box regressor, and a Softmax classifier; the region of interest ROI pooling layer is used to convert candidate boxes of different sizes into fixed-size candidate boxes to pass into the fully connected FC layer; the bounding box regressor is used to determine the positioning of the candidate box; the Softmax classifier is used to determine the specific class of the target in the candidate box.
[0042] We input the pictures taken in the actual scene into the improved Faster RCNN network to detect the metal particles in the pictures, achieving ideal results.
[0043] So far, the test stage of the GIS inner surface metal particle detection method based on the improved Faster RCNN is completed.
[0044] In conclusion, the GIS inner surface metal particle detection method based on the improved Faster RCNN is disclosed, and the training method is mainly described.
[0045] Firstly, the pre-trained weight of the ResNet-50 network on the self-made particle data set is loaded as the initial value of the parameter of the ResNet-50 network to generate a feature map, so that a good initial value of the model parameter is obtained, and the network convergence is accelerated while avoiding falling into a poor local minimum. The RPN region generation model obtained by random initialization parameters is trained, the bounding box regression loss function of the RPN model is improved, the measurement standard IoU is directly used as the loss function for training, and the accuracy of the recognition candidate frame is better improved. The class (foreground or background) and positioning feature of the target in the candidate frame are obtained by using the RPN model to generate the candidate frame on the feature map of the image. Finally, the ROI Head model obtained by random initialization parameters is trained by using the class, positioning feature and feature map of the target, and the specific class and positioning of the target are obtained.
[0046] The GIS inner surface metal particle detection method based on the improved Faster RCNN disclosed by the application is based on the CNN feature, can process images of any scale, has high detection speed, and the improvement mainly considers the application scene, data distribution, and prevention of overfitting. Meanwhile, the method disclosed by the application does not need to be specially designed for the network, fully utilizes the existing available data, and still can achieve good experimental effect by using the general network structure, fully plays the advantages of the deep convolutional network, has the advantages of simple design, good robustness, high detection accuracy, and low missed detection rate.
Claims
1. A method for detecting metal particles on the inner surface of GIS equipment based on an improved Faster RCNN, characterized in that, First, feature maps are extracted from the sample images using a ResNet-50 neural network. Then, the obtained feature maps are input into the RPN model, and the bounding box regression loss function of the RPN model is modified to generate candidate boxes. Finally, the feature maps and candidate boxes are sent to the ROI Head model to obtain the target's category and location. The specific operation steps are as follows: Step 1: Pre-train the ResNet-50 network to extract feature maps of metal particles on the inner surface of GIS equipment; Step 2: Use the RPN model to generate candidate boxes on the feature map of the image to obtain positive and negative samples in a 1:1 ratio; use the positive and negative samples as labels to train the RPN model to obtain the category and preliminary localization information of the target in the candidate box, where the category includes foreground and background; Step 3: Use the RPN model to obtain the target category, preliminary localization information, and feature map obtained by the ResNet-50 network in the candidate box. Train the ROI Head model obtained by randomly initializing parameters to obtain the target category and localization.
2. The method for detecting metal particles on the inner surface of GIS equipment based on the improved Faster RCNN as described in claim 1, characterized in that, Step 1 is as follows: The ResNet-50 neural network was pre-trained using a self-built particle dataset to obtain network weights. The pre-trained weights were then loaded into the ResNet-50 network of Faster RCNN to obtain the pre-trained ResNet-50 network. During the subsequent training process, the ResNet-50 network was frozen, meaning that the parameters of the ResNet-50 network were not backpropagated and gradient updates were not performed. The image was then input into the pre-trained ResNet-50 network to obtain the feature maps of the metal particles.
3. The method for detecting metal particles on the inner surface of GIS equipment based on improved Faster RCNN according to claim 1, characterized in that, The specific method for generating candidate boxes in step 2 is as follows: The RPN model initialization information is set, including the anchor box size and ratio. The anchor box sizes of the RPN model are 8×8, 16×16, and 32×32, where 8×8 and 16×16 are suitable for detecting small metal particles, and 32×32 is suitable for detecting large metal particles. Each of these three sizes is scaled at a ratio of 1:1, 1:2, and 2:1, resulting in a total of 9 sizes as candidate boxes to be evaluated by the RPN model. The goal of training the RPN model is to predict whether each of the 9 anchor boxes corresponding to each anchor point in the feature map is a box containing a target. A box with an Intersection over Union (IoU) > 0.7 is considered a candidate box, and otherwise not. The feature map obtained in step 1 is input into the RPN model to improve the bounding box regression loss of the RPN model, thereby improving the detection and recognition accuracy, and finally generating candidate boxes.
4. The method for detecting metal particles on the inner surface of GIS equipment based on the improved Faster RCNN according to claim 3, characterized in that, The specific method for training the RPN model is as follows: A binary classification label is assigned to each anchor box, where 0 represents a negative sample and 1 represents a positive sample. If an anchor box has an IoU greater than 0.7 with any of the ground truth boxes, it is considered a positive sample; if an anchor box has an IoU less than 0.3 with any of the ground truth boxes, it is considered a negative sample. The remaining boxes are neither positive nor negative samples and are not used for final training. The ground truth boxes are compared with the locations of the candidate boxes output by the regression, and the RPN model is trained using gradient descent. The loss function for training the RPN model is defined as follows: A training mini-batch consists of all the positive and negative samples ultimately selected from an image, with a 1:1 ratio of positive to negative samples; i represents the i-th anchor point in a mini-batch, and p... i This represents the probability that the anchor box corresponding to the i-th anchor point is the target, with a value between [0,1]; 1 for positive samples and 0 for negative samples; b i This indicates the positioning information of the anchor box. This is the classification loss function, and this is the bounding box regression loss function; it indicates that bounding box regression is performed only on positive samples; the classification and regression operations respectively output the probability p that the candidate box is the target. i and the location information of the candidate box b i These two items are respectively determined by N cls and N reg And the normalization of the balancing weights λ, N cls Nr is the size of the mini-batch. eg The number of anchor points; an anchor point is a pixel on the feature map output by the ResNet-50 network; the anchor box is the candidate box corresponding to each anchor point. The classification loss function is defined as follows: The bounding box regression loss function is defined as follows:
5. The method for detecting metal particles on the inner surface of GIS equipment based on improved Faster RCNN according to claim 1, characterized in that, Step 3 is as follows: The feature map output by the ResNet-50 network and the candidate boxes output by the RPN model are sent to the ROI Head network. The ROI Head network includes a Region of Interest (ROI) pooling layer, a fully connected (FC) layer, a bounding box regressor, and a Softmax classifier. The ROI pooling layer is used to convert candidate boxes of different sizes into candidate boxes of a fixed size so that they can be fed into the fully connected (FC) layer. The bounding box regressor is used to determine the location of the candidate boxes. The Softmax classifier is used to determine the specific category of the target in the candidate box.