Power scene anomaly detection method and device based on difficult sample mining
By generating and iterating candidate boxes using a hard sample mining method and training a classification model using a power scene image dataset, the problem of poor algorithm performance in substation anomaly detection is solved, and more efficient anomaly detection is achieved.
Patent Information
- Application Number
- CN202310707028.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-14
- Publication Date
- 2026-01-06
- Estimated Expiration
- 2043-06-14
AI Technical Summary
Existing deep learning-based target detection algorithms perform poorly in substation anomaly detection scenarios, especially due to the differences in image data distribution and the insignificant defect features of substation equipment in substation business scenarios, making it difficult to effectively identify abnormal targets in complex environments.
By using a hard sample mining method, candidate boxes are generated and their types are determined. Iterative operations and loss ranking are performed. By using the weight constraints of classification loss and bounding box loss, hard samples are generated to train the classification model, thereby improving the anomaly detection effect.
This improved the accuracy and efficiency of substation anomaly detection by increasing the proportion of difficult-to-classify samples and decreasing the proportion of easy-to-classify samples, thereby enhancing the value of training samples and improving the training effect of the classification model.
Smart Images
Figure CN116704175B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer data processing technology, and in particular to a method and apparatus for detecting anomalies in power scenarios based on hard sample mining. Background Technology
[0002] With the gradual expansion of the power system, safe and efficient power transmission and transformation projects are particularly important for social stability and people's livelihood. Due to the wide coverage and complex operating environment of my country's power system, the possibility of substation anomalies is increased. Therefore, it is urgent to leverage algorithms to conduct safety inspections of substation systems, identifying abnormal substation scenarios in a timely manner to ensure the stable operation of the substation system.
[0003] With the rapid development of cloud computing and deep learning in recent years, many scholars have proposed general-purpose deep learning-based object detection algorithms based on public datasets. However, the application results of these algorithms in substation anomaly detection scenarios are still unsatisfactory. Firstly, the distribution of image data in substation business scenarios differs significantly from that of general public datasets. For example, in complex environments, substation equipment captured by inspection robots or monitoring cameras varies in angle and size. Secondly, some substation equipment defects have insignificant features that are difficult for algorithms to identify, such as damaged / blurred markings or broken silicone cylinders. These factors all contribute to the poor performance of general-purpose object detection algorithms in power business scenarios. Summary of the Invention
[0004] Therefore, it is necessary to provide a power scene anomaly detection method and device based on difficult sample mining that can improve the detection effect of abnormal targets, in order to address the above-mentioned technical problems.
[0005] Firstly, this application provides a method for anomaly detection in power scenarios based on hard sample mining. The method includes:
[0006] Obtain a dataset of power scene images; the power scene image dataset includes anomalous targets;
[0007] Several candidate bounding boxes for abnormal targets are generated based on the power scene image dataset, and the type of each candidate bounding box is determined; the candidate bounding box types include positive samples and negative samples.
[0008] Perform iterative operations on the candidate boxes: propagate the candidate boxes output from the previous iteration forward and obtain the target loss corresponding to the candidate boxes output from the previous iteration; sort the target losses and select candidate boxes output from the candidate boxes output from the previous iteration based on the sorting results and the preset ratio of positive and negative samples; propagate the candidate boxes output from the current iteration backward, update the target parameters and share them in the forward and backward propagation processes, and perform the next iteration operation based on the target parameters;
[0009] When the iteration reaches the preset condition, the iteration operation ends and the corresponding candidate box output of the last iteration is used as a difficult sample to train the classification model. The classification model is then used to detect anomalies in the power scene.
[0010] In one embodiment, the target loss includes a classification loss and a bounding box loss; the target parameters include the corresponding weights of the classification loss and the corresponding weights of the bounding box loss, and there is a constraint between the corresponding weights of the classification loss and the corresponding weights of the bounding box loss that the corresponding weight of the classification loss is not less than the corresponding weight of the bounding box loss.
[0011] In one embodiment, several candidate bounding boxes for abnormal targets are generated based on a power scene image dataset, and the type of each candidate bounding box is determined, including:
[0012] A high-resolution feature extractor is used to extract features from a power scene image dataset to obtain image feature maps;
[0013] The RPN module is used to generate several candidate boxes based on the image feature map; the corresponding type of the candidate box is determined by obtaining the RPN loss function corresponding to each candidate box.
[0014] In one embodiment, performing an iteration operation on the candidate boxes includes:
[0015] Forward and backward propagation are performed using ROI pooling layers and classification / regression layers;
[0016] The ROI pooling layer is used to divide and pool the feature maps corresponding to candidate boxes that are propagated forward or backward, and obtain feature maps of a preset size.
[0017] The classification and regression layer is used to obtain category and location information from feature maps of a preset size, and to make category judgments based on the category information and adjust the localization of candidate boxes for forward or backward propagation based on the location information.
[0018] In one embodiment, the target loss is sorted, and candidate boxes for the current iteration are selected from the candidate boxes output in the previous iteration based on the sorting results and a preset ratio of positive to negative samples.
[0019] Sort the target losses in descending order;
[0020] Use Soft NMS to obtain the confidence scores of the candidate boxes output in the previous iteration;
[0021] Candidate boxes for this iteration are selected from the candidate boxes output in the previous iteration based on confidence level, ranking results, and the ratio of positive to negative samples.
[0022] In one embodiment, when the iteration reaches a condition that is met, the iteration operation ends and the corresponding candidate box output from the last iteration is used as a hard sample to train a classification model. The classification model is then used for power scene anomaly detection, which includes:
[0023] The classification model is trained using GIOU loss.
[0024] Secondly, this application provides a power scene anomaly detection device based on hard sample mining, characterized in that the device includes:
[0025] The data acquisition module is used to acquire power scene image datasets; the power scene image datasets include abnormal targets;
[0026] The candidate box generation module is used to generate several candidate boxes for abnormal targets based on the power scene image dataset, and to determine the type of each candidate box; the candidate box types include positive samples and negative samples;
[0027] The iteration module is used to perform iterative operations on the candidate boxes: it forward-propagates the candidate boxes output from the previous iteration and obtains the target loss corresponding to the candidate boxes output from the previous iteration; it sorts the target losses and selects the candidate boxes output from the candidate boxes output from the previous iteration based on the sorting results and the preset ratio of positive and negative samples; it back-propagates the candidate boxes output from the current iteration, updates the target parameters and shares them in the forward and back propagation processes, and performs the next iteration operation based on the target parameters.
[0028] The training module is used to end the iteration operation when the preset conditions are met, and use the corresponding candidate box output of the last iteration as a hard sample to train the classification model, and use the classification model to detect anomalies in power scenarios.
[0029] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to perform the following steps:
[0030] Obtain a dataset of power scene images; the power scene image dataset includes anomalous targets;
[0031] Several candidate bounding boxes for abnormal targets are generated based on the power scene image dataset, and the type of each candidate bounding box is determined; the candidate bounding box types include positive samples and negative samples.
[0032] Perform iterative operations on the candidate boxes: propagate the candidate boxes output from the previous iteration forward and obtain the target loss corresponding to the candidate boxes output from the previous iteration; sort the target losses and select candidate boxes output from the candidate boxes output from the previous iteration based on the sorting results and the preset ratio of positive and negative samples; propagate the candidate boxes output from the current iteration backward, update the target parameters and share them in the forward and backward propagation processes, and perform the next iteration operation based on the target parameters;
[0033] When the iteration reaches the preset condition, the iteration operation ends and the corresponding candidate box output of the last iteration is used as a difficult sample to train the classification model. The classification model is then used to detect anomalies in the power scene.
[0034] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, performs the following steps:
[0035] Obtain a dataset of power scene images; the power scene image dataset includes anomalous targets;
[0036] Several candidate bounding boxes for abnormal targets are generated based on the power scene image dataset, and the type of each candidate bounding box is determined; the candidate bounding box types include positive samples and negative samples.
[0037] Perform iterative operations on the candidate boxes: propagate the candidate boxes output from the previous iteration forward and obtain the target loss corresponding to the candidate boxes output from the previous iteration; sort the target losses and select candidate boxes output from the candidate boxes output from the previous iteration based on the sorting results and the preset ratio of positive and negative samples; propagate the candidate boxes output from the current iteration backward, update the target parameters and share them in the forward and backward propagation processes, and perform the next iteration operation based on the target parameters;
[0038] When the iteration reaches the preset condition, the iteration operation ends and the corresponding candidate box output of the last iteration is used as a difficult sample to train the classification model. The classification model is then used to detect anomalies in the power scene.
[0039] Fifthly, this application provides a computer program product, including a computer program, characterized in that, when executed by a processor, the computer program performs the following steps:
[0040] Obtain a dataset of power scene images; the power scene image dataset includes anomalous targets;
[0041] Several candidate bounding boxes for abnormal targets are generated based on the power scene image dataset, and the type of each candidate bounding box is determined; the candidate bounding box types include positive samples and negative samples.
[0042] Perform iterative operations on the candidate boxes: propagate the candidate boxes output from the previous iteration forward and obtain the target loss corresponding to the candidate boxes output from the previous iteration; sort the target losses and select candidate boxes output from the candidate boxes output from the previous iteration based on the sorting results and the preset ratio of positive and negative samples; propagate the candidate boxes output from the current iteration backward, update the target parameters and share them in the forward and backward propagation processes, and perform the next iteration operation based on the target parameters;
[0043] When the iteration reaches the preset condition, the iteration operation ends and the corresponding candidate box output of the last iteration is used as a difficult sample to train the classification model. The classification model is then used to detect anomalies in the power scene.
[0044] The aforementioned power scene anomaly detection method and apparatus based on hard sample mining generates candidate boxes for anomaly targets based on a power scene image dataset and determines whether the candidate boxes are positive or negative samples. The candidate boxes are forward-propagated to obtain a target loss; the target loss is sorted, and candidate boxes are selected based on the target loss, while the ratio of positive to negative samples is constrained by a preset ratio; the selected candidate boxes are backward-propagated to update the target parameters, and the updated target parameters are shared in both the forward and backward propagation processes; based on the target parameters, the selected candidate boxes are forward-propagated again, iteratively updating the target parameters; when the iteration reaches the preset condition, the iteration ends, and the candidate boxes output from the last iteration are used as hard samples to train a classification model. The trained classification model is then used for power scene anomaly detection. This application continuously updates the target parameters through iterative operations until the preset condition is met, thus obtaining hard samples. The classification model trained with hard samples can increase the proportion of difficult samples and decrease the proportion of easy samples, thereby increasing the value of the training samples and improving the training effect of the classification model. Attached Figure Description
[0045] Figure 1 This is an application environment diagram of a power scene anomaly detection method based on hard sample mining in one embodiment;
[0046] Figure 2 This is a flowchart illustrating a power scene anomaly detection method based on hard sample mining in one embodiment.
[0047] Figure 3 This is a schematic diagram illustrating the process of generating and iterating candidate boxes in one embodiment;
[0048] Figure 4 This is a flowchart illustrating a high-resolution feature extractor in one embodiment;
[0049] Figure 5This is a structural block diagram of a power scene anomaly detection device based on hard sample mining in one embodiment;
[0050] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0052] The power scene anomaly detection method based on hard sample mining provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, and tablets. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.
[0053] In one embodiment, such as Figure 2 As shown, a method for anomaly detection in power scenarios based on hard sample mining is provided, and this method is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:
[0054] Step 202: Obtain the power scene image dataset; the power scene image dataset includes abnormal targets.
[0055] Power scenarios include substations, transmission lines, and power plants. Power scenario image datasets can be obtained through inspection robots, drones, and fixed cameras. Anomalies in power scenarios can be categorized into three main types: equipment defects, abnormal personnel behavior, and abnormal equipment status. Equipment defects include broken respirator oil seals, blurred or damaged meters, broken insulators, oil stains on the ground, broken silicone sleeves, abnormal door closure, hanging debris, bird nests, and damaged covers. Abnormal personnel behavior includes smoking, not wearing work clothes, and not wearing safety helmets. Abnormal equipment status includes abnormal respirator oil seal levels and discolored silicone seals. These anomalies in power scenarios are considered anomalous targets. Collecting images containing anomalous targets and organizing them into a dataset yields a power scenario image dataset.
[0056] In this embodiment, the power scene image dataset also includes anomaly markers for abnormal targets, which include the type and location of the abnormal targets.
[0057] Step 204: Generate several candidate boxes for abnormal targets based on the power scene image dataset, and determine the type of each candidate box; the candidate box types include positive samples and negative samples.
[0058] Methods for generating candidate bounding boxes include sliding window and selective search. The sliding window method uses windows of different sizes and aspect ratios to slide across the image from left to right and from top to bottom, classifying these windows to find candidate bounding boxes for the entire image. The window size and stride of the sliding window method have a significant impact on the candidate bounding box search results. Selective search improves efficiency by searching the regions in the image most likely to contain the target. First, the input image is segmented to generate many small regions (e.g., 2000). Based on the similarity of these small regions (color, texture, size, etc.), a sub-region merging method is used to perform region algebraic merging, generating circumscribed rectangles, which are the candidate bounding boxes. Selective search improves computational efficiency and can include candidate bounding boxes of various sizes.
[0059] After generating candidate bounding boxes, it is necessary to determine whether the candidate boxes are positive or negative samples. Positive samples are those containing foreground images, i.e., candidate boxes containing anomalous targets; negative samples are those containing only background images and no foreground images, i.e., candidate boxes not containing anomalous targets. The type of candidate box is determined by the labels of anomalous targets in the power scene image dataset.
[0060] Step 206: Perform iterative operations on the candidate boxes: propagate the candidate boxes output from the previous iteration forward and obtain the target loss corresponding to the candidate boxes output from the previous iteration; sort the target losses and select candidate boxes output from the candidate boxes output from the previous iteration based on the sorting results and the preset ratio of positive and negative samples; propagate the candidate boxes output from the current iteration backward, update the target parameters and share them in the forward and backward propagation processes, and perform the next iteration operation based on the target parameters.
[0061] In each iteration, forward propagation only performs forward inference, selects candidate boxes from the inference results, and then performs backpropagation on the selected candidate boxes to obtain the target parameters and update the gradient.
[0062] In this embodiment, when selecting candidate boxes, the candidate boxes with larger target losses are selected based on the ranking results, and the ratio between positive samples and negative samples in the selected candidate boxes is maintained to reduce the impact of imbalance between positive and negative samples.
[0063] Step 208: When the iteration reaches the preset condition, the iteration operation ends and the corresponding candidate box output of the last iteration is used as a difficult sample to train the classification model. The classification model is then used to detect anomalies in the power scene.
[0064] The candidate boxes output by the iterative process that meet the preset conditions have both a large target loss and relatively accurate positions. Therefore, they can be used as difficult samples to train the classification model and improve the training effect of the classification model.
[0065] In the aforementioned power scene anomaly detection method based on hard sample mining, candidate boxes for anomaly targets are generated from a power scene image dataset, and it is determined whether the candidate boxes are positive or negative samples. Forward propagation is performed on the candidate boxes to obtain the target loss; the target loss is sorted, and candidate boxes are selected based on the target loss, while the ratio of positive to negative samples is constrained by a preset ratio; the selected candidate boxes are then backpropagated to update the target parameters, and the updated target parameters are shared in both the forward and backpropagation processes; based on the target parameters, the selected candidate boxes are forward propagated again, iteratively updating the target parameters; when the iteration reaches the preset condition, the iteration ends, and the candidate boxes output from the last iteration are used as hard samples to train a classification model. The trained classification model is then used for power scene anomaly detection. This application continuously updates the target parameters through iterative operations until the preset condition is met, thus obtaining hard samples. The classification model trained with hard samples can increase the proportion of difficult samples and decrease the proportion of easy samples, thereby increasing the value of the training samples and improving the training effect of the classification model.
[0066] In one embodiment, the target loss includes a classification loss and a bounding box loss; the target parameters include the corresponding weights of the classification loss and the corresponding weights of the bounding box loss, and there is a constraint between the corresponding weights of the classification loss and the corresponding weights of the bounding box loss that the corresponding weight of the classification loss is not less than the corresponding weight of the bounding box loss.
[0067] The loss includes two main components: classification loss (cls_loss) and bounding box loss (box_loss). The classification loss measures the difference between the model's predicted bounding box and the true bounding box, helping to ensure the model can accurately locate objects.
[0068] The target loss can be expressed as a weighted sum of the classification loss and the bounding box loss, where the weight of the classification loss should not be less than the weight of the bounding box loss. In this embodiment, the weights of the classification loss and the bounding box loss are learnable weights. For example, a weight parameter 'a' is set, constrained to 0.5-1. The weight of the bounding box loss is 'a', and the weight of the classification loss is 1-'a'.
[0069] like Figure 3As shown, in one embodiment, step 204 includes: using a high-resolution feature extractor to extract features from the power scene image dataset to obtain an image feature map; using the RPN module to generate several candidate boxes based on the image feature map; and determining the corresponding type of the candidate box by obtaining the RPN loss function corresponding to each candidate box.
[0070] Among them, the high-resolution feature extractor (Deep High Resolution Net, HRNet) is as follows: Figure 4 As shown in the diagram. First, stage 1 constructs a high-resolution sub-network. Subsequent stages progressively add high-to-low resolution sub-networks, connecting the multi-resolution sub-networks in parallel. Multi-scale fusion is guided by exchanging information throughout the parallel multi-resolution sub-networks, and this process is repeated continuously. Throughout the processing, the high-resolution representation of the extracted features is maintained. The acquired image feature maps are represented in matrix form, thus obtaining the feature vector matrix.
[0071] The Region Proposal Network (RPN) module generates and classifies candidate boxes. Within the RPN module, a softmax function is used to perform binary classification on each generated candidate box, determining whether it belongs to the foreground and labeling it as a positive sample; otherwise, it is labeled as a negative sample. Then, regression is used to adjust the position and size of the foreground candidate boxes to more closely approximate the true foreground region. Finally, the adjusted foreground candidate boxes are input into the iterative process. In this embodiment, the binary classification calculation yields the RPN loss function, which can be IOU (Intersection over Union).
[0072] In this embodiment, in order to achieve a wider coverage and detect more objects, for each point in the feature map, a transformation is performed based on anchor boxes of three different sizes and three different scale transformations, resulting in nine candidate boxes in the original image area.
[0073] In one embodiment, step 206 includes: performing forward and backward propagation using an ROI pooling layer and a classification and regression layer; the ROI pooling layer is used to divide and pool the feature maps corresponding to the candidate boxes for forward or backward propagation to obtain feature maps of a preset size; the classification and regression layer is used to obtain category information and location information from the feature maps of the preset size, and to make category judgments based on the category information, and to adjust the positioning of the candidate boxes for forward or backward propagation based on the location information.
[0074] like Figure 3As shown, both forward and backward propagation use the same network structure: a connection between an ROI (Region of Interest) pooling layer and a classification / regression layer. The classification / regression layer consists of two fully connected layers. The structure connecting the ROI pooling layer and the classification / regression layer only performs forward inference and does not update the parameters. In one iteration, the forward propagation result from the upper layer is fed into a hard sample sampler to obtain hard samples with a fixed ratio of positive to negative samples. These samples are then used for backpropagation through the structure connecting the ROI pooling layer and the classification / regression layer to update the target parameters, and consequently, the gradient.
[0075] The ROI pooling layer divides the feature maps corresponding to the foreground candidate boxes of varying sizes selected by the RPN module into a fixed-size grid, and then performs pooling operations to obtain feature maps of a preset size. This feature map serves as... Figure 3 The input for the difficult sample mining section is shown.
[0076] The classification and regression layer uses the feature map output from the ROI pooling layer to obtain features containing both category and location information through two fully connected layers. These features are then used to determine the category of candidate regions (i.e., the anomaly category in the power scene), and the candidate bounding boxes are repositioned to improve the accuracy of anomaly detection and localization.
[0077] In one embodiment, the step 206 of sorting the target loss and selecting candidate boxes for the current iteration from the candidate boxes output in the previous iteration based on the sorting results and a preset ratio of positive and negative samples includes: sorting the target loss in descending order; obtaining the confidence level corresponding to the candidate boxes output in the previous iteration using Soft NMS; and selecting candidate boxes for the current iteration from the candidate boxes output in the previous iteration based on the confidence level, the sorting results, and the ratio of positive and negative samples.
[0078] Figure 3 The difficult sampler shown performs the steps in this embodiment.
[0079] Soft NMS adds an IOU function to the classic NMS (Non-Maximum Suppression). This function mainly suppresses the confidence Si of each box. For candidate boxes with low confidence, the confidence will fall below the confidence threshold after suppression. For candidate boxes with high confidence, the confidence will remain high even after suppression, and they will be retained in the end.
[0080] As can be seen from the above, the IOU in Soft-NMS is mainly used to suppress the confidence of candidate boxes. This embodiment uses a Gaussian function to suppress the confidence. Specifically, the larger the IOU, the greater the impact on Si, making Si smaller, thus updating the Si value of each candidate box. Finally, candidate boxes are selected by retaining those with Si greater than the confidence threshold.
[0081] In this embodiment, each positive sample is first sorted in descending order according to the class confidence. Then, the bounding box with the highest confidence in the class is selected and retained. When the intersection-union ratio (IOU) between the remaining boxes and the candidate box with the highest confidence is greater than the threshold, a Gaussian weighting function is used to decay the score.
[0082] The formula for score decay using the Gaussian weighting function is:
[0083]
[0084] Where Si represents the confidence score of the i-th candidate box, N represents the manually set threshold, Max represents the candidate box with the highest confidence score, and B represents the remaining candidate boxes.
[0085] This embodiment filters redundant candidate boxes using confidence scores to obtain valid candidate boxes. It then uses the ranking results of the target loss to select candidate boxes with higher training value. Finally, it avoids the problem of imbalanced positive and negative samples by limiting the ratio of positive to negative samples.
[0086] In one embodiment, step 208 includes training the classification model using GIOU loss.
[0087] First, calculate the area of the minimum closure region of the two candidate boxes. Then, calculate the proportion of the area of the closure region that does not belong to either candidate box. Next, calculate the Intersection over Union (IOU). Finally, subtract the proportion from the IOU to obtain the GIOU (generalized intersection over union) loss. Here, the minimum closure region is the smallest bounding rectangle of the two candidate boxes.
[0088] The GIOU loss takes values of [-1, 1]. It is 1 when the two values overlap and -1 when they do not overlap and are infinitely far apart. Therefore, it is a much better distance metric than IOU.
[0089] The algorithm flow of GIOU loss is shown in Table 1 below:
[0090] Table 1 Algorithm Flow: GIOU loss
[0091]
[0092] This method shows a significant performance improvement compared to directly using traditional Faster-R-CNN for power scene anomaly detection. The specific testing method is as follows:
[0093] Following step 202, a power scene image dataset of approximately 200 images per category was collected, totaling 2664 images. 2364 images were randomly selected as the training set, and the remaining 300 images were used as the test set.
[0094] The mean precision (mAP), representing the average precision (AP) for each target category, is used as the evaluation metric. AP is the area under the precision and recall curves of the classification model at different IOU thresholds. The formulas for precision and recall are as follows:
[0095]
[0096]
[0097] Where precision represents accuracy, recall represents recall, TP represents positive samples predicted as positive by the classification model, FP represents negative samples predicted as positive by the classification model, and FN represents positive samples predicted as negative by the classification model.
[0098] To demonstrate the effectiveness of the proposed method, ablation experiments were conducted on various improvements compared to the traditional Faster-R-CNN, as shown in Table 2. Table 2 shows that improvements in positive and negative sample mining strategies, regression loss functions, and candidate box deduplication strategies during the testing phase all yielded good results.
[0099] Table 2. Algorithm Comparison Experiment Results
[0100]
[0101] The reasons for the performance improvement shown in the above experiments are analyzed as follows:
[0102] 1) Substation systems exhibit a wide variety of abnormal scenarios. Different anomaly categories, due to varying degrees of feature distinctness, can lead to an imbalance in sample difficulty during classification model training. To address this, a positive and negative sample sampling strategy algorithm based on difficult sample mining is designed to rank samples by difficulty, allowing the model to prioritize sampling difficult samples and improve training efficiency.
[0103] 2) Traditional Faster R-CNN uses Smooth L1 Loss as the regression loss function. This function calculates the loss independently for the coordinates of the four vertices of the detection box and then sums them. This approach has two drawbacks: a) it does not utilize the correlation between vertices; b) in substation scenarios, there are targets of varying sizes, and the same vertex coordinate shift will result in drastically different IOUs for targets of different sizes. In power scenarios, using GIOU loss can more fully utilize the correlation between the predicted box vertices and handle the impact of vertex coordinate shifts on targets of different sizes.
[0104] 3) Traditional Faster R-CNN uses the traditional Non-Maximum Score (NMS) algorithm during inference. The traditional NMS algorithm first sorts each positive sample in descending order of class confidence, then selects the bounding box with the highest confidence within each class for retention, and then deletes the remaining boxes that are close to it based on a fixed IOU threshold. In power scene image datasets, very close adjacent targets often appear, and using the traditional NMS algorithm will lead to missed detections. Soft NMS uses a method of suppressing the scores of non-maximum boxes to retain the boxes missed by traditional NMS.
[0105] As can be seen from the above experiments, the difficult sample mining method based on power scenarios proposed in this application has a significant advantage in abnormal target detection compared with existing technologies, and improves the detection effect.
[0106] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0107] Based on the same inventive concept, this application also provides a power scene anomaly detection device based on hard sample mining for implementing the power scene anomaly detection method based on hard sample mining described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the power scene anomaly detection device based on hard sample mining provided below can be found in the limitations of the power scene anomaly detection method based on hard sample mining described above, and will not be repeated here.
[0108] In one embodiment, such as Figure 5 As shown, a power scene anomaly detection device based on hard sample mining is provided, including: a data acquisition module 502, a candidate box generation module 504, an iteration module 506, and a training module 508, wherein:
[0109] The data acquisition module 502 is used to acquire a power scene image dataset; the power scene image dataset includes abnormal targets.
[0110] The candidate box generation module 504 is used to generate several candidate boxes for abnormal targets based on the power scene image dataset, and to determine the type of each candidate box; the candidate box types include positive samples and negative samples.
[0111] The iteration module 506 is used to perform iterative operations on the candidate boxes: forward propagation of the candidate boxes output by the previous iteration and obtaining the target loss corresponding to the candidate boxes output by the previous iteration; sorting the target losses and selecting candidate boxes output by the current iteration from the candidate boxes output by the previous iteration according to the sorting results and the preset ratio of positive and negative samples; backward propagation of the candidate boxes output by the current iteration, updating the target parameters and sharing them in the forward and backward propagation processes, and performing the next iteration operation based on the target parameters.
[0112] The training module 508 is used to end the iteration operation when the preset conditions are met, and use the corresponding candidate box output of the last iteration as a hard sample to train the classification model, and use the classification model to perform power scene anomaly detection.
[0113] The target loss includes classification loss and bounding box loss; the target parameters include the corresponding weights of classification loss and bounding box loss, and there is a constraint between the corresponding weights of classification loss and bounding box loss that the corresponding weights of classification loss are not less than the corresponding weights of bounding box loss.
[0114] The candidate box generation module 504 is also used to extract features from the power scene image dataset using a high-resolution feature extractor to obtain image feature maps; generate several candidate boxes based on the image feature maps using the RPN module; and determine the corresponding type of the candidate boxes by obtaining the RPN loss function corresponding to each candidate box.
[0115] The iteration module 506 is also used to perform forward and backward propagation using ROI pooling layers and classification and regression layers; the ROI pooling layer is used to divide and pool the feature maps corresponding to the candidate boxes for forward or backward propagation to obtain feature maps of a preset size; the classification and regression layer is used to obtain category information and location information from the feature maps of the preset size, and to make category judgments based on the category information, and to adjust the positioning of the candidate boxes for forward or backward propagation based on the location information.
[0116] The iteration module 506 is also used to sort the target loss in descending order; obtain the confidence scores of the candidate boxes output by the previous iteration using Soft NMS; and select the candidate boxes output by the current iteration from the candidate boxes output by the previous iteration based on the confidence scores, the sorting results, and the ratio of positive to negative samples.
[0117] In training module 508, the classification model is trained using GIOU loss.
[0118] Each module in the aforementioned power scenario anomaly detection device based on hard sample mining can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.
[0119] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores a dataset of power scene images. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a power scene anomaly detection method based on hard sample mining.
[0120] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a power scene anomaly detection method based on hard sample mining.
[0121] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0122] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement all of the above-described method embodiments.
[0123] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements all of the above method embodiments.
[0124] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements all of the above method embodiments.
[0125] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0126] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0127] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0128] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for power scenario anomaly detection of difficult sample mining, characterized in that, The method comprises: acquiring power scene image data sets; the power scene image data sets comprise abnormal targets; the abnormal targets comprise equipment defects, personnel behavior abnormalities and equipment state abnormalities; performing feature extraction on the power scene image data sets by using a high-resolution feature extractor to obtain image feature maps; generating a plurality of candidate boxes based on the image feature maps by using an RPN module; determining the corresponding types of the candidate boxes by obtaining the RPN loss functions corresponding to the candidate boxes; the types of the candidate boxes comprise positive samples and negative samples; the positive samples are candidate boxes containing the abnormal targets, and the negative samples are candidate boxes not containing the abnormal targets; performing an iteration operation on the candidate boxes: performing forward propagation on the candidate boxes output by the last iteration by using an ROI pooling layer and a classification regression layer, and obtaining the target loss corresponding to the candidate boxes output by the last iteration; sorting the target loss in descending order; obtaining the confidence corresponding to the candidate boxes output by the last iteration by using a Soft NMS; selecting the candidate boxes output by this iteration from the candidate boxes output by the last iteration according to the confidence, the sorting result and the proportion of positive and negative samples; performing backward propagation on the candidate boxes output by this iteration by using the ROI pooling layer and the classification regression layer, updating the target parameters and sharing the target parameters to the forward propagation and backward propagation processes, and performing the next iteration operation based on the target parameters; the ROI pooling layer is used to divide and pool the feature maps corresponding to the candidate boxes subjected to forward propagation or backward propagation to obtain feature maps of a preset size; the classification regression layer is used to obtain class information and position information from the feature maps of the preset size, and perform class judgment according to the class information and adjust the positioning of the candidate boxes subjected to forward propagation or backward propagation according to the position information; when the iteration meets a preset condition, ending the iteration operation, taking the corresponding candidate boxes output by the last iteration as difficult samples to train a classification model, and training the classification model by using a GIOU loss, the classification model being used for power scene anomaly detection.
2. The method of claim 1, wherein: the target loss comprises a classification loss and a bounding box loss; the target parameters comprise corresponding weights of the classification loss and the bounding box loss, and the corresponding weight of the classification loss is not less than the corresponding weight of the bounding box loss.
3. The method of claim 2, wherein, the target loss is a weighted sum of the classification loss and the bounding box loss.
4. The method of claim 1, wherein, the power scene image data sets further comprise abnormal labels of the abnormal targets, the abnormal labels comprising the types and positions of the abnormal targets.
5. The method of claim 1, wherein, the forward propagation and the backward propagation both use the same network structure, which is an ROI pooling layer and a classification regression layer, and the classification regression layer comprises two fully connected layers.
6. The method of claim 1, wherein, the training of the classification model by using the GIOU loss comprises: Calculate the minimum closed area of the two candidate boxes, and calculate the proportion of the area in the closed area that does not belong to the two candidate boxes; the minimum closed area is the minimum rectangle that circumscribes the two candidate boxes; Calculate the intersection over union of the two candidate boxes, and determine the generalized intersection over union loss according to the difference between the intersection over union and the proportion.
7. A power scenario anomaly detection device based on difficult sample mining, characterized in that, The device comprises: The data acquisition module is configured to acquire a power scene image dataset; the power scene image dataset comprises an abnormal target; the abnormal target comprises a device defect, a personnel behavior anomaly, and a device state anomaly; The candidate box generation module is configured to extract features of the power scene image dataset by using a high-resolution feature extractor to obtain an image feature map; generate a plurality of candidate boxes based on the image feature map by using an RPN module; determine the corresponding type of each candidate box by obtaining an RPN loss function corresponding to each candidate box; the type of the candidate box comprises a positive sample and a negative sample; the positive sample is a candidate box containing the abnormal target, and the negative sample is a candidate box not containing the abnormal target; The iteration module is configured to perform an iteration operation on the candidate boxes: forward propagate the candidate boxes output by the last iteration by using an ROI pooling layer and a classification regression layer, and obtain a target loss corresponding to the candidate boxes output by the last iteration; sort the target losses in descending order; obtain a confidence corresponding to the candidate boxes output by the last iteration by using a Soft NMS; select candidate boxes output by this iteration from the candidate boxes output by the last iteration according to the confidence, the sorting result, and the proportion of positive and negative samples; backward propagate the candidate boxes output by this iteration by using the ROI pooling layer and the classification regression layer, update target parameters, and share the target parameters to the forward propagation and backward propagation processes, and perform a next iteration operation based on the target parameters; the ROI pooling layer is configured to divide and pool the feature map corresponding to the candidate boxes that are subjected to forward propagation or backward propagation to obtain a feature map of a preset size; the classification regression layer is configured to obtain class information and position information from the feature map of the preset size, perform class judgment according to the class information, and adjust the positioning of the candidate boxes that are subjected to forward propagation or backward propagation according to the position information; The training module is configured to end the iteration operation and train a classification model by using the candidate boxes output by the last iteration as difficult samples when the iteration meets a preset condition, and train the classification model by using a GIOU loss, wherein the classification model is used for power scene anomaly detection.
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 6.
9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.