Abnormal power equipment detection model training method, device, equipment and storage medium

Abstract

The present application relates to a method and apparatus for training an abnormal power equipment detection model. The method comprises: obtaining a sample image of abnormal power equipment and the actual abnormal power equipment detection result corresponding to the sample image, and inputting the sample image and the actual abnormal power equipment detection result into an abnormal power equipment detection model to be trained; obtaining image features corresponding to the sample image through the abnormal power equipment detection model to be trained, and obtaining connectivity information between any two image feature elements in the image features based on the image features; obtaining a predicted abnormal power equipment detection result corresponding to the sample image based on the connectivity information and the image features; and training the abnormal power equipment detection model based on the difference between the predicted abnormal power equipment detection result and the actual abnormal power equipment detection result to obtain a trained abnormal power equipment detection model. This method can accurately detect abnormal power equipment.

Description

Technical Field

[0001] The present application relates to the field of power detection technology, and in particular to a method, device, computer equipment, storage medium and computer program product for training an abnormal power equipment detection model. Background Art

[0002] With the development of the field of power detection technology, a method for detecting abnormal power equipment has emerged. This method detects abnormal power equipment through a trained abnormal power equipment detection model.

[0003] In the above technical solution, the abnormal power equipment detection model is trained by using abnormal power equipment sample images. However, in some scenarios, the abnormal power equipment sample images are very scarce, resulting in low abnormal power equipment detection accuracy of the trained abnormal power equipment detection model. Summary of the Invention

[0004] Based on this, it is necessary to provide an abnormal power equipment detection model training method, device, computer equipment, computer-readable storage medium and computer program product that can accurately detect abnormal power equipment to address the above technical problems.

[0005] In a first aspect, the present application provides a method for training an abnormal power equipment detection model. The method comprises:

[0006] Acquire a sample image of abnormal power equipment and an actual abnormal power equipment detection result corresponding to the sample image, and input the sample image and the actual abnormal power equipment detection result into an abnormal power equipment detection model to be trained;

[0007] Obtaining image features corresponding to the sample image through the abnormal power equipment detection model to be trained, and obtaining connectivity information between any two image feature elements in the image features based on the image features;

[0008] Based on the connectivity information and the image features, obtaining a predicted abnormal power equipment detection result corresponding to the sample image;

[0009] The abnormal power equipment detection model is trained according to the difference between the predicted abnormal power equipment detection result and the actual abnormal power equipment detection result to obtain a trained abnormal power equipment detection model.

[0010] In one embodiment, obtaining connectivity information between any two image feature elements in the image feature based on the image feature includes: obtaining the shortest path between any two image feature elements in the image feature based on the image feature; obtaining the connectivity probability between any two image feature elements in the image feature for the shortest path; and obtaining the connectivity information based on the shortest path and the connectivity probability.

[0011] In one embodiment, obtaining the connectivity probability between any two image feature elements in the image feature for the shortest path includes: obtaining an initial connectivity probability between any two adjacent image feature elements on the shortest path based on the image feature; and obtaining a connectivity probability between any two image feature elements in the image feature for the shortest path based on the initial connectivity probability.

[0012] In one embodiment, the any two adjacent image feature elements include: a first image feature element and a second image feature element; obtaining the initial connectivity probability between any two adjacent image feature elements on the shortest path based on the image features includes: obtaining a first initial connectivity probability of the first image feature element being unidirectionally connected to the second image feature element, and a second initial connectivity probability of the second image feature element being unidirectionally connected to the first image feature element, based on the image features; and obtaining the initial connectivity probability between any two adjacent image feature elements on the shortest path based on the first initial connectivity probability and the two initial connectivity probabilities.

[0013] In one embodiment, the predicted abnormal power equipment detection result corresponding to the sample image is obtained based on the connectivity information and the image features, including: obtaining the predicted position information and predicted category information corresponding to each area in the sample image based on the connectivity information and the image features; and obtaining the predicted abnormal power equipment detection result according to the predicted position information and the predicted category information.

[0014] In one embodiment, after obtaining the trained abnormal power equipment detection model, it also includes: obtaining an image to be detected for the power equipment to be detected, and inputting the image to be detected into the trained abnormal power equipment detection model; obtaining the position information and category information of each area of ​​the image to be detected through the abnormal power equipment detection model; based on the position information and category information, detecting whether there is an abnormal power equipment in the image to be detected.

[0015] In a second aspect, the present application also provides a device for training an abnormal power equipment detection model. The device comprises:

[0016] A sample image input module is used to obtain a sample image of abnormal power equipment and an actual abnormal power equipment detection result corresponding to the sample image, and input the sample image and the actual abnormal power equipment detection result into the abnormal power equipment detection model to be trained;

[0017] a connectivity information acquisition module, configured to obtain image features corresponding to the sample image using the abnormal power equipment detection model to be trained, and obtain connectivity information indicating the connectivity between any two image feature elements in the image features based on the image features;

[0018] A prediction result acquisition module, configured to obtain a predicted abnormal power equipment detection result corresponding to the sample image based on the connectivity information and the image features;

[0019] The model training module is used to train the abnormal power equipment detection model according to the difference between the predicted abnormal power equipment detection result and the actual abnormal power equipment detection result to obtain a trained abnormal power equipment detection model.

[0020] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0021] Acquire a sample image of abnormal power equipment and an actual abnormal power equipment detection result corresponding to the sample image, and input the sample image and the actual abnormal power equipment detection result into an abnormal power equipment detection model to be trained;

[0022] Obtaining image features corresponding to the sample image through the abnormal power equipment detection model to be trained, and obtaining connectivity information between any two image feature elements in the image features based on the image features;

[0023] Based on the connectivity information and the image features, obtaining a predicted abnormal power equipment detection result corresponding to the sample image;

[0024] The abnormal power equipment detection model is trained according to the difference between the predicted abnormal power equipment detection result and the actual abnormal power equipment detection result to obtain a trained abnormal power equipment detection model.

[0025] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0026] Acquire a sample image of abnormal power equipment and an actual abnormal power equipment detection result corresponding to the sample image, and input the sample image and the actual abnormal power equipment detection result into an abnormal power equipment detection model to be trained;

[0027] Obtaining image features corresponding to the sample image through the abnormal power equipment detection model to be trained, and obtaining connectivity information between any two image feature elements in the image features based on the image features;

[0028] Based on the connectivity information and the image features, obtaining a predicted abnormal power equipment detection result corresponding to the sample image;

[0029] The abnormal power equipment detection model is trained according to the difference between the predicted abnormal power equipment detection result and the actual abnormal power equipment detection result to obtain a trained abnormal power equipment detection model.

[0030] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0031] Acquire a sample image of abnormal power equipment and an actual abnormal power equipment detection result corresponding to the sample image, and input the sample image and the actual abnormal power equipment detection result into an abnormal power equipment detection model to be trained;

[0032] Obtaining image features corresponding to the sample image through the abnormal power equipment detection model to be trained, and obtaining connectivity information between any two image feature elements in the image features based on the image features;

[0033] Based on the connectivity information and the image features, obtaining a predicted abnormal power equipment detection result corresponding to the sample image;

[0034] The abnormal power equipment detection model is trained according to the difference between the predicted abnormal power equipment detection result and the actual abnormal power equipment detection result to obtain a trained abnormal power equipment detection model.

[0035] The above-mentioned abnormal power equipment detection model training method, device, computer equipment, storage medium and computer program product obtain sample images of abnormal power equipment and actual abnormal power equipment detection results corresponding to the sample images, and input the sample images and actual abnormal power equipment detection results into the abnormal power equipment detection model to be trained; obtain image features corresponding to the sample images through the abnormal power equipment detection model to be trained, and obtain connectivity information between any two image feature elements in the image features based on the image features; based on the connectivity information and the image features, obtain the predicted abnormal power equipment detection results corresponding to the sample images; and train the abnormal power equipment detection model based on the difference between the predicted abnormal power equipment detection results and the actual abnormal power equipment detection results to obtain a trained abnormal power equipment detection model. The present application obtains image features corresponding to the sample images by sending sample images and actual abnormal power equipment detection results to the abnormal power equipment detection model to be trained, and then obtains connectivity information between any two image feature elements in the image features based on the image features, obtains predicted abnormal power equipment detection results based on the connectivity information, and finally trains the abnormal power equipment detection model based on the difference between the predicted abnormal power equipment detection results and the actual abnormal power equipment detection results to obtain a trained abnormal power equipment detection model, so that the abnormal power equipment detection model can accurately detect abnormal power equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 1 is a flow chart of a method for training a model for detecting abnormal power equipment in one embodiment;

[0037] Figure 2 A schematic diagram of a process for obtaining connectivity information in one embodiment;

[0038] Figure 3 A schematic diagram of a process for obtaining connectivity probability in one embodiment;

[0039] Figure 4 A schematic diagram of a process for obtaining an initial connectivity probability in one embodiment;

[0040] Figure 5 This is a structural block diagram of an abnormal power equipment detection model training device in one embodiment;

[0041] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0043] It should be noted that the terms "first" and "second" as used in the embodiments of the present invention are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that the terms "first" and "second" may interchangeably represent a specific order or precedence, where permitted. It should be understood that the objects distinguished by "first" and "second" may be interchangeable, where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0044] In one embodiment, Figure 1 As shown, a method for training a model for detecting abnormal power equipment is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0045] Step S101 : obtaining a sample image of abnormal power equipment and an actual abnormal power equipment detection result corresponding to the sample image, and inputting the sample image and the actual abnormal power equipment detection result into an abnormal power equipment detection model to be trained.

[0046] Among them, abnormal power equipment refers to power equipment that operates abnormally. The abnormal power equipment can be a faulty relay, transformer, etc., and the sample image is an image taken of the abnormal power equipment. The sample image is used to train the abnormal power equipment detection model. As for the actual abnormal power equipment detection result, it refers to the label pre-labeled with the sample image. The actual abnormal power equipment detection result is used to characterize the abnormal operating status of the abnormal power equipment. Then, the abnormal power equipment detection model to be trained is a model to be trained for detecting abnormal power equipment. The abnormal power equipment detection model is a target detection neural network (DETR), which includes a convolutional neural network module (CNN) and a Transformer neural network module.

[0047] Specifically, sample images of abnormal power equipment are obtained from the power database, and category labels and position labels corresponding to each area of ​​the sample image are obtained. The category labels and position labels are used to characterize the actual values ​​of the abnormal power equipment detection results. Then, the sample images and the actual abnormal power equipment detection results are input into the abnormal power equipment detection model to be trained.

[0048] Step S102 : obtaining image features corresponding to the sample image through the abnormal power equipment detection model to be trained, and obtaining connectivity information between any two image feature elements in the image features based on the image features.

[0049] Among them, the image feature is the image feature map corresponding to the sample image, and the image feature contains multiple image feature elements. In this embodiment, the image feature is a sequence composed of multiple image feature elements, and the image feature elements are the image features of each area in the sample image. As for the connectivity information, it refers to the connectivity attention matrix in the multi-head attention mechanism (MHA) in the Transformer neural network module of the abnormal power equipment detection model. The connectivity information is used to characterize the connectivity between any pair of image feature elements and emphasize the dependency relationship between nearby image feature elements.

[0050] Specifically, the image features corresponding to the sample image are obtained through the convolutional neural network module (CNN) of the abnormal power equipment detection model to be trained. The image features are a sequence. For example, the image features can be expressed as where x n is a d-dimensional embedding vector, that is, an image feature element, and according to the image features, the connectivity information between any two image feature elements in the image features is obtained.

[0051] Step S103: obtaining a predicted abnormal power equipment detection result corresponding to the sample image based on the connectivity information and the image features.

[0052] The predicted abnormal power equipment detection result is a predicted result of abnormal power equipment detection.

[0053] Specifically, the image features are passed through an encoder-decoder composed of multiple connected attention layers containing the above-mentioned connectivity information, and finally the inference layer (FFN) of the abnormal power equipment detection model to obtain the predicted abnormal power equipment detection results corresponding to the sample image.

[0054] Step S104 : training an abnormal power equipment detection model based on the difference between the predicted abnormal power equipment detection result and the actual abnormal power equipment detection result to obtain a trained abnormal power equipment detection model.

[0055] Here, the difference refers to the loss function value corresponding to the abnormal power equipment detection model.

[0056] Specifically, based on the predicted abnormal power equipment detection results and the actual abnormal power equipment detection results, the loss function value corresponding to the abnormal power equipment detection model is obtained, and the abnormal power equipment detection model is trained using the loss function value to obtain a trained abnormal power equipment detection model. To predict the abnormal power equipment detection results, y is the label used to characterize the actual abnormal power equipment detection results, y i =(c i , b i ), c i Where b is the actual category label, i ∈[0, 1] 4 is the actual position label, specifically the center coordinates of the borders of each region of the sample image, and the vectors of the height and width of the image size of each region of the sample image. For the abnormal power equipment detection model with index σ(i), the predicted category information is The predicted location information is Then the final prediction loss function is:

[0057]

[0058] Where N represents the number of elements. Represents the matching loss of bounding box prediction. The above formula is used to train the abnormal power equipment detection model to obtain the trained abnormal power equipment detection model.

[0059] In the above-mentioned abnormal power equipment detection model training method, a sample image of abnormal power equipment and the actual abnormal power equipment detection result corresponding to the sample image are obtained, and the sample image and the actual abnormal power equipment detection result are input into the abnormal power equipment detection model to be trained; the image features corresponding to the sample image are obtained through the abnormal power equipment detection model to be trained, and based on the image features, the connectivity information between any two image feature elements in the image features is obtained; based on the connectivity information and the image features, the predicted abnormal power equipment detection result corresponding to the sample image is obtained; based on the difference between the predicted abnormal power equipment detection result and the actual abnormal power equipment detection result, the abnormal power equipment detection model is trained to obtain a trained abnormal power equipment detection model. The present application obtains image features corresponding to the sample images by sending sample images and actual abnormal power equipment detection results to the abnormal power equipment detection model to be trained, and then obtains connectivity information between any two image feature elements in the image features based on the image features, obtains predicted abnormal power equipment detection results based on the connectivity information, and finally trains the abnormal power equipment detection model based on the difference between the predicted abnormal power equipment detection results and the actual abnormal power equipment detection results to obtain a trained abnormal power equipment detection model, so that the abnormal power equipment detection model can accurately detect abnormal power equipment.

[0060] In one embodiment, Figure 2 As shown, according to the image features, obtaining the connectivity information between any two image feature elements in the image features includes the following steps:

[0061] Step S201: Obtain the shortest path between any two image feature elements in the image feature according to the image feature.

[0062] The shortest path is the minimum distance between any two image feature elements.

[0063] Specifically, based on the image features, the shortest path between any two image feature elements in the image features can be calculated.

[0064] Step S202 : Obtain the connectivity probability between any two image feature elements in the image feature for the shortest path.

[0065] The connectivity probability is the feasibility of connectivity between any two image feature elements in the image feature under the shortest path condition.

[0066] Specifically, based on the image features, the connectivity probability between any two image feature elements in the image features for the shortest path is calculated.

[0067] Step S203: Obtain connectivity information based on the shortest path and connectivity probability.

[0068] Specifically, as shown in the following formula:

[0069]

[0070] Among them, C i,j is the connectivity information between any two image feature elements, p i，j Represents the shortest path set of image feature elements i and j on the sample image. Each path p has the same number of connectivity probabilities w∈W all , Represents the set p i，j The number of paths contained in , that is, the number of shortest paths of the same length, is used to obtain connectivity information based on the shortest path and connectivity probability.

[0071] In this embodiment, the connectivity information between any two image feature elements can be accurately calculated through the shortest path and connectivity probability between any two image feature elements.

[0072] In one embodiment, Figure 3 As shown, obtaining the connectivity probability between any two image feature elements in the image feature for the shortest path includes the following steps:

[0073] Step S301 : obtaining the initial connectivity probability between any two adjacent image feature elements on the shortest path according to the image features.

[0074] The initial connectivity probability is the probability of connectivity between any two adjacent image feature elements on the shortest path.

[0075] Specifically, based on the image features, the initial connectivity probability between any two adjacent image feature elements on the shortest path is calculated.

[0076] Step S302 : Based on the initial connectivity probability, the connectivity probability between any two image feature elements in the image feature for the shortest path is obtained.

[0077] Specifically, the average value of multiple initial connectivity probabilities is calculated to obtain the connectivity probability between any two image feature elements in the image feature for the shortest path.

[0078] In this embodiment, by obtaining the initial connectivity probability between any two adjacent image feature elements on the shortest path, the connectivity probability between any two image feature elements in the image feature can be accurately obtained based on the initial connectivity probability.

[0079] In one embodiment, Figure 4 As shown, any two adjacent image feature elements include: a first image feature element and a second image feature element; obtaining the initial connectivity probability between any two adjacent image feature elements on the shortest path according to the image features includes the following steps:

[0080] Step S401 : obtaining a first initial connectivity probability of a first image feature element being unidirectionally connected to a second image feature element, and a second initial connectivity probability of a second image feature element being unidirectionally connected to the first image feature element, based on image features.

[0081] Among them, the first image feature element and the second image feature element are any two adjacent image feature elements, and the first initial connectivity probability is the probability that the first image feature element is unidirectionally connected to the second image feature element, and the second initial connectivity probability is the probability that the second image feature element is unidirectionally connected to the first image feature element.

[0082] Specifically, based on the image features and a preset formula, a first initial connectivity probability of the first image feature element being unidirectionally connected to the second image feature element and a second initial connectivity probability of the second image feature element being unidirectionally connected to the first image feature element are calculated.

[0083] Step S402 : Based on the first initial connectivity probability and the second initial connectivity probability, an initial connectivity probability between any two adjacent image feature elements on the shortest path is obtained.

[0084] Specifically, as shown in the following formula: in, is the initial connectivity probability, where i∈[1, n-1] represents the first initial connectivity probability of the first image feature element being unidirectionally connected to the second image feature element, Represents the probability that the second image feature element is unidirectionally connected to the first image feature element. and Perform an averaging operation to obtain the initial connectivity probability between any two adjacent image feature elements on the shortest path.

[0085] In this embodiment, by obtaining a first initial connectivity probability that the first image feature element is unidirectionally connected to the second image feature element, and a second initial connectivity probability that the second image feature element is unidirectionally connected to the first image feature element, and performing an averaging operation on the first initial connectivity probability and the second initial connectivity probability, the initial connectivity probability between any two adjacent image feature elements on the shortest path can be accurately obtained.

[0086] In one embodiment, obtaining a predicted abnormal power equipment detection result corresponding to a sample image based on connectivity information and image features includes the following steps:

[0087] Based on the connectivity information and image features, the predicted position information and predicted category information corresponding to each area in the sample image are obtained; based on the predicted position information and predicted category information, the predicted abnormal power equipment detection results are obtained.

[0088] The predicted position information is the predicted values ​​of the center coordinates, height, and width corresponding to each region in the sample image, and the predicted category information is the predicted value of the image content corresponding to each region in the sample image.

[0089] Specifically, the image features are passed through an encoder-decoder composed of multiple connected attention layers containing the above-mentioned connectivity information, and finally the inference layer (FFN) of the abnormal power equipment detection model to obtain the predicted position information and predicted category information corresponding to each area in the sample image. The predicted position information and predicted category information are fused to obtain the predicted abnormal power equipment detection results.

[0090] In this embodiment, by obtaining the predicted position information and predicted category information corresponding to each area in the sample image, and then according to the predicted position information and predicted category information, the predicted abnormal power equipment detection result can be accurately obtained.

[0091] In one embodiment, after obtaining the trained abnormal power equipment detection model, the following steps are further included:

[0092] An image of the power equipment to be detected is obtained and input into a trained abnormal power equipment detection model. The abnormal power equipment detection model is used to obtain the location information and category information of each area of ​​the image to be detected. Based on the location information and category information, it is detected whether there is abnormal power equipment in the image to be detected.

[0093] Among them, the image to be detected is a continuous image taken by the inspection equipment while inspecting the power equipment to be detected, and the position information is the position coordinates, height and width of each area of ​​the image to be detected. As for the category information, it is the image content type of each area of ​​the image to be detected.

[0094] Specifically, an image to be detected of the power equipment to be detected is obtained, and the image to be detected is input into the trained abnormal power equipment detection model; the image to be detected passes through an encoder-decoder composed of multiple connected attention layers containing the above-mentioned connectivity information, and finally the inference layer (FFN) of the abnormal power equipment detection model to obtain the position information and category information of each area of ​​the image to be detected. Based on the result obtained by fusion of the position information and the category information, it is detected whether there is abnormal power equipment in the image to be detected.

[0095] In this embodiment, by inputting the image to be detected into the trained abnormal power equipment detection model, the position information and category information of each area of ​​the image to be detected are obtained. Then, based on the position information and category information, it is possible to accurately detect whether there is abnormal power equipment in the image to be detected.

[0096] In one application embodiment, a method for detecting abnormal power targets based on connected attention is provided, which specifically includes the following steps:

[0097] 1. Training a connected attention-based power anomaly target detection model

[0098] The Transformer module of the DETR target detection model is improved, and the global self-attention mechanism of its attention layer is modified to a connected attention mechanism.

[0099] First, input the image and flatten the two-dimensional output feature map of the CNN backbone network into a sequence, which is expressed as Each token x1 is a d-dimensional embedding vector. The sequence is then fed into an encoder-decoder structure. Finally, a task head uses the decoded sequence to predict a set of m objects. Each object Consists of two parts: bounding box position and bounding box category distribution Where C is the set of all categories.

[0100] Each Transformer module mainly consists of two parts: the multi-head attention mechanism (MHA) and the multi-layer perceptron (MLP). MHA can be seen as the result of linear projection after the output of multiple dot-product attention modules are connected. Taking the above sequence X as input, its output is:

[0101]

[0102] in is the global attention matrix of the h-th head, H is the number of heads, is the projection matrix. Here, the attention matrix A is obtained by and The one-to-one inner product similarity between them is normalized and calculated as follows:

[0103]

[0104] Where Q = XW Q and K = XW k They are all obtained by linearly projecting the input sequence X. Under this MHA computational approach, inner product similarity is calculated between each element in the sequence X, and each element is weighted relative to all other elements. This inevitably extracts irrelevant dependencies, assigning smaller weights to truly relevant elements. This excessive flexibility of MHA encourages the generation of negative Hessian eigenvalues ​​and non-convex losses, hindering the optimization of the Transformer model. This problem is particularly severe in data-scarce environments; without sufficient annotations or a strong data augmentation strategy, it can lead to poor model performance.

[0105] We modify the way the attention matrix A is calculated. Instead of calculating it arbitrarily, we inject data-dependent sparsity patterns to suppress the dependencies between pairs of elements that are spatially unconnected. We propose to use the connected attention matrix To represent the connectivity between any pair of elements, and thus constrain the original attention matrix A:

[0106]

[0107] Where ⊙ represents an element-wise multiplication that injects the sparsity pattern into A. All attention heads in the same layer will be constrained by the same C. Unlike directly generated by self-attention, the connected attention matrix C is obtained based on the estimated breakpoints between elements. First, we add an additional "connected head" to the original attention head, which is also calculated by dot product attention:

[0108]

[0109] in i∈[1, n-1] represents the element x i One-way connection to x i+1 probability. Then we have and Perform an averaging operation:

[0110]

[0111] If the value is higher, then it means that the element x i and element x i+1 are interconnected, otherwise, if The value of is low, then it means that the element x i and element x i+1 There are breakpoints between them, which are not connected. At this point, we obtain the probability that each element is connected to its adjacent neighbors W = {w1, ...w i ,...w n}, which is the result of flattening the feature map horizontally into a sequence, so W represents the probability that each element in the image is connected to its horizontal neighbors. Similarly, we flatten the feature map vertically into a sequence, and we can calculate the probability that each element in the image feature is connected to its vertical neighbors W′={w′1,...w′ i ,...w′ n}, add up the two sets of connectivity probabilities to get W all ={w1, w′1, ... w i , w′ i ,...w n , w′ n Finally, our connected attention matrix C is defined as follows:

[0112]

[0113] where p i，j Represents the shortest path set between image feature graph element i and element j on the image, and each path p i，j There are several connected probabilities w∈W with the same number all , Represents the set p i，j The number of paths contained in , that is, the number of shortest paths of equal length. C imposes explicit spatial constraints to emphasize the dependencies between nearby elements.

[0114] After the image sequence passes through the encoder-decoder composed of multiple connected attention layers, the final reasoning layer (FFN) predicts the center coordinates, height and width of each bounding box, as well as its corresponding category, defined as These predictions are then compared with the labels and the model is trained using the loss. Specifically, each element in the label y can be viewed as y i =(c i , b i ), where c i is the target category label, b i ∈[0, 1] 4 is a vector defining the center coordinates of the target box and its height and width relative to the image size. For the model prediction with index σ(i), we will class c i The probability of And define the bounding box prediction as Then the final prediction loss is:

[0115]

[0116] Where N represents the number of elements, Represents the matching loss for bounding box prediction, defined as:

[0117]

[0118] in represents the generalized IOU loss, λ iou and λ L1 is a hyperparameter, and ||·||1 represents the norm. Based on the above model and loss function, a power anomaly target detection dataset is collected for training to obtain a trained power anomaly target detection model.

[0119] 2. Use the model to perform preliminary reasoning on test data

[0120] Use the power anomaly target detection model trained in the previous step to perform inference on the power data to be tested to obtain the image bounding box and its corresponding category.

[0121] 3. Use SIFT feature matching to correct inference results

[0122] For test images that have already been identified as certain classes and bounding boxes in the second step, SIFT features are used to correct incorrect inference results. Each class in the test image results is matched with images of the same category in the annotated image. If the match fails, these classes and bounding boxes are corrected to the background. For the test image and the annotated image, the SIFT feature extraction algorithm is used to obtain the key points and feature descriptions of the images respectively, and then the K-nearest neighbor algorithm is used to calculate the matching key points between the two images. If the number of matching key point pairs is 0, it is determined that the test image does not contain this class, and these classes and their bounding boxes are corrected to the background; if the number of matching key point pairs is greater than 0, the minimum distance is calculated. If the minimum distance is less than a certain threshold, it is determined that the test image contains this class and no correction is made. Through the above corrections, the power anomaly target detection results are obtained.

[0123] In the above method, by replacing the global self-attention mechanism in the Transformer module of DETR with a connected attention mechanism, each feature of the image embedding is encouraged to pay attention to the connected features around it, reducing redundant attention and alleviating the difficulty of network optimization, thereby improving the data validity of the power anomaly target detection model and improving the performance of the power anomaly target detection model, so that abnormal power targets can be accurately detected.

[0124] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0125] Based on the same inventive concept, embodiments of the present application also provide an abnormal power equipment detection model training device for implementing the aforementioned abnormal power equipment detection model training method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more embodiments of the abnormal power equipment detection model training device provided below can be found in the limitations of the abnormal power equipment detection model training method described above and will not be repeated here.

[0126] In one embodiment, Figure 5As shown, a device for training a model for detecting abnormal power equipment is provided, comprising: a sample image input module 501, a connectivity information acquisition module 502, a prediction result acquisition module 503, and a model training module 504, wherein:

[0127] The sample image input module 501 is used to obtain sample images of abnormal power equipment and actual abnormal power equipment detection results corresponding to the sample images, and input the sample images and actual abnormal power equipment detection results into the abnormal power equipment detection model to be trained;

[0128] The connectivity information acquisition module 502 is used to obtain image features corresponding to the sample image through the abnormal power equipment detection model to be trained, and obtain connectivity information between any two image feature elements in the image features based on the image features;

[0129] The prediction result acquisition module 503 is used to obtain the predicted abnormal power equipment detection result corresponding to the sample image based on the connectivity information and image features;

[0130] The model training module 504 is used to train the abnormal power equipment detection model according to the difference between the predicted abnormal power equipment detection result and the actual abnormal power equipment detection result to obtain a trained abnormal power equipment detection model.

[0131] In one embodiment, the connectivity information acquisition module 502 is further used to obtain the shortest path between any two image feature elements in the image feature based on the image feature; obtain the connectivity probability between any two image feature elements in the image feature for the shortest path; and obtain connectivity information based on the shortest path and the connectivity probability.

[0132] In one embodiment, the connectivity information acquisition module 502 is further used to obtain an initial connectivity probability between any two adjacent image feature elements on the shortest path based on the image features; based on the initial connectivity probability, obtain a connectivity probability between any two image feature elements in the image features for the shortest path.

[0133] In one embodiment, the prediction result acquisition module 502 is further used to obtain, based on the image features, a first initial connectivity probability that the first image feature element is unidirectionally connected to the second image feature element, and a second initial connectivity probability that the second image feature element is unidirectionally connected to the first image feature element; based on the first initial connectivity probability and the second initial connectivity probability, obtain the initial connectivity probability between any two adjacent image feature elements on the shortest path.

[0134] In one embodiment, the model training module 503 is further used to obtain predicted position information and predicted category information corresponding to each area in the sample image based on connectivity information and image features; and obtain predicted abnormal power equipment detection results based on the predicted position information and predicted category information.

[0135] In one embodiment, the connectivity information acquisition module 504 is further used to obtain an image to be detected for the power equipment to be detected, and input the image to be detected into a trained abnormal power equipment detection model; through the abnormal power equipment detection model, the position information and category information of each area of ​​the image to be detected are obtained; based on the position information and category information, it is detected whether there is an abnormal power equipment in the image to be detected.

[0136] Each module in the abnormal power equipment detection model training device described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0137] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for training an abnormal power equipment detection model is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0138] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0139] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0140] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0141] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0142] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0143] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0144] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for training an abnormal power equipment detection model, characterized in that: The method comprises: Acquire a sample image of abnormal power equipment and an actual abnormal power equipment detection result corresponding to the sample image, and input the sample image and the actual abnormal power equipment detection result into an abnormal power equipment detection model to be trained; Obtaining image features corresponding to the sample image through the abnormal power equipment detection model to be trained, and obtaining connectivity information between any two image feature elements in the image features based on the image features; Based on the connectivity information and the image features, obtaining a predicted abnormal power equipment detection result corresponding to the sample image; training the abnormal power equipment detection model according to the difference between the predicted abnormal power equipment detection result and the actual abnormal power equipment detection result to obtain a trained abnormal power equipment detection model; The obtaining, based on the image features, connectivity information between any two image feature elements in the image features, includes: Obtaining the shortest path between any two image feature elements in the image feature according to the image feature; Obtaining, for the shortest path, a connectivity probability between any two image feature elements in the image feature; Obtaining the connectivity information based on the shortest path and the connectivity probability; The obtaining of the connectivity probability between any two image feature elements in the image feature for the shortest path includes: Obtaining, based on the image features, an initial connectivity probability between any two adjacent image feature elements on the shortest path; Based on the initial connectivity probability, a connectivity probability between any two image feature elements in the image feature for the shortest path is obtained.

2. The method according to claim 1, characterized in that The arbitrary two adjacent image feature elements include: a first image feature element and a second image feature element; and obtaining, based on the image features, an initial connectivity probability between any two adjacent image feature elements on the shortest path, includes: According to the image features, obtaining a first initial connectivity probability that the first image feature element is unidirectionally connected to the second image feature element, and a second initial connectivity probability that the second image feature element is unidirectionally connected to the first image feature element; Based on the first initial connectivity probability and the second initial connectivity probability, an initial connectivity probability between any two adjacent image feature elements on the shortest path is obtained.

3. The method according to claim 1, characterized in that The obtaining, based on the connectivity information and the image features, a predicted abnormal power equipment detection result corresponding to the sample image, includes: Based on the connectivity information and the image features, obtaining predicted position information and predicted category information corresponding to each region in the sample image; The predicted abnormal power equipment detection result is obtained based on the predicted location information and the predicted category information.

4. The method according to claim 1, wherein After obtaining the trained abnormal power equipment detection model, the method further includes: Acquire an image to be detected for the power equipment to be detected, and input the image to be detected into a trained abnormal power equipment detection model; Obtaining location information and category information of each area of ​​the image to be detected through an abnormal power equipment detection model; Based on the position information and the category information, it is detected whether there is any abnormal power equipment in the image to be detected.

5. A device for training a model for detecting abnormal power equipment, characterized in that: The device comprises: A sample image input module is used to obtain a sample image of abnormal power equipment and an actual abnormal power equipment detection result corresponding to the sample image, and input the sample image and the actual abnormal power equipment detection result into the abnormal power equipment detection model to be trained; a connectivity information acquisition module, configured to obtain image features corresponding to the sample image using the abnormal power equipment detection model to be trained, and obtain connectivity information indicating the connectivity between any two image feature elements in the image features based on the image features; A prediction result acquisition module, configured to obtain a predicted abnormal power equipment detection result corresponding to the sample image based on the connectivity information and the image features; a model training module, configured to train the abnormal power equipment detection model according to the difference between the predicted abnormal power equipment detection result and the actual abnormal power equipment detection result, to obtain a trained abnormal power equipment detection model; The connectivity information acquisition module is further configured to obtain, based on the image feature, a shortest path between any two image feature elements in the image feature; obtain a connectivity probability between any two image feature elements in the image feature for the shortest path; and obtain the connectivity information based on the shortest path and the connectivity probability. The connectivity information acquisition module is also used to obtain an initial connectivity probability between any two adjacent image feature elements on the shortest path based on the image features; and based on the initial connectivity probability, obtain a connectivity probability between any two image feature elements in the image features for the shortest path.

6. The device according to claim 5, characterized in that The any two adjacent image feature elements include: a first image feature element and a second image feature element; the prediction result acquisition module is further used to obtain, based on the image features, a first initial connectivity probability of the first image feature element being unidirectionally connected to the second image feature element, and a second initial connectivity probability of the second image feature element being unidirectionally connected to the first image feature element; based on the first initial connectivity probability and the two initial connectivity probabilities, obtain an initial connectivity probability between any two adjacent image feature elements on the shortest path.

7. The device according to claim 5, characterized in that The prediction result acquisition module is also used to obtain the predicted position information and predicted category information corresponding to each area in the sample image based on the connectivity information and the image features; and obtain the predicted abnormal power equipment detection result according to the predicted position information and the predicted category information.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images