Equipment exception identification method and device, computer equipment and storage medium

Abstract

The invention relates to an equipment exception identification method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a feature map of a transformer substation equipment image, creating a plurality of prior frames for the feature map according to a set prior frame size, avoiding subsequent repeated detection and saving repeated detection time, classifying image information contained in the prior frames in the feature map for each prior frame, and obtaining the image information of the transformer substation equipment image according to the classified image information. And determining a priori category corresponding to the priori frame, by adopting the mode, the priori frame is taken as a kind of priori information, so that the search space is reduced, the calculation efficiency is improved, and then the priori frame is mapped into the substation equipment image according to the mapping relationship between the feature map and the substation equipment image under the condition that the priori category is an abnormal category. The abnormal area of the substation equipment is rapidly determined, and the abnormal identification efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of equipment detection, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for identifying equipment anomalies. Background Art

[0002] As the core facility of the power system, the normal operation of substation equipment is crucial to ensure the safe, reliable and efficient operation of the entire power grid. Through regular inspection and maintenance, potential fault hazards can be discovered and resolved in a timely manner, thereby avoiding power outages or safety accidents caused by equipment failure.

[0003] In the traditional method, infrared imaging technology is used to image the power supply equipment. By analyzing the temperature distribution in the infrared image, the location of the equipment can be detected. Since temperature change is a slow process, this method has the problem of slow abnormality recognition speed. Summary of the invention

[0004] Based on this, it is necessary to provide a device abnormality identification method, device, computer equipment, computer-readable storage medium and computer program product that can improve the speed of device abnormality identification in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for identifying device anomalies. The method comprises:

[0006] A feature map of the substation equipment image is obtained; multiple prior frames are created for the feature map according to the set prior frame size; for each prior frame, the image information contained in the feature map by the prior frame is classified to determine the prior category corresponding to the prior frame; when the prior category is an abnormal category, the prior frame is mapped to the substation equipment image according to the mapping relationship between the feature map and the substation equipment image to determine the abnormal area of ​​the substation equipment.

[0007] In one of the embodiments, a feature map of a substation equipment image is obtained, including: obtaining a substation equipment image; performing multi-size feature extraction on the substation equipment image to obtain feature maps of the substation equipment image corresponding to multiple different sizes; creating multiple prior frames for the feature map according to a set prior frame size, including: for each feature map, creating multiple prior frames for the feature map according to a set prior frame size.

[0008] In one of the embodiments, multiple prior frames are created for the feature map according to the set prior frame size, including: obtaining the set multiple prior frame ratios and the size of the feature map; determining the prior frame sizes corresponding to the prior frame ratios based on the size of the feature map and the prior frame ratios; and creating multiple prior frames that match the prior frame sizes for the feature map.

[0009] In one of the embodiments, the image information contained in the prior frame in the feature map is classified to determine the prior category corresponding to the prior frame, including: obtaining standard image information corresponding to different candidate categories; performing similarity matching on the image information contained in the prior frame in the feature map with each standard image information to obtain image similarities corresponding to each standard image information; and determining the candidate category corresponding to the greatest image similarity as the prior category corresponding to the prior frame.

[0010] In one of the embodiments, according to the mapping relationship between the feature map and the substation equipment image, the prior box is mapped to the substation equipment image to determine the abnormal area of ​​the substation equipment, including: determining the prior box whose confidence of the prior category is greater than the confidence threshold as the candidate box; according to the mapping relationship between the feature map to which the candidate box belongs and the substation equipment image, the candidate box is mapped to the substation equipment image to obtain the recognition box of the substation equipment image; based on the recognition image contained in the recognition box in the substation equipment image, the abnormal area of ​​the substation equipment is determined.

[0011] In one of the embodiments, there are multiple identification frames; based on the identification images contained in the identification frames in the substation equipment image, determining the abnormal area of ​​the substation equipment, including: determining the identification images contained in each identification frame in the substation equipment image; for each prior category, when there is image overlap among the multiple identification images corresponding to the prior category, determining at least two target images in an image overlapping state and candidate images not in an image overlapping state among the multiple identification images corresponding to the prior category; in each target image, determining a selected image whose confidence of the prior category meets the confidence condition; based on the selected images and candidate images corresponding to each prior category, determining the abnormal area of ​​the substation equipment.

[0012] In a second aspect, the present application also provides a device for identifying equipment abnormalities. The device comprises:

[0013] An acquisition module is used to acquire a feature map of a substation equipment image; a creation module is used to create multiple prior frames for the feature map according to a set prior frame size; a classification module is used to classify the image information contained in the prior frame in the feature map for each prior frame, and determine the prior category corresponding to the prior frame; an identification module is used to map the prior frame to the substation equipment image according to the mapping relationship between the feature map and the substation equipment image when the prior category is an abnormal category, and determine the abnormal area of ​​the substation equipment.

[0014] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the above method steps when executing the computer program.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the above method steps when executed by a processor.

[0016] In a fifth aspect, the present application also provides a computer program product. The computer program implements the above method steps.

[0017] The above-mentioned equipment abnormality identification method, device, computer equipment, storage medium and computer program product obtain the feature map of the substation equipment image, create multiple prior frames for the feature map according to the set prior frame size, create multiple prior frames to avoid subsequent repeated detection, and save the time of repeated detection. For each prior frame, the image information contained in the prior frame in the feature map is classified to determine the prior category corresponding to the prior frame. In the above-mentioned manner, the prior frame is used as a prior information to help reduce the search space and improve the calculation efficiency. Then, when the prior category is an abnormal category, the prior frame is mapped to the substation equipment image according to the mapping relationship between the feature map and the substation equipment image, so as to quickly determine the abnormal area of ​​the substation equipment and improve the efficiency of abnormal identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A diagram of an application environment of a method for identifying device anomalies in an embodiment;

[0019] Figure 2 A schematic diagram of a flow chart of a method for identifying device abnormality in one embodiment;

[0020] Figure 3 A schematic diagram of a flow chart of a method for monitoring oil leakage in substation equipment based on an SSD model in one embodiment;

[0021] Figure 4 A flowchart of system training in one embodiment;

[0022] Figure 5 The structure diagram of an oil leakage identification algorithm for substation equipment based on SSD in one embodiment;

[0023] Figure 6 A system prediction flow chart in one embodiment;

[0024] Figure 7 is a structural block diagram of a device abnormality identification device in one embodiment;

[0025] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] The device abnormality identification method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers.

[0028] The terminal 102 may obtain a feature map of the substation equipment image from the server 104. The terminal 102 may create multiple prior frames for the feature map according to the set prior frame size. The terminal 102 may classify the image information contained in the prior frame in the feature map for each prior frame, and determine the prior category corresponding to the prior frame. When the prior category is an abnormal category, the terminal 102 may map the prior frame to the substation equipment image according to the mapping relationship between the feature map and the substation equipment image, and determine the abnormal area of ​​the substation equipment.

[0029] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0030] In one embodiment, Figure 2 As shown, a device abnormality identification method is provided, and the method is applied to Figure 1 The following is an example of a terminal in the example, including:

[0031] S202, obtaining a feature map of the substation equipment image.

[0032] Among them, substation equipment can be a key equipment in the power system, responsible for converting voltage, receiving and distributing electric energy, controlling the flow of power and adjusting voltage. Substations connect power grids of different voltage levels through transformers and are an indispensable part of power engineering.

[0033] Substation equipment can be transformers, switchgear, lines, etc. Transformers are the core equipment of substations, used to convert high voltage electricity to low voltage electricity, or convert low voltage electricity to high voltage electricity. Transformers can be divided into step-up transformers and step-down transformers, which are used to increase or decrease voltage to meet different grid requirements. Transformers can also be oil-immersed transformers. Switchgear can be short-circuiters, contactors, and isolating switch lights. It is used to cut off or connect circuits to achieve circuit conversion, control and protection. Lines are used to collect current and connect various electrical equipment in the substation.

[0034] Substation equipment usually requires the use of special oils such as insulating oil or lubricating oil to ensure the normal operation and maintenance of the equipment. In the event of oil leakage or seepage, it may cause serious harm to human health, the environment and the equipment itself. It is necessary to detect oil or water leakage in substation equipment.

[0035] The feature map may be a map obtained by extracting features from an image of the substation equipment.

[0036] Specifically, the terminal may obtain the substation equipment image. The terminal may extract features of the substation equipment image to obtain a feature map of the substation equipment image.

[0037] The terminal can also obtain the substation equipment image. The terminal can perform multi-size feature extraction on the substation equipment image to obtain feature maps corresponding to multiple different sizes of the substation equipment image.

[0038] In one embodiment, the terminal can use oil leakage or water seepage as a detection target, perform target recognition on the substation equipment image, and obtain a feature map of the substation equipment image, which includes relevant image features reflecting oil leakage or water seepage.

[0039] S204, creating multiple prior boxes for the feature map according to the set prior box size.

[0040] The a priori frame size may indicate the size information of the a priori frame. For example, if the size of a certain a priori frame is 6*6, it may indicate that the a priori frame can accommodate 36 feature points.

[0041] Prior Box can be a mechanism in target detection. It is used to help the model predict the location and size of the target object. The setting and generation methods of the prior box vary in different target detection algorithms, but the core idea is to cover possible target shapes by pre-defining a set of rectangular boxes of different sizes and aspect ratios.

[0042] Specifically, there may be multiple prior frame sizes, and the terminal may create respective prior frame sizes for the feature map according to the prior frame sizes.

[0043] In one embodiment, the terminal may create a priori boxes centered at each creation point at multiple creation points of the feature map according to the size of the priori box. The creation point in the feature map may represent each feature point in the feature map. For example, for a feature map with a size of 19*19, the feature map includes 361 feature points.

[0044] In one of the embodiments, the terminal may create a plurality of prior boxes of different sizes centered at each creation point of the feature map.

[0045] Specifically, the terminal may also obtain a plurality of set a priori frame ratios and the size of the feature map. The terminal may determine the a priori frame size corresponding to each a priori frame ratio based on the size of the feature map and each a priori frame ratio. For the feature map, the terminal may respectively create a plurality of a priori frames that match each a priori frame size.

[0046] Exemplarily, the terminal may first determine multiple size samplings and aspect ratios of the prior frame. The size sampling may indicate the ratio of the prior frame to the feature map, such as 0.5, 0.2, 0.1, etc. The aspect ratio may be a fixed value, such as 1:1 or 1:2, etc. The terminal may create multiple prior frames at each creation point (cell) in the feature map with aspect ratios and different size samplings.

[0047] S206, for each a priori frame, classify the image information contained in the a priori frame in the feature map to determine the a priori category corresponding to the a priori frame.

[0048] The image information may be the image features of the part contained in the priori box in the feature map or the recognition result of the image features.

[0049] The priori category may indicate the category to which the image feature of the portion contained in the priori frame belongs (water seepage, oil seepage, water seepage and oil seepage, background, etc.). Specifically, the category may also include water seepage category 1, water seepage category 2, oil seepage category 1, oil seepage category 2, oil seepage category 3, etc. The category may be set according to the classic categories of water seepage and oil seepage.

[0050] Specifically, the terminal may classify the image information contained in the priori frame in the feature map for each priori frame, and determine the priori category corresponding to the priori frame.

[0051] In one embodiment, the terminal may determine multiple candidate categories corresponding to the priori frame and scores of the candidate categories based on the image information. The terminal may determine the candidate category with the highest score as the priori category corresponding to the priori frame.

[0052] In one of the embodiments, when all scores are less than a preset value, the terminal may determine that the priori category corresponding to the priori frame is a background category.

[0053] Specifically, the terminal can also obtain the standard image information corresponding to different candidate categories. The terminal can match the image information contained in the priori box in the feature map with each standard image information respectively to obtain the image similarity of the image information corresponding to each standard image information. The terminal can determine the candidate category corresponding to the maximum image similarity as the priori category corresponding to the priori box. The terminal can obtain the confidence level of each of the multiple categories.

[0054] S208, when the priori category is an abnormal category, according to the mapping relationship between the feature map and the substation equipment image, the priori box is mapped to the substation equipment image to determine the abnormal area of ​​the substation equipment.

[0055] The abnormality category may represent a priori categories for the abnormality, for example, water seepage category, oil seepage category, water seepage and oil seepage category, etc.

[0056] The mapping relationship between the feature map and the substation equipment image can be characterized by the sampling parameters of upsampling and downsampling.

[0057] The substation equipment image may be an original image corresponding to a feature map of the substation equipment. It can be understood that the feature map can be obtained by extracting features from the original image, and the original image can be obtained by a series of up- and down-sampling and feature fusion of the feature map.

[0058] Specifically, the terminal may map the priori frame to the substation equipment image according to the sampling parameters, and determine the abnormal area of ​​the substation equipment.

[0059] In one of the embodiments, the terminal may determine a priori box whose confidence of the prior category is greater than a confidence threshold as a candidate box. The terminal may map the candidate box to the substation equipment image according to the mapping relationship between the feature map to which the candidate box belongs and the substation equipment image to obtain a recognition box of the substation equipment image. The terminal may determine the abnormal area of ​​the substation equipment based on the recognition image contained in the recognition box in the substation equipment image.

[0060] In one embodiment, the number of identification frames can be multiple, and the terminal can determine the identification images contained in each identification frame in the substation equipment image. The terminal can determine at least two target images in an image overlapping state and candidate images not in an image overlapping state among the multiple identification images corresponding to the priori category for each priori category when there is image overlap among the multiple identification images corresponding to the priori category. The terminal can determine, among each target image, a selected image whose confidence of the priori category meets the confidence condition. The terminal can determine the abnormal area of ​​the substation equipment based on the selected images and candidate images corresponding to each priori type.

[0061] In the above-mentioned device abnormality identification method, a feature map of the substation equipment image is obtained, and multiple prior frames are created for the feature map according to the set prior frame size. Creating multiple prior frames avoids subsequent repeated detection and saves the time of repeated detection. For each prior frame, the image information contained in the prior frame in the feature map is classified to determine the prior category corresponding to the prior frame. In the above-mentioned manner, the prior frame, as a kind of prior information, helps to reduce the search space and improve the calculation efficiency. Furthermore, when the prior category is an abnormal category, the prior frame is mapped to the substation equipment image according to the mapping relationship between the feature map and the substation equipment image, so as to quickly determine the abnormal area of ​​the substation equipment and improve the efficiency of abnormality identification.

[0062] In one embodiment, obtaining a feature map of a substation equipment image includes: obtaining the substation equipment image, performing multi-size feature extraction on the substation equipment image, and obtaining feature maps of the substation equipment image corresponding to a plurality of different sizes.

[0063] According to the set priori frame size, multiple prior frames are created for the feature map, including: for each feature map, according to the set priori frame size, multiple prior frames are created for the feature map.

[0064] The substation equipment image may represent an image of the substation equipment. Specifically, the image may be an image captured in real time by a camera.

[0065] The size can indicate the expected size of the feature map, for example, 19*19, 10*10, 5*5, etc.

[0066] Specifically, the terminal can perform multi-scale feature extraction on the substation equipment image based on the feature extraction network to obtain feature maps of the substation equipment image corresponding to multiple different sizes.

[0067] In one embodiment, the feature extraction network includes a preset number of feature extraction layers. The terminal extracts an input image input to each feature extraction layer at each feature extraction layer, and outputs a feature map (output image) of a corresponding size of the feature extraction layer.

[0068] Exemplarily, the preset number of feature extraction layers includes 38*38 feature layers, 19*19 feature layers, 10*10 feature layers, 5*5 feature layers, 3*3 feature layers and 1*1 feature layers connected in sequence. The terminal can input the substation equipment image into the 38*38 feature layer, and then obtain the feature maps output by the 38*38 feature layer, the 19*19 feature layer, the 10*10 feature layer, the 5*5 feature layer, the 3*3 feature layer and the 1*1 feature layer.

[0069] Among them, each feature map has a different corresponding size.

[0070] Specifically, the terminal may create multiple prior frames for each feature map according to a set prior frame size.

[0071] In one embodiment, the terminal may create a priori boxes corresponding to each creation point at different creation points (units) of the feature map according to the set a priori box size. The size of the a priori box is the a priori box size.

[0072] In this embodiment, a substation equipment image is acquired, and multi-size feature extraction is performed on the substation equipment image to obtain feature maps corresponding to a plurality of different sizes of the substation equipment image. For each feature map, multiple prior frames are created for the feature map according to a set prior frame size. Multi-size feature extraction is performed on the substation equipment image to obtain feature maps of different sizes, which are suitable for the detection of large and small targets and can improve the accuracy of subsequent abnormality recognition.

[0073] In one embodiment, multiple prior frames are created for a feature map according to a set prior frame size, including: obtaining multiple set prior frame ratios and the size of the feature map, determining the prior frame sizes corresponding to the prior frame ratios based on the size of the feature map and the prior frame ratios, and creating multiple prior frames that match the prior frame sizes for the feature map.

[0074] The a priori box ratio may be the ratio between the expected a priori box size and the feature map size. For example, the ratio may be 0.5, and when the feature map size is 10*10, the expected a priori box size is 5*5.

[0075] The size of the feature map can represent the length and width of the feature map.

[0076] Specifically, for the feature map, the terminal can create multiple prior boxes that match the sizes of each prior box.

[0077] In one embodiment, the terminal may determine a creation point in the feature map. The terminal may use the creation point as the center and the size of the priori box as the creation size to create a priori box that matches the size of the priori box. The center of the priori box may be the creation point.

[0078] For example, for a 19*19 feature map, 6 prior frame ratios can be set. The terminal can create 6 prior frames that match the 6 prior frame sizes corresponding to the 6 prior frame ratios for the feature map. For a feature map or a feature extraction layer, there is a one-to-one correspondence between the prior frame ratio, the prior frame size, and the prior frame.

[0079] In this embodiment, multiple set prior frame ratios and feature map sizes are obtained, and based on the feature map size and each prior frame ratio, the prior frame sizes corresponding to each prior frame ratio are determined. For the feature map, multiple prior frames that match each prior frame size are created respectively. Multiple prior frames of different sizes are created, which are suitable for the detection of large and small targets and can improve the accuracy of subsequent abnormality recognition.

[0080] In one embodiment, the image information contained in the prior frame in the feature map is classified to determine the prior category corresponding to the prior frame, including: obtaining standard image information corresponding to different candidate categories, performing similarity matching on the image information contained in the prior frame in the feature map with each standard image information, respectively, to obtain the image similarity of the image information corresponding to each standard image information, and determining the candidate category corresponding to the largest image similarity as the prior category corresponding to the prior frame.

[0081] The candidate category may be a category that is manually or computer-annotated in the image. Specifically, the candidate category may be a category to which an object belongs in the image that is manually or computer-annotated. For example, the candidate category may include a marked water seepage category, a marked oil seepage category, a marked water seepage and oil seepage category, etc.

[0082] The standard image information may be relevant information of the image corresponding to the candidate category. The relevant information may be texture features, position features, spectral information, etc. of the image.

[0083] Image similarity can indicate the degree of similarity between image information and standard image information.

[0084] If the information similarity between a certain image information and the standard image information is higher, it can represent that the category to which the object in the image corresponding to the image information belongs is more likely to be the candidate category corresponding to the standard image information.

[0085] There are multiple image similarities corresponding to one piece of image information. Specifically, the terminal may determine the candidate category corresponding to the greatest image similarity as the priori category corresponding to the priori box.

[0086] In this embodiment, the standard image information corresponding to each candidate category is obtained, and the image information contained in the prior frame in the feature map is matched with each standard image information respectively to obtain the image similarity corresponding to each standard image information. The candidate category corresponding to the largest image similarity is determined as the prior category corresponding to the prior frame. Accurately determining the prior category corresponding to the prior frame helps to improve the accuracy of subsequent determination of abnormal areas of substation equipment.

[0087] In one embodiment, according to the mapping relationship between the feature map and the substation equipment image, the prior box is mapped to the substation equipment image to determine the abnormal area of ​​the substation equipment, including: determining the prior box whose confidence of the prior category is greater than the confidence threshold as a candidate box, mapping the candidate box to the substation equipment image according to the mapping relationship between the feature map to which the candidate box belongs and the substation equipment image, obtaining the recognition box of the substation equipment image, and determining the abnormal area of ​​the substation equipment based on the recognition image contained in the recognition box in the substation equipment image.

[0088] The confidence threshold may represent a threshold configured for confidence.

[0089] The recognition box may represent a box in the substation equipment image that corresponds to the candidate box of the feature map.

[0090] The mapping relationship between the feature map and the substation equipment image can be characterized by the sampling parameters of upsampling and downsampling. Specifically, the mapping relationship can be the size change during the upsampling and downsampling process. For example, during the process of converting the feature map to the substation equipment image, the size change is 10 times the size.

[0091] Specifically, the terminal may map the candidate frame to the substation equipment image according to the size change between the feature map to which the candidate frame belongs and the substation equipment image, and obtain the recognition frame of the substation equipment image.

[0092] There may be multiple candidate boxes in one feature map. After the multiple candidate boxes in different feature maps are mapped to the substation equipment image, there are multiple identification boxes in the substation equipment image.

[0093] In order to simplify the representation of identification boxes in the substation equipment image, some identification boxes can be screened out from multiple identification boxes according to certain rules.

[0094] Specifically, the terminal may determine whether the identification frames overlap, and may retain one of the overlapping identification frames, thereby obtaining a substation equipment image that does not include the identification frames in the overlapping state.

[0095] The terminal may also determine the confidence level of each recognition frame, and may retain the recognition frame with the highest confidence level.

[0096] In this embodiment, a priori box whose confidence of the prior category is greater than the confidence threshold is determined as a candidate box, and the candidate box is mapped to the substation equipment image according to the mapping relationship between the feature map to which the candidate box belongs and the substation equipment image, so as to obtain the recognition box of the substation equipment image, and the abnormal area of ​​the substation equipment is determined based on the recognition image contained in the recognized box in the substation equipment image, so as to quickly determine the abnormal area of ​​the substation equipment, thereby improving the efficiency of abnormal recognition.

[0097] In one embodiment, there are multiple identification frames, and based on the identification images contained in the identification frames in the substation equipment image, determining the abnormal area of ​​the substation equipment includes: determining the identification images contained in each identification frame in the substation equipment image, and for each prior category, when there is image overlap among the multiple identification images corresponding to the prior category, determining at least two target images in an image overlapping state and candidate images not in an image overlapping state among the multiple identification images corresponding to the prior category, and in each target image, determining a selected image whose confidence of the prior category meets the confidence condition, and determining the abnormal area of ​​the substation equipment based on the selected images and candidate images corresponding to each prior type.

[0098] The situation that multiple recognition images overlap may indicate that at least two recognition images overlap.

[0099] In each priori category, if the recognition image and another recognition image have image overlap, the recognition image and the other recognition image can be considered as target images. If the recognition image and any recognition image do not have image overlap, the recognition image can be considered as a candidate image.

[0100] The confidence condition may be to select a target image corresponding to the maximum confidence, or the confidence condition may be to select target images corresponding to several maximum confidences. The selected image may represent a target image whose confidence satisfies the confidence condition.

[0101] Specifically, the terminal may determine the selected images and candidate images corresponding to each a priori type and the areas corresponding to each of the selected images and candidate images as abnormal areas of the substation equipment.

[0102] Specifically, for each prior category, when there is image overlap between multiple recognition images corresponding to the prior category, the terminal can also determine the image intersection and union ratio of at least two target images in an image overlap state. If the intersection and union ratio is greater than the intersection and union ratio threshold, the terminal can determine at least two target images in an image overlap state among the multiple recognition images corresponding to the prior category. If it is less than or equal to the intersection and union ratio threshold, the terminal can use the target image as a candidate image. The intersection and union ratio threshold can be set to 0.5.

[0103] In this embodiment, the identification images contained in each identification box in the substation equipment image are determined. For each prior category, when there is image overlap among multiple identification images corresponding to the prior category, at least two target images in an image overlapping state and candidate images not in an image overlapping state are determined among the multiple identification images corresponding to the prior category. In each target image, a selected image whose confidence of the prior category meets the confidence condition is determined. Based on the selected images and candidate images corresponding to each prior type, the abnormal area of ​​the substation equipment is determined, and the identification images in an overlapping state and not meeting the confidence condition are screened out, thereby reducing the difficulty of calculation, improving the calculation efficiency, and thus improving the efficiency of abnormal identification.

[0104] In one embodiment, Figure 3 As shown, a substation equipment oil leakage monitoring method based on the SSD model is provided, including:

[0105] S301, acquiring substation equipment images.

[0106] S302, performing multi-size feature extraction on the substation equipment image to obtain feature maps of the substation equipment image corresponding to a plurality of different sizes.

[0107] Exemplarily, multi-scale feature extraction may be performed on substation equipment images based on the SSD model.

[0108] Among them, the SSD model can be

[0109] The first part is to create a dataset, including:

[0110] The purpose is to train a deep learning detection network for substation equipment oil leakage that can be applied in various scenarios. Therefore, it is necessary to collect various pictures of substation equipment oil leakage. Relying on the open source data set of electrical engineering, we download the substation equipment picture data set without oil leakage, water stains, and shadows, the substation equipment picture data set with oil leakage but no water stains and shadows, the substation equipment picture data set with water stains and shadows but no oil leakage, and the substation equipment picture data set with oil leakage, water stains, and shadows.

[0111] To facilitate training and testing, 80% of the above data is made into a training set for training the neural network, and 20% of the data is made into a test set for testing the performance of the neural network. The four different data sets mentioned above need to be integrated into a data set for training the substation equipment oil leakage detection network. The target areas in the above data sets are marked, that is, the target areas of oil leakage that need to be detected (ground truth) are manually selected.

[0112] The second part, the model training phase, includes:

[0113] Reference Figure 4 The system training flowchart shown, and Figure 5 The structure diagram of the substation equipment oil leakage identification algorithm based on SSD is shown.

[0114] 1. During the data preprocessing stage, the images can be transformed such as random cropping, random flipping, photometric distortion, etc. to improve the diversity of the data set.

[0115] 2. The image enters the backbone network for feature extraction. The size of the sample image used in the figure is 300*300*3, and the image size can be adjusted according to the actual situation. The backbone network is improved based on the VGG16 model. The specific improvement is to convert the fully connected layers fc6 and fc7 of VGG16 into 3*3 convolutional layers conv6 and 1*1 convolutional layers conv7, and at the same time, the pooling layer pool5 is changed from the original 2*2 pooling layer with stride = 2 to a 3*3 pooling layer with stride = 1.

[0116] 3. Generate a priori boxes (default boxes) from feature maps. The concept of SSD is to set a priori boxes with different scales or aspect ratios for each unit. Here, each unit sets 6 priori boxes with different scales or aspect ratios. The center point of each priori box is at the center of the feature map unit, and the size is determined by the preset scaling ratio. For example, for a 19*19 feature layer, the length and width of the 6 priori boxes are obtained by the scaling ratio, and then multiplied by the image size of 300 to get the actual length and width. A 19*19 feature layer can generate 2166 priori boxes.

[0117] 4. At the end of the network, which is the detection part, 6 feature layers are used for prediction. Shallow feature layers are used to detect small targets, and deep feature layers are used to detect large targets. Specifically, 3*3 convolutions are performed on each of the 6 feature layers. The number of channels changes according to the order of the feature layers, the number of default boxes, and the number of categories. In the network where objects need to be detected, the output is directly obtained through a 3*3 convolution. The number of channels of the convolution is determined by the number of default boxes and the number of categories, specifically (number of default boxes * (number of categories + 4)).

[0118] 5. Match positive and negative samples for each feature layer, that is, match the generated default box with the pre-labeled data (ground truth). According to the matching strategy, match the default box with the ground truth box. For each ground truth, select the one with the largest IOU among all the default boxes as the match of the current ground truth box. For the remaining unmatched default boxes, if the IOU with a certain ground truth is greater than a certain threshold (generally 0.5), then the default box is also matched with this ground truth. If there are multiple default boxes greater than the threshold, the default box is only matched with the ground truth box with the largest IOU. If they match, they are positive samples (positive), otherwise they are negative samples (negative). In this process, the negative samples are far more than the positive samples, so hard negative mining is performed, and the negative samples with larger confidence loss are retained, so that the positive-negative sample ratio is 1:3.

[0119] 6. Finally, the positioning loss and classification loss are calculated for positive samples, and the classification loss is calculated for negative samples. The obtained losses are used to update the parameters. Finally, the coordinates of the positive samples are constantly close to the coordinates of the ground truth, and the classification confidence rate is constantly close to the real classification distribution.

[0120] The loss function for oil leakage detection in substation equipment includes two weighted parts: position loss function and confidence loss function.

[0121]

[0122] The first part of the loss function is the category loss, and the second part is the position loss. N is the number of matched prior boxes.

[0123] Confidence loss is the softmax loss on multi-class confidence (c), and the formula is as follows

[0124]

[0125]

[0126] in Refers to the predicted prior box number, Refers to the ground truth box number, Refers to the category number, Represents the background. Take 1 to represent the The prior box is matched to the A ground truth box, and the category of this ground truth box is . Indicates Prior boxes correspond to categories The first half of the formula is the positive sample ( ), that is, the loss of being classified as a certain category (excluding the background), and the latter half is the negative sample ( ), that is, the loss of the background category.

[0127] That is, the first Prior boxes correspond to categories The predicted probability of is taken as logarithm, and the first Take the logarithm of the predicted probability of the background corresponding to the prior box, sum it up and take the negative.

[0128] The category loss is a typical smooth L1 loss

[0129]

[0130]

[0131]

[0132]

[0133]

[0134] in is the prior frame, is the ground truth. is the center coordinate of the default box, The width and height of the default box. Refers to the predicted prior box number, Refers to the ground truth box number, Refers to the category number, Represents the background. Take 1 to represent the The prior box is matched to the A groundtruth box, and the category of this ground truth box is .

[0135] Represents the jth ground truth box Coordinates (center position x coordinate), Represents the i-th prior (default box) box Coordinates (center position x coordinate), Represents the i-th default box value(width,width).

[0136] That is, all positive samples ( The prior box matches the The prior box position parameter value in the ground truth box and the adjusted ground truth position parameter value of The sum of the losses.

[0137] The third part, the prediction phase, includes:

[0138] The prediction process is similar to the training process.

[0139] Reference Figure 6 The system prediction flow chart shown.

[0140] 1. Collect substation equipment pictures in real time, input the pictures to be tested into the trained model, and the pictures enter the prediction layer after feature extraction.

[0141] 2. For each prior frame, determine its category (the one with the largest confidence) and confidence value according to the category confidence, and filter out the prior frames belonging to the background.

[0142] 3. Filter out the priori boxes with a lower confidence threshold (such as 0.5). Decode the remaining priori boxes and get the true position parameters of the priori boxes based on the position and size of the ground truth box (cropping is required after decoding if necessary to prevent the priori box from exceeding the image). After decoding, sort in descending order according to confidence, and only keep the k priori boxes with the highest confidence.

[0143] 4. Perform the non-maximum suppression algorithm to filter out those prior frames with large overlap. The remaining prior frames are the detection results.

[0144] S303: For each feature map, obtain a plurality of set priori box ratios and a size of the feature map.

[0145] S304: Based on the size of the feature map and the ratios of the prior boxes, determine the sizes of the prior boxes corresponding to the ratios of the prior boxes.

[0146] S305 , for the feature map, create a plurality of priori boxes that match the sizes of the priori boxes respectively.

[0147] S306: For each priori frame, obtain standard image information corresponding to different candidate categories.

[0148] S307, performing similarity matching on the image information contained in the priori frame in the feature map and each standard image information, respectively, to obtain the image similarity of the image information corresponding to each standard image information.

[0149] S308: Determine the candidate category corresponding to the maximum image similarity as the priori category corresponding to the priori box.

[0150] S309: When the priori category is an abnormal category, a priori box whose confidence of the priori category is greater than a confidence threshold is determined as a candidate box.

[0151] S310, mapping the candidate box to the substation equipment image according to the mapping relationship between the feature map to which the candidate box belongs and the substation equipment image, to obtain a recognition box of the substation equipment image.

[0152] S312: Determine the identification images contained in the identification boxes in the substation equipment image.

[0153] S313, for each prior category, when there is image overlap among multiple recognition images corresponding to the prior category, determine at least two target images in an image overlap state and candidate images that are not in an image overlap state among the multiple recognition images corresponding to the prior category.

[0154] S314, among the target images, determine the selected images whose confidence of the prior category satisfies the confidence condition.

[0155] S315 , determining abnormal areas of the substation equipment based on the selected images and candidate images corresponding to each priori category.

[0156] In this embodiment, a feature map of the substation equipment image is obtained, and multiple prior frames are created for the feature map according to the set prior frame size. The creation of multiple prior frames avoids subsequent repeated detection and saves the time of repeated detection. For each prior frame, the image information contained in the prior frame in the feature map is classified, and the prior category corresponding to the prior frame is determined. In the above manner, the prior frame, as a kind of prior information, helps to reduce the search space and improve the calculation efficiency. Furthermore, when the prior category is an abnormal category, the prior frame is mapped to the substation equipment image according to the mapping relationship between the feature map and the substation equipment image, so as to quickly determine the abnormal area of ​​the substation equipment and improve the efficiency of abnormal identification.

[0157] The above method can also bring the following beneficial effects:

[0158] (1) Based on the SSD detection method, feature maps of different scales are extracted for detection. Large-scale feature maps (feature maps closer to the front) can be used to detect small objects, while small-scale feature maps (feature maps closer to the back) are used to detect large objects. Prior frames of different scales and aspect ratios are used, and convolution is used to directly output the detection results. This allows SSD to overcome the shortcoming of the Yolo algorithm that it is difficult to detect small targets to a certain extent, thereby improving the accuracy of oil leakage detection in substation equipment.

[0159] (2) It can automatically identify and determine oil leakage, effectively improving the accuracy and efficiency of detection, and has high application value and market prospects.

[0160] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0161] Based on the same inventive concept, the embodiment of the present application also provides a device abnormality identification device for implementing the device abnormality identification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more device abnormality identification device embodiments provided below can refer to the limitations of the device abnormality identification method above, and will not be repeated here.

[0162] In one embodiment, Figure 7 As shown, a device for identifying abnormalities of equipment is provided, including: an acquisition module 701, a creation module 702, a classification module 703 and an identification module 704, wherein:

[0163] An acquisition module 701 is used to acquire a feature map of a substation equipment image;

[0164] A creation module 702 is used to create multiple prior boxes for the feature map according to the set prior box size;

[0165] A classification module 703 is used to classify the image information contained in the priori frame in the feature map for each priori frame, and determine the priori category corresponding to the priori frame;

[0166] The identification module 704 is used to map the priori box to the substation equipment image according to the mapping relationship between the feature map and the substation equipment image when the priori category is an abnormal category, and determine the abnormal area of ​​the substation equipment.

[0167] In one of the embodiments, the acquisition module 701 is also used to acquire the substation equipment image; perform multi-size feature extraction on the substation equipment image to obtain feature maps of the substation equipment image corresponding to multiple different sizes; the creation module 702 is also used to create multiple prior frames for each feature map according to the set prior frame size.

[0168] In one of the embodiments, the creation module 702 is also used to obtain multiple set prior box ratios and the size of the feature map; based on the size of the feature map and the prior box ratios, determine the prior box size corresponding to each prior box ratio; for the feature map, create multiple prior boxes that match each prior box size.

[0169] In one embodiment, the classification module 703 is also used to obtain standard image information corresponding to different candidate categories; the image information contained in the prior frame in the feature map is matched with each standard image information to obtain the image similarity corresponding to each standard image information; the candidate category corresponding to the maximum image similarity is determined as the prior category corresponding to the prior frame.

[0170] In one of the embodiments, the recognition module 704 is further used to determine a priori boxes whose confidence of the prior category is greater than a confidence threshold as candidate boxes; map the candidate boxes to the substation equipment image according to the mapping relationship between the feature map to which the candidate boxes belong and the substation equipment image to obtain a recognition box of the substation equipment image; and determine the abnormal area of ​​the substation equipment based on the recognition image contained in the recognition box in the substation equipment image.

[0171] In one of the embodiments, there are multiple identification frames; the identification module 704 is further used to determine the identification images contained in each identification frame in the substation equipment image; for each prior category, when there is image overlap among the multiple identification images corresponding to the prior category, at least two target images in an image overlapping state and candidate images not in an image overlapping state are determined among the multiple identification images corresponding to the prior category; in each target image, a selected image whose confidence of the prior category meets the confidence condition is determined; based on the selected images and candidate images corresponding to each prior category, the abnormal area of ​​the substation equipment is determined.

[0172] Each module in the above-mentioned device abnormality identification device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0173] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store feature map data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a device abnormality identification method is implemented.

[0174] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0175] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method steps are implemented when the processor executes the computer program.

[0176] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method steps are implemented.

[0177] In one embodiment, a computer program product is provided, comprising a computer program, which implements the above method steps when executed by a processor.

[0178] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0179] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0180] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for identifying device abnormality, characterized in that: The method comprises: Obtain feature maps of substation equipment images; Creating a plurality of prior frames for the feature map according to the set prior frame size; For each of the a priori frames, classify the image information contained in the a priori frame in the feature map to determine a priori category corresponding to the a priori frame; In the case where the priori category is an abnormal category, the priori box is mapped to the substation equipment image according to the mapping relationship between the feature map and the substation equipment image to determine the abnormal area of ​​the substation equipment.

2. The method according to claim 1, characterized in that The step of obtaining a feature map of a substation equipment image includes: Obtain images of substation equipment; Performing multi-size feature extraction on the substation equipment image to obtain feature maps of the substation equipment image corresponding to a plurality of different sizes; The step of creating a plurality of priori boxes for the feature map according to the set priori box size includes: For each of the feature maps, multiple prior boxes are created for the feature map according to the set prior box size.

3. The method according to claim 1, characterized in that The step of creating a plurality of priori boxes for the feature map according to the set priori box size includes: Obtaining a plurality of set prior frame ratios and a size of the feature map; Based on the size of the feature map and the ratios of the prior frames, determining the sizes of the prior frames corresponding to the ratios of the prior frames; For the feature map, a plurality of prior boxes are created respectively, each matching the size of each prior box.

4. The method according to claim 1, characterized in that: The classifying the image information contained in the priori frame in the feature map to determine the priori category corresponding to the priori frame includes: Obtain standard image information corresponding to different candidate categories; The image information contained in the priori frame in the feature map is matched with each of the standard image information respectively to obtain the image similarity between the image information and each of the standard image information; The candidate category corresponding to the maximum image similarity is determined as the priori category corresponding to the priori box.

5. The method according to claim 1, characterized in that The mapping of the prior frame to the substation equipment image according to the mapping relationship between the feature map and the substation equipment image to determine the abnormal area of ​​the substation equipment includes: Determine the priori box whose confidence of the priori category is greater than the confidence threshold as a candidate box; According to the mapping relationship between the feature map to which the candidate frame belongs and the substation equipment image, the candidate frame is mapped to the substation equipment image to obtain the recognition frame of the substation equipment image; Based on the identification image contained in the identification frame in the substation equipment image, an abnormal area of ​​the substation equipment is determined.

6. The method according to claim 5, characterized in that The number of the identification frames is multiple; The determining the abnormal area of ​​the substation equipment based on the identification image contained in the identification frame in the substation equipment image includes: Determine the identification images contained in the identification frames in the substation equipment image; For each of the a priori categories, when there is image overlap among the multiple recognition images corresponding to the a priori category, determining at least two target images in an image overlap state and candidate images that are not in an image overlap state among the multiple recognition images corresponding to the a priori category; Determine, among the target images, a selected image whose confidence of the prior category satisfies a confidence condition; Based on the selected images and candidate images corresponding to the respective a priori categories, abnormal areas of the substation equipment are determined.

7. A device for identifying abnormalities of equipment, characterized in that: The device comprises: An acquisition module, used for acquiring a feature map of a substation equipment image; A creation module, used to create multiple prior frames for the feature map according to the set prior frame size; A classification module, for classifying, for each of the a priori frames, the image information contained in the a priori frame in the feature map, and determining a priori categories corresponding to the a priori frames; The identification module is used to map the priori box to the substation equipment image according to the mapping relationship between the feature map and the substation equipment image when the priori category is an abnormal category, so as to determine the abnormal area of ​​the substation equipment.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images