Image-difference-based power device state change detection method

Abstract

The application discloses a power device state change detection method based on image difference, acquires a power device image data set, and contains images in a normal state and part of images in an abnormal state; the images in the normal state and a to-be-compared image are simultaneously input into an ORB feature extraction algorithm for preliminary registration, and the registered regional images are subjected to differential processing by using the registration result; then, binarization processing is carried out to highlight the difference regions, and finally, a MSER algorithm is used to detect suitable regions; the maximum number of key points to be positioned by the ORB detection is adjusted according to the regional images; the images in the normal state and the images in the abnormal state of the power equipment are input, and a detected detection frame is output. The abnormal target detection method based on image registration and then image difference and finally target detection is used, and the application solves the problem of detecting some device abnormalities in the same category of images.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and specifically to a method for detecting changes in the state of power equipment based on image difference. Background Technology

[0002] Object detection generally consists of two stages: object localization and object classification. Localization determines the object's category and location, and bounding boxes are commonly used to label the detection results. Object detection is a crucial pillar of computer vision and image processing, and a core algorithm driving significant advancements in artificial intelligence. Object detection technology has wide applications, including face detection, vehicle detection, pedestrian counting, autonomous driving, and medical assistance.

[0003] Currently, deep learning-based object detection algorithms have achieved satisfactory results, surpassing manual recognition and detection in many scenarios, such as security monitoring, crowd control, and medical disease detection and diagnosis. However, these algorithms require large amounts of data for training, and their application is significantly limited when initial images are not readily available. Therefore, using traditional image processing methods to solve the object detection problem is of practical significance.

[0004] Currently, there are some existing technologies that extract features using ORB (Oriented Representational Block), such as patent CN110309831A, which discloses a non-smart water meter reading method based on machine vision. This method first reads a user-uploaded water meter image and a template image. It then uses the SIFT algorithm to obtain key points between the water meter image and the template image, matching these key points to determine the position of the template image within the water meter image. Through key point matching, the location of matching points in the feature region is obtained, thus determining the location of the feature region. After determining the feature region's location, it is cropped. The cropped feature region image is corrected using the matching points of the feature region to obtain a correction angle, thereby correcting the water meter image. The corrected water meter image is then repositioned to determine the location of the feature region, thus obtaining the water meter reading position. The water meter reading region is cropped, binarized, and denoised, then segmented into images of individual digit characters. Finally, the template matching algorithm is used to identify the digits and output them. However, this patent requires a specific template image and can only perform single-image, single-water-meter recognition. During the detection process, after feature point matching, the area to be detected in the image is uncertain, as are the anomalies to be detected.

[0005] For example, patent CN114972458A discloses a method and system for registering visible light and infrared thermal imaging images, which involves fitting affine transformation curves; acquiring images and distance information of the target object; using a single-stage target detection model to identify and locate the target, and outputting the target pixel region; initial registration of the visible light image and the infrared thermal imaging image; and precise registration of the visible light image and the infrared thermal imaging image to obtain feature point pairs. Since the positions of the processed object's thermal infrared image and its original image are basically close, except that one is a thermal infrared pixel image and the other is a visible light image, this technical solution can be used. However, if the processed image is not a visible light image and an infrared thermal image, the matching is very inaccurate. Summary of the Invention

[0006] Purpose of the invention: The purpose of this invention is to address the shortcomings of existing technologies and provide a method for detecting changes in the state of power devices based on image difference. This method is an abnormal target detection method that performs image registration, image difference, and finally target detection. This invention solves the problem of detecting abnormalities in some devices in images of the same category.

[0007] Technical solution: The present invention provides a method for detecting state changes in power devices based on image difference, specifically including the following steps:

[0008] Step 1: Collect images of different categories of power equipment to obtain an image dataset. Each category of power equipment images must include images under normal conditions and some images under abnormal conditions.

[0009] Step 2: Classify the images of power devices in the image dataset obtained in Step 1 according to different discrimination categories, that is, classify abnormal state images into the abnormal category and normal state images into the normal category.

[0010] Step 3: Input the normal image and the image to be compared into the ORB feature extraction algorithm simultaneously. Perform preliminary registration using the ORB feature extraction algorithm. The specific method is as follows:

[0011] Step 3.1: First, extract the corresponding feature points from the normal state image and the image to be compared based on the FAST algorithm;

[0012] For a pixel p in an image, its gray value is Ip. Consider a discretized Bresenham circle with a radius of 3 centered on this pixel, with 16 pixels on the circle's boundary. Set a suitable threshold t. If the gray values ​​of n consecutive pixels on the circle's boundary are less than Ip-t or greater than Ip+t, then this pixel can be identified as a feature point (corner point). n takes the value 12 or 9.

[0013] Step 3.2: Within the description region (coefficient * scale) of the feature points obtained in Step 3.1, 256 position pairs are randomly generated; for each position pair, if the pixel value of position 1 is greater than the pixel value of position 2, the output of that position pair is 1, otherwise it is 0; since there are a total of 256 position pairs, the resulting ORB feature description is a binary feature vector group, in which all 256 elements are binary numbers;

[0014] Step 3.3: Use the corresponding ORB feature descriptors obtained from the normal state image and the image to be compared to perform key point matching through homography matrix transformation;

[0015] Step 4: Using the registration results of key point matching obtained in Step 3.3, first perform differential processing on the registered region images; then perform binarization processing to highlight the difference regions; finally, use the MSER algorithm to detect suitable regions.

[0016] Step 5: Adjust parameters based on the region image, including the maximum number of key points to be located by ORB detection; The image detected by this invention is in a complex scene, and the state of the power device in the two images that need to be aligned is quite different. In order to determine the parameters, visualization processing is performed in the key point detection, registration, homography transformation and difference point detection stages. The parameter of the maximum number of key points to be located by ORB detection can be adjusted according to the visualization results during key point detection.

[0017] Step 6: After adjustment, simultaneously input images of the power equipment under normal and abnormal conditions, and output the detected detection boxes.

[0018] Furthermore, the specific method of step 3.1 is as follows:

[0019] Step 3.1.1: For positions 1 and 9 on the circular boundary of pixel p in the image, first determine whether the gray values ​​are both less than Ip-t or greater than Ip+t. If so, then pixel p is determined to be a feature point; otherwise, pixel p is determined not to be a feature point. Here, the determination is based on the pixel value of a single pixel.

[0020] Step 3.1.2: Determine if any three gray values ​​at positions 1, 9, 5, and 13 on the circular boundary of pixel p are all less than Ip-t or greater than Ip+t. If so, pixel p is considered a feature point; otherwise, pixel p is not considered a feature point. Here, three related points are considered simultaneously to determine whether a pixel is a key point, thus improving accuracy.

[0021] Step 3.1.3: Determine whether there are n consecutive pixels whose gray values ​​are less than Ip-t or greater than Ip+t. If so, the current pixel can be identified as a feature point (corner point). Here, n refers to the point on the circular boundary of the pixel to be judged.

[0022] Further, in step 4, the obtained ORB feature descriptors are first subjected to homography transformation, and the detection points of the normal state image are mapped to the corresponding positions in the image to be compared. The range of detection point positions in the normal state image is determined by the location of the region where a correct matching pair exists, thus obtaining the difference points between the two images. To obtain the bounding boxes of the difference parts, the obtained difference points are clustered to form multiple elliptical regions. For elliptical regions with a certain degree of repetition and the repetition exceeds the threshold, they are determined to be difference regions. At this time, the difference regions are merged into one bounding box (for example, if the repetition of the elliptical region is higher than 0.2, it can be determined that the two regions detected the same anomaly). For elliptical boxes (i.e., elliptical regions) containing fewer points, they are determined to be noise regions and discarded.

[0023] The homography matrix transformation mentioned above refers to the coordinate transformation of corresponding points on the image plane of the normal state image and the image to be compared.

[0024] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0025] (1) The present invention can achieve a relatively ideal detection effect by analyzing and testing the characteristics of the power equipment image itself when the sample size is small but the image features are obvious.

[0026] (2) The computational load required by this invention is relatively small compared to deep learning, and its memory space is less than 10MB.

[0027] (3) In order to solve the problem of low matching accuracy of existing ORB algorithms, this invention uses homography matrix matching, which can obtain better matching results. Attached Figure Description

[0028] Figure 1 This is an overall flowchart of the present invention;

[0029] Figure 2 To register images of the same type in the example;

[0030] Figure 3 The grayscale image obtained by performing differential processing on images of the same category in the embodiment;

[0031] Figure 4 This is a diagram showing the effect of using the algorithm with adjusted parameters to detect images in the example.

[0032] Figure 5 This is a diagram showing the effect of using the algorithm with adjusted parameters to detect images in the example.

[0033] Figure 6 This is a schematic diagram of the homography matrix matching results in the embodiment;

[0034] Figure 7 This is a schematic diagram illustrating the comparison and pairing of a normal state and an image to be detected simultaneously, as shown in the example. Detailed Implementation

[0035] The technical solution of the present invention will be described in detail below, but the scope of protection of the present invention is not limited to the embodiments described.

[0036] like Figure 1 As shown, the image difference-based power device state change detection method of the present invention specifically includes the following steps:

[0037] Step 1: Collect images of different categories of power equipment to obtain an image dataset. Each category of power equipment images must include images under normal conditions and some images under abnormal conditions.

[0038] Step 2: Classify the images of power devices in the image dataset obtained in Step 1 according to different discrimination categories, that is, classify abnormal state images into the abnormal category and normal state images into the normal category.

[0039] Step 3: Input the normal image and the image to be compared into the ORB feature extraction algorithm simultaneously. Perform preliminary registration using the ORB feature extraction algorithm. The specific method is as follows:

[0040] Step 3.1: First, extract the corresponding feature points from the normal state image and the image to be compared based on the FAST algorithm;

[0041] For a pixel p in an image, its gray value is Ip. Consider a discretized Bresenham circle with a radius of 3 centered on this pixel, with 16 pixels on the circle's boundary. Set a suitable threshold t. If the gray values ​​of n consecutive pixels on the circle's boundary are less than Ip-t or greater than Ip+t, then this pixel can be identified as a feature point (corner point). n takes the value 12 or 9.

[0042] Step 3.2: Within the description region (coefficient * scale) of the feature points obtained in Step 3.1, 256 position pairs are randomly generated; for each position pair, if the pixel value of position 1 is greater than the pixel value of position 2, the output of that position pair is 1, otherwise it is 0; since there are a total of 256 position pairs, the resulting ORB feature description is a binary feature vector group, in which all 256 elements are binary numbers;

[0043] Step 3.3: Use the corresponding ORB feature descriptors obtained from the normal state image and the image to be compared to perform key point matching through homography matrix transformation;

[0044] Step 4: Using the registration results obtained in Step 3.3, first perform image difference processing; then perform binarization processing to highlight the difference regions; finally, use the MSER algorithm to detect suitable regions.

[0045] Step 5: Based on the image, adjust the appropriate method parameters, such as the maximum number of key points to be located using ORB detection;

[0046] Step 6: After adjustment, simultaneously input images of the power equipment under normal and abnormal conditions, and output the detected bounding boxes.

[0047] Example 1

[0048] This embodiment is... Figure 2 The electrical equipment (fire extinguishing device) is subjected to anomaly detection. The specific process is as follows:

[0049] Step 1: Collect images of the placement of fire-fighting equipment, including its normal state. Figure 2 (middle left image) and abnormal states ( Figure 2 (Right image in the middle)

[0050] Step 2: Divide the images into different categories according to different devices. Each category should contain images of normal status and at least one image of abnormal status.

[0051] Step 3: Perform preliminary image registration, image differencing, and binarization sequentially (e.g., ...). Figure 3 (As shown) and area detection, preliminary registration results are as follows: Figure 4 and Figure 5 As shown, the homography matrix matching results are as follows: Figure 6 As shown.

[0052] Step 4: Adjust the parameters according to the image registration results to adapt to the scene detection of the device.

[0053] Step Six: Configure the final algorithm in the appropriate environment. When using it, input both the normal state and the image to be detected for comparison and matching, such as... Figure 7 As shown.

[0054] As can be seen from the above embodiments, the present invention is suitable for power equipment condition detection in complex scenarios, and does not require template images. It can detect a single type of power equipment, thus improving efficiency. In the actual detection process, after feature point matching, the area to be detected and the abnormal points to be compared and detected in the image are basically determined. Then, based on the position and quantity of the abnormal points, appropriate abnormal areas are selected and marked. The present invention does not require a large amount of data to train the neural network, saving computation. In addition, since the angle position of the two images changes greatly, it is easy to generate a lot of noise. By using homography transformation to filter abnormal areas, abnormal areas with a significantly lower number of abnormal points than those at other positions in the same image are discarded as noise areas, thus improving detection accuracy.

Claims

1. A method for detecting state changes in power equipment based on image difference, characterized in that, Specifically, the following steps are included: Step 1: Collect images of different categories of power equipment to obtain an image dataset. Each category of power equipment images must include images under normal conditions and some images under abnormal conditions. Step 2: Classify the images of power devices in the image dataset obtained in Step 1 according to different discrimination categories, that is, classify abnormal state images into the abnormal category and normal state images into the normal category. Step 3: Input the normal image and the image to be compared into the ORB feature extraction algorithm simultaneously. Perform preliminary registration using the ORB feature extraction algorithm. The specific method is as follows: Step 3.1: First, extract the corresponding feature points from the normal state image and the image to be compared based on the FAST algorithm; For a pixel p in an image, its gray value is Ip. Consider a discretized Bresenham circle with a radius of 3 centered on this pixel. There are 16 pixels on the boundary of the Bresenham circle. Set a threshold t. If the gray values ​​of n consecutive pixels on the boundary of the Bresenham circle are less than Ip-t or greater than Ip+t, then this pixel can be identified as a feature point. n takes the value 12 or 9. Step 3.1.1: For positions 1 and 9 on the circular boundary of pixel p in the image, first determine whether the gray values ​​are both less than Ip-t or greater than Ip+t. If so, then determine that pixel p is a feature point; otherwise, determine that pixel p is not a feature point. Step 3.1.2: Determine whether there are three gray values ​​at positions 1, 9, 5 and 13 on the circular boundary of pixel p that are all less than Ip-t or greater than Ip+t. If so, then pixel p is a feature point. If not, then pixel p is not a feature point. Step 3.1.3: Determine if there are n consecutive pixels whose gray values ​​are less than Ip-t or greater than Ip+t. If so, the current pixel can be identified as a feature point. Step 3.2: Within the description region of the feature points obtained in Step 3.1, 256 position pairs are randomly generated. For each position pair, if the pixel value of position 1 is greater than the pixel value of position 2, the output of that position pair is 1, otherwise it is 0. Since there are a total of 256 position pairs, the resulting ORB feature description is a binary feature vector group, in which all 256 elements are binary numbers. Step 3.3: Use the corresponding ORB feature descriptors obtained from the normal state image and the image to be compared to perform keypoint matching through homography transformation; Step 4: Using the registration results of key point matching obtained in Step 3.3, first perform differential processing on the registered region images; then perform binarization processing to highlight the difference regions; finally, use the MSER algorithm to detect suitable regions. Step 4 first performs homography transformation on the obtained ORB feature descriptor and maps the detection points of the normal state image to the corresponding positions in the image to be compared; the range of the detection point positions in the normal state image is determined by the location of the region where a correct matching pair exists, thereby obtaining the difference points between the two images; To obtain the bounding boxes of the differences, the obtained differences are clustered to form multiple elliptical regions; For elliptical regions with a certain degree of repetition that exceeds the threshold, they are identified as difference regions, and these difference regions are then merged into a single label box. If the ellipse contains few points, it is identified as a noise region and discarded. Step 5: Adjust parameters based on the region image, including the maximum number of key points to be located using ORB detection; Step 6: After adjustment, simultaneously input images of the power equipment under normal and abnormal conditions, and output the detected detection boxes.

Images