Substation equipment inspection image automatic naming method based on intelligent identification and rapid matching

Abstract

The invention is suitable for the technical field of substation inspection, and discloses a substation equipment inspection image automatic naming method based on intelligent identification and rapid matching, which comprises the following steps: receiving a substation equipment image training set, and establishing image feature point description information for the preprocessed image by using an image identification algorithm; training the preprocessed power transformation equipment image training set by using machine learning; the method comprises the following steps: receiving a to-be-identified power transformation equipment image, preprocessing the to-be-identified power transformation equipment image, extracting feature points, carrying out image search based on a feature point detection SIFT algorithm, and identifying the type of power transformation equipment; and carrying out image similarity analysis by utilizing a maximum cross-correlation matching method, calculating the position deviation between the picture in the image recognition target frame and the template picture, obtaining the category of the power equipment, and naming the power equipment.

Description

Technical Field

[0001] The present invention belongs to the technical field of power systems, and in particular relates to an automatic naming method for substation equipment inspection images based on intelligent recognition and fast matching. Background Art

[0002] In recent years, based on the application of image processing and recognition technology in power systems, some beneficial explorations have been carried out, and some gratifying achievements have been made. However, at present, it still depends on the on-duty personnel to directly observe and analyze the collected images to judge the operating status of the power equipment. There is a lack of automatic recognition and analysis functions for substation power equipment. This lags behind other industries. The reason is that the analysis of substation images with complex backgrounds and the research on the judgment methods of power equipment operation failures are still immature.

[0003] CN115062178A discloses a method, device, computer equipment and storage medium for organizing unmanned aerial vehicle equipment inspection images. The method includes: obtaining equipment inspection images collected during the unmanned aerial vehicle inspection of power transmission equipment, and determining the inspection route of the unmanned aerial vehicle; then, the image shooting points of the equipment inspection images can be obtained, the image shooting points can be matched with each inspection shooting point in the inspection route, the target inspection shooting point corresponding to the equipment inspection image can be determined, the power transmission equipment located at the target inspection shooting point can be determined, and the equipment inspection image can be named according to the identification associated with the power transmission equipment, so as to obtain the image name including the identification device associated with the power transmission equipment. By matching the equipment inspection image shooting point with each inspection shooting point, the power transmission equipment located at the target inspection shooting point can be determined, and the equipment inspection image can be automatically named using the identification associated with the power transmission equipment, so as to improve the efficiency of unmanned aerial vehicle inspection image organization. However, there is still room for improvement. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the present invention proposes an automatic naming method for substation equipment inspection images based on intelligent recognition and fast matching. The specific steps of an automatic naming method for substation equipment inspection images based on intelligent recognition and fast matching are as follows:

[0005] Step 1: receiving a training set of substation equipment images, using an image recognition algorithm to establish image feature point description information for the preprocessed images, and using machine learning to train the preprocessed training set of substation equipment images;

[0006] Step 2: Receive the substation equipment image to be identified, pre-process it, extract feature points, and perform image search based on the feature point detection SIFT algorithm;

[0007] Step 3: Use the maximum cross-correlation matching method to perform image similarity analysis and calculate the position deviation between the image in the image recognition target box and the template image. When the obtained deviation is the smallest, name the device in the current image.

[0008] Secondly, in the step 1, in this method, the specific steps of the pretreatment include:

[0009] Accepting a valid position identification command, marking a valid position that can be used for identification in the substation equipment image;

[0010] Furthermore, in the method, the substation equipment image with the valid position marked is corrected and de-skewed.

[0011] Furthermore, in the method, the image recognition algorithm is used to establish image feature point description information for the preprocessed image and name the device.

[0012] Furthermore, in the step 2, the substation equipment image to be identified is received, pre-processed, feature points are extracted, and image search is performed based on the feature point detection SIFT algorithm.

[0013] Furthermore, in step three, the maximum cross-correlation matching method is used to perform image similarity analysis, and the position deviation between the image in the image recognition target frame and the template image is calculated. When the obtained deviation is the smallest, the device in the current image is named. The specific steps are:

[0014] make Is a length and width The picture (denoted as W), Is a length and width The image (denoted as M) is used to find the parts similar to M in W using correlation matching. Represents W in A The sub-block of W whose upper left corner is the same size as M also represents the matrix corresponding to the sub-block, that is:

[0015] ,

[0016] ,

[0017] In the formula, represent The correlation coefficient with M, E is the variance of M; for Covariance with M.

[0018] ,

[0019] ,

[0020] ,

[0021] In the formula, and M represent images respectively and the grayscale mean of M, yes The gray value of the pixel in the i-th row and j-th column, is the grayscale value of the pixel in the i-th row and j-th column in M.

[0022] If σ(x, y) is large or close to 1, it means that image M matches image W at point (x, y). For a given image W, we need to scan from the upper left corner to the lower right corner, and record (Aa)×(Bb) σ(x, y) in total. The point (x, y) corresponding to the maximum value is the point that matches image W. The final result is the deviation value of the two images in the horizontal and vertical directions. The method is as follows: take two images, W is the current image, M is a small area in the center of the template image, S x，y is any area in W that is exactly the same size as M. Scan the image M as above and determine the S corresponding to the largest σ(x,y). x，y The center coordinate of the image W is subtracted from the center coordinate of the image W to obtain the deviation between the current image and the template image (c x , c y ). When the obtained deviation is the smallest, it is the device in the template image, and the device in the current image is named according to the device name in the template image.

[0023] Compared with the prior art, the beneficial effects of the present invention include:

[0024] The present invention provides an automatic naming method for substation equipment inspection images based on intelligent recognition and fast matching. The method has high detection target recognition accuracy and fast matching speed in the substation scene, and is an automatic naming method for substation equipment inspection images in actual projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0026] Figure 1 This is a flow chart of an automatic naming method for substation equipment inspection images based on intelligent recognition and fast matching.

[0027] Figure 2 The present invention is a specific flow chart of a method for automatic naming of substation equipment inspection images based on intelligent recognition and fast matching.

[0028] Figure 3 For automatic naming of pre-substation equipment images.

[0029] Figure 4 The image of the substation equipment is automatically named. DETAILED DESCRIPTION

[0030] Step 1: receiving a training set of substation equipment images, using an image recognition algorithm to establish image feature point description information for the preprocessed images, and using machine learning to train the preprocessed training set of substation equipment images;

[0031] Due to the defects of the image acquisition equipment itself and the influence of environmental factors, the power equipment images input into the computer will inevitably contain distortion and noise, which will seriously interfere with the subsequent image processing, feature extraction and recognition analysis, and affect the correctness of the processing results.

[0032] Image preprocessing: Marking the valid position of the image: receiving the valid position identification instruction, marking the valid position in the power equipment image; by inputting a large number of physical photos of objects, the developer uses the computer to pre-mark the valid position in the image (such as nameplates, equipment panels, wiring layouts, etc.). Furthermore, in this method, the power equipment image with the valid position marked is corrected and rectified.

[0033] Feature point analysis: Substation images contain important information about the operation of power equipment. The key to identifying power equipment images in substations is to analyze the various features of power equipment. After preprocessing, image features that can distinguish the types of power equipment are selected as input vectors for identifying power equipment. Pattern recognition methods are used to classify and identify substation power equipment, and the results of the recognition experiment are summarized and analyzed.

[0034] Feature point extraction: The so-called feature point is the local extreme point with directional information detected in images of different scale spaces. Feature points have three characteristics: scale, direction, and size.

[0035] For images of normal size, we can calculate ~ Each eigenvalue is a 128-dimensional floating-point vector. The spatial information of the vector is used to construct a kd tree, that is, a k-dimensional vector tree, where k = 128.

[0036] A. Generate Gaussian Difference Pyramid (DOG Pyramid), scale space construction

[0037] By scaling the original image, we obtain scale space representation sequences of the image at multiple scales. We extract the main contours of the scale space from these sequences, and use the main contours as a feature vector to achieve edge and corner detection and key point extraction at different resolutions.

[0038] B. Spatial extreme point detection (preliminary exploration of key points).

[0039] C. Accurate positioning of stable key points.

[0040] D. Stable key point direction information distribution.

[0041] E. Description of key points.

[0042] The description of key points is a key step in the subsequent matching. The description is actually a process of defining the key points in a mathematical way. The descriptor contains not only the key points, but also the neighborhood points around the key points that contribute to them.

[0043] Image search: Image search is to calculate the Euclidean distance of vectors and find the sample vector with the smallest distance. The smaller the Euclidean distance, the higher the similarity. When the Euclidean distance is less than the set threshold, it can be determined that the match is successful.

[0044] In order to exclude key points with no matching relationship due to image occlusion and background clutter, the SIFT matching method that compares the nearest neighbor distance and the next nearest neighbor distance is used: take a SIFT key point in one image, and find the first two key points with the closest Euclidean distance in the other image. Among these two key points, if the ratio of the closest distance divided by the next closest distance is less than a certain threshold T, then accept this pair of matching points. Because for wrong matches, due to the high dimensionality of the feature space, similar distances may have a large number of other wrong matches, so its ratio value is relatively high. Obviously, if this ratio threshold T is lowered, the number of SIFT matching points will decrease, but it will be more stable, and vice versa. The experimental ratio value principle is shown in the following table:

[0045]

[0046] The idea described is: divide the pixel area around the key point into blocks, calculate the fast inner gradient histogram, and generate a unique vector, which is an abstract representation of the image information in the area.

[0047] The image recognition algorithm of OPENCV is used to correct and rectify the marked areas in these photos, and an image feature point description information and device naming information database are established.

[0048] Step 2: Receive the substation equipment image to be identified, pre-process it, extract feature points, and perform image search based on the feature point detection SIFT algorithm;

[0049] 1. Extract feature points of the image to be identified;

[0050] 2. Match the image to be identified with the library. If the match is successful, return the ID of the corresponding image;

[0051] 3. Query the associated power equipment through the obtained ID.

[0052] Image training and matching:

[0053] The training set images need to be consistent with the image environment to be recognized in the actual scene;

[0054] Considering the various possibilities of actual application scenarios, each labeled image needs to cover the possibilities in the actual scene, such as shooting angles and changes in light brightness. The more scenarios the training set covers, the stronger the generalization ability of the model.

[0055] Of course, during real-time recognition, the object is not exactly the same as that in the previous training. There may be differences in perspective, model, color, size, etc. Then the algorithm will sort according to the size of these differences, putting the smallest difference in front and the larger difference in the back, that is, the sorting of the confidence of the results. Finally, a confidence threshold will be determined based on the actual recognition effect. We will then consider the recognition above this threshold to be effective and can be used. Furthermore, in this method, in the image search and matching of the power equipment image to be identified, the confidence is calculated, sorted, and a confidence threshold is set. The calculated confidence greater than the confidence threshold is an effective recognition.

[0056] Step 3: Use the maximum cross-correlation matching method to perform image similarity analysis and calculate the position deviation between the image in the image recognition target box and the template image. When the obtained deviation is the smallest, name the device in the current image.

[0057] In order to realize automatic naming of unnamed devices, this paper uses the maximum cross-correlation matching method to perform image similarity analysis to calculate whether the device in the current image is the same device as the device in the template image. The maximum cross-correlation matching method model is as follows:

[0058] make Is a length and width The picture (denoted as W), Is a length and width The image (denoted as M) is used to find the parts similar to M in W using correlation matching. Represents W in A The sub-block of W whose upper left corner is the same size as M also represents the matrix corresponding to the sub-block, that is:

[0059] ,

[0060] ,

[0061] In the formula, represent The correlation coefficient with M, E is the variance of M; for Covariance with M.

[0062] ,

[0063] ,

[0064] ,

[0065] In the formula, and M represent images respectively and the grayscale mean of M, yes The gray value of the pixel in the i-th row and j-th column, is the grayscale value of the pixel in the i-th row and j-th column in M.

[0066] If σ(x, y) is large or close to 1, it means that image M matches image W at point (x, y). For a given image W, it is necessary to scan from the upper left corner to the lower right corner, and a total of (Aa)×(Bb) σ(x, y) need to be recorded. The point (x, y) corresponding to the maximum value is the point that matches image W. The final result is the deviation value of the two images in the horizontal and vertical directions respectively. The method is as follows: In the two images, W is the current image, M is a small area in the center of the template image, and S x,y is any area in W that is exactly the same size as M. Scan the image M in the above way and determine the S corresponding to the largest σ(x, y). x,y The center coordinate of the image W is subtracted from the center coordinate of the image W to obtain the deviation between the current image and the template image (c x , c y ). When the obtained deviation is the smallest, it is the device in the template image, and the device in the current image is named according to the device name in the template image.

[0067] The present invention provides an automatic naming method for substation equipment inspection images based on intelligent recognition and fast matching. The method has high detection target recognition accuracy and fast matching speed in the substation scene, and is an automatic naming method for substation equipment inspection images in actual projects.

[0068] The above embodiments are only preferred technical solutions of the present invention and should not be regarded as limiting the present invention. The protection scope of the present invention shall be the technical solutions recorded in the claims, including equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.

[0069] Example: See Figure 1-2 , a method for automatically naming substation equipment inspection images based on intelligent recognition and fast matching, including the following specific steps:

[0070] 1. Receive a training set of substation equipment images, use an image recognition algorithm to establish image feature point description information for the preprocessed images, and use machine learning to train the preprocessed training set of substation equipment images;

[0071] 2. Receive the substation equipment image to be identified, pre-process it, extract feature points, perform image search based on feature point detection SIFT algorithm, and identify the substation equipment;

[0072] 3. Use the maximum cross-correlation matching method to perform image similarity analysis and calculate the position deviation between the image in the image recognition target box and the template image. When the obtained deviation is the smallest, name the device in the current image.

[0073] See also Figure 3-4 , an automatic naming method for substation equipment inspection images based on intelligent recognition and fast matching. When used specifically: substation equipment images are not named before operation, and substation images are automatically named after operation. In the substation scene, the detection target recognition accuracy is high and the matching speed is fast. It is an automatic naming method for substation equipment inspection images in actual projects.

[0074] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for automatically naming substation equipment inspection images based on intelligent recognition and fast matching, characterized in that: The specific steps are: Step 1: receiving a training set of substation equipment images, using an image recognition algorithm to establish image feature point description information for the preprocessed images, and using machine learning to train the preprocessed training set of substation equipment images; Step 2: Receive the substation equipment image to be identified, pre-process it, extract feature points, perform image search based on feature point detection SIFT algorithm, and identify the substation equipment; Step 3: Use the maximum cross-correlation matching method to perform image similarity analysis and calculate the position deviation between the image in the image recognition target box and the template image. When the obtained deviation is the smallest, name the device in the current image.

2. According to claim 1, a method for automatically naming substation equipment inspection images based on intelligent recognition and fast matching is characterized in that: In step 1, the specific steps of preprocessing include: A training set of power equipment images is received, the power equipment images are preprocessed, image feature point description information is established for the preprocessed images using an image recognition algorithm, and the equipment is named.

3. The method for automatically naming substation equipment inspection images based on intelligent recognition and fast matching according to claim 2 is characterized in that: In step 2, the image of the electric power equipment to be identified is received, the image of the electric power equipment to be identified is preprocessed, feature points in the preprocessed image of the electric power equipment are extracted, and image search is performed based on a feature point detection SIFT algorithm; In the image search and matching of the power equipment image to be identified, the confidence is calculated, the confidence is sorted, and a confidence threshold is set. The calculated confidence greater than the confidence threshold is considered to be an effective identification.

4. According to claim 1, a method for automatically naming substation equipment inspection images based on intelligent recognition and fast matching is characterized in that: In step 3, the maximum cross-correlation matching method is used to perform image similarity analysis, and the position deviation between the image in the image recognition target frame and the template image is calculated. When the obtained deviation is the smallest, the device in the current image is named; In order to realize automatic naming of unnamed devices, this paper uses the maximum cross-correlation matching method to perform image similarity analysis to calculate whether the device in the current image is the same device as the device in the template image. The maximum cross-correlation matching method model is as follows: make It is a length and width The picture (denoted as W), It is a length and width The image (denoted as M) is used to find the parts similar to M in W using correlation matching. Represents W in A The sub-block of W whose upper left corner is the same size as M also represents the matrix corresponding to the sub-block, that is: ， ， In the formula, represent The correlation coefficient with M, E is the variance of M; for The covariance with M, ， ， ， In the formula, and M represent images respectively and the grayscale mean of M, yes The gray value of the pixel in the i-th row and j-th column, is the grayscale value of the pixel in the i-th row and j-th column in M; if is large or close to 1, it means that the image M is Points are matched with image W. For a given image W, it is necessary to scan from the upper left corner to the lower right corner, and a total of indivual , whose maximum value corresponds to Point is the point that matches the image W. The final result is the deviation value of the two images in the horizontal and vertical directions respectively. The method is as follows: W is the current image, M is a small area in the center of the template image, is any area in W that is exactly the same size as M. Scan the image M according to the above method to determine the largest Corresponding The center coordinates of the image W are subtracted from the center coordinates of the image W to obtain the deviation between the current image and the template image. , when the obtained deviation is the smallest, it is the device in the template image, and the device in the current image is named according to the device name in the template image.

Images