A method and terminal for detecting surface defects of power equipment based on deep learning

Abstract

The present invention provides a method and terminal for detecting surface defects of power equipment based on deep learning, including: obtaining real-time pictures automatically and regularly collected by the power intelligent patrol system; using the deep learning template library to distinguish the real-time pictures, if the distinction fails, using the image recognition mode to distinguish the real-time pictures, if the distinction fails again, using manual marking processing; automatically marking the type, position and outline of the equipment for the real-time pictures successfully distinguished by the deep learning template library or the image recognition mode; importing the marked real-time pictures into the deep learning template library for deep learning, and after updating the deep learning template library, continuing to identify the next real-time picture. The present invention realizes a self-feedback image recognition processing method, makes full use of a large number of pictures collected by the intelligent patrol system, completes the automatic maintenance and automatic update of the template library of the power equipment, and realizes more accurate intelligent discrimination of the working status of the power equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid power transmission and transformation equipment status monitoring systems, and in particular to a method and terminal for detecting surface defects of power equipment based on deep learning. Background Art

[0002] With the rapid development of automation technology, many links in industrial production that require manual operation are gradually being completed by machines. Industrial production automation will also liberate more and more workers from boring work, allowing them to play a greater role.

[0003] Surface defect detection of power equipment is an important part of power grid safety production and a key step in the automatic inspection of intelligent patrol systems. With the help of surface defect detection technology, the quality and efficiency of intelligent patrol systems can be effectively improved. The traditional surface defect detection algorithm structure obtains an image that is easy to detect through image preprocessing, and then uses statistical machine learning methods to extract image features to achieve the goal of defect detection. Among them, the image preprocessing steps include: histogram equalization, filtering and denoising, grayscale binarization and re-filtering to obtain simplified image information with foreground and background separation, and then use mathematical morphology, Fourier transform, Gabor transform and other algorithms and machine learning models to complete the marking and detection of defects.

[0004] However, traditional surface defect detection algorithms require manual design of complex algorithm processes, which has led to the emergence of surface defect detection of power equipment based on deep learning algorithms.

[0005] The deep learning algorithm uses deep convolutional neural networks (DCNNs) combined with target detection algorithms such as SSD (Single Shot MultiBox Detector) and Yolo (You Only Look Once) to build a coarse-to-fine cascade detection network, including equipment positioning, defect detection and classification. The specific steps are as follows:

[0006] (1) Device surface extraction: With the help of the SSD framework, which performs well in terms of speed and accuracy, the cantilever nodes in the image are located, and the casing is located based on the fast localization architecture of the Yolo framework;

[0007] (2) Equipment surface defect detection and classification: Defects are judged based on the detection of the equipment surface in the second stage, and defects are classified through four convolutional layers with the help of DCNN.

[0008] Deep learning algorithms avoid the need for traditional algorithms to manually design complex algorithmic processes, and have extremely high robustness and accuracy. However, the essence of deep learning is machine learning, and its most fundamental principle lies in statistics, that is, it requires sufficient data, through the definition of models, collection and analysis of data, injection of training to improve the model and finally output structure, and the above process is repeated to continuously improve accuracy. In short, deep learning algorithms need to provide enough and various device image templates for algorithm training and modeling.

[0009] The device image template needs to be trained manually by manually marking the device on the picture, and the marking requires manual operation, that is, marking the device on each picture, and completing the training of the deep learning template through the marked pictures. In the process of using deep learning templates, from template picture → training → template → image recognition, it is a one-way process. In the recognition process, only pre-trained templates can be used, which is easy to cause inaccurate recognition. For pictures that cannot be recognized by the template, manual recognition is required, which reduces the recognition efficiency. Summary of the invention

[0010] The technical problem to be solved by the present invention is: to provide a method and terminal for detecting surface defects of power equipment based on deep learning, so as to realize more accurate intelligent judgment of the working status of power equipment through automatic maintenance and automatic updating of the template library of power equipment.

[0011] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method and terminal for detecting surface defects of power equipment based on deep learning, comprising the steps of:

[0012] S1. Obtain real-time images automatically and regularly collected by the power intelligent patrol system;

[0013] S2, using the deep learning template library to identify the real-time image, if the identification fails, proceed to step S3, otherwise proceed to step S4;

[0014] S3, using the image recognition mode to identify the real-time picture, if the identification fails, importing the real-time picture after manual marking into the deep learning template library for deep learning, after updating the deep learning template library, returning to S1 to identify the next real-time picture, otherwise entering step S4;

[0015] S4. Automatically mark the type, location and outline of the device on the real-time image, and import the marked real-time image into the deep learning template library for deep learning. After updating the deep learning template library, return to S1 to identify the next real-time image.

[0016] In order to solve the above technical problems, another technical solution adopted by the present invention is: a power equipment surface defect detection terminal based on deep learning, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0017] S1. Obtain real-time images automatically and regularly collected by the power intelligent patrol system;

[0018] S2, using the deep learning template library to identify the real-time image, if the identification fails, proceed to step S3, otherwise proceed to step S4;

[0019] S3, using the image recognition mode to identify the real-time picture, if the identification fails, importing the real-time picture after manual marking into the deep learning template library for deep learning, after updating the deep learning template library, returning to S1 to identify the next real-time picture, otherwise entering step S4;

[0020] S4. Automatically mark the type, location and outline of the device on the real-time image, and import the marked real-time image into the deep learning template library for deep learning. After updating the deep learning template library, return to S1 to identify the next real-time image.

[0021] The beneficial effects of the present invention are as follows: the present invention provides a method and terminal for detecting surface defects of power equipment based on deep learning. First, the real-time pictures automatically and regularly collected by the power intelligent patrol system are identified based on the pre-obtained deep learning template library. The unrecognizable pictures are distinguished by the image recognition mode. Both can realize automatic discrimination of the type, position and outline of the marked equipment. The pictures that cannot be distinguished by the image recognition mode are manually marked. After each marking, the marked real-time pictures are imported into the deep learning template library for training to update the deep learning template library, realize a self-feedback image recognition processing method, make full use of a large number of pictures collected by the intelligent patrol system, complete the automatic maintenance and automatic update of the power equipment template library, and realize more accurate intelligent discrimination of the working status of the power equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a main flow chart of a method for detecting surface defects of power equipment based on deep learning according to an embodiment of the present invention;

[0023] Figure 2 A specific flow chart of a method for detecting surface defects of power equipment based on deep learning according to an embodiment of the present invention;

[0024] Figure 3A schematic diagram of automatically marking a real-time image in an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of using manual marking to mark a real-time image in an embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of the structure of a deep learning-based surface defect detection terminal for power equipment according to an embodiment of the present invention.

[0027] Description of labels:

[0028] 1. A surface defect detection terminal for power equipment based on deep learning; 2. Memory; 3. Processor. DETAILED DESCRIPTION

[0029] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0030] Please refer to Figures 1 to 4 , a method for detecting surface defects of power equipment based on deep learning, comprising the steps of:

[0031] S1. Obtain real-time images automatically and regularly collected by the power intelligent patrol system;

[0032] S2, using the deep learning template library to identify the real-time image, if the identification fails, proceed to step S3, otherwise proceed to step S4;

[0033] S3, using the image recognition mode to identify the real-time picture, if the identification fails, importing the real-time picture after manual marking into the deep learning template library for deep learning, after updating the deep learning template library, returning to S1 to identify the next real-time picture, otherwise entering step S4;

[0034] S4. Automatically mark the type, location and outline of the device on the real-time image, and import the marked real-time image into the deep learning template library for deep learning. After updating the deep learning template library, return to S1 to identify the next real-time image.

[0035] From the above description, it can be seen that the beneficial effects of the present invention are: first, based on the pre-obtained deep learning template library, the real-time pictures automatically and regularly collected by the electric power intelligent patrol system are recognized, and the image recognition mode is used to distinguish the unrecognizable pictures. Both can realize automatic identification of the type, position and outline of the marked equipment, and the pictures that cannot be distinguished by the image recognition mode are manually marked, and after each marking, the marked real-time pictures are imported into the deep learning template library for training to update the deep learning template library, realize self-feedback image recognition processing method, make full use of the large number of pictures collected by the intelligent patrol system, complete the automatic maintenance and automatic update of the electric power equipment template library, and realize more accurate intelligent judgment of the working status of the electric power equipment.

[0036] Furthermore, in S3, the real-time image is identified by using an image recognition mode, specifically:

[0037] An image recognition template library is used to perform image recognition on the real-time pictures that cannot be recognized in the deep learning template library to determine the type, position and outline of the equipment in the real-time picture. The image recognition template library is composed of all historical pictures automatically and regularly collected by the electric power intelligent patrol system after identification, and each of the historical pictures includes the type, position and outline of the corresponding equipment.

[0038] From the above description, it can be seen that for real-time pictures that cannot be recognized by the deep learning template library, further recognition is performed through the image recognition template library, so that the type, location and outline of the equipment in the real-time picture can be marked, further realizing accurate intelligent judgment of the working status of the power equipment.

[0039] Furthermore, in S3, the real-time image after manual labeling is imported into the deep learning template library for deep learning, specifically:

[0040] Receive the type and position of the equipment manually marked on the real-time image, and the outline of the equipment obtained by manually copying the real-time image using polygons, and import the real-time image and its corresponding type, position and outline of the equipment into the image recognition template library and the deep learning template library for deep learning.

[0041] From the above description, it can be seen that for real-time pictures that the image recognition template library cannot recognize, the staff will manually mark the equipment type, location and outline in the real-time picture to further realize the intelligent judgment of the working status of the power equipment. At the same time, the real-time pictures after marking will be imported into the image recognition template library to further enrich the template materials in the image recognition template library, that is, to continuously improve the image recognition template library, so that the next time a similar real-time picture is encountered, there is no need to mark it through complex machine recognition, but it can be directly judged through the image recognition template library, which further improves the efficiency of intelligent judgment.

[0042] Furthermore, the step of automatically marking the type, location and outline of the device on the real-time image in S4 further includes:

[0043] The real-time picture and the type and outline of the corresponding device are imported into the image recognition template library.

[0044] From the above description, it can be seen that each time the real-time image is marked with the equipment type, position and outline, the marked real-time image is imported into the image recognition template library, which can further enrich the template materials in the image recognition template library, that is, continuously improve the image recognition template library to improve the efficiency of intelligent identification of power equipment.

[0045] Furthermore, the step of automatically marking the type, location and outline of the device on the real-time image in S4 specifically includes the following steps:

[0046] S41, obtaining the type of the device in the real-time picture by using the historical picture corresponding to the real-time picture in the image recognition template library identified in the image recognition mode, and taking the historical picture as a reference image and the real-time picture as an image to be processed;

[0047] S42, selecting four vertices of the polygonal outline of the device on the reference image as feature points P1, P2, P3, and P4;

[0048] S43, taking a certain feature point as the center, selecting a brightness data block T with a length and a width of M pixels as a data calculation basis for calculating similarity measurement, wherein the value of M is 34 to 40 pixels;

[0049] S44, using a global search method in the image to be processed to calculate the data block with the highest similarity to the data block T, the calculation formula is as follows:

[0050]

[0051] Among them, P i,jrepresents the image brightness data of M pixels in length and width centered at the coordinate position (i, j) in the image to be processed, where i is in the range of (M / 2) to (WM / 2), j is in the range of (M / 2) to (HM / 2), W is the width of the image to be processed, and H is the height of the image to be processed;

[0052] S45, counting the maximum values ​​of all R(i, j), where (i, j) is the position point on the image to be processed that matches the feature point in the reference image, and repeating steps S43 to S44 to obtain target position points T1, T2, T3 and T4 that match the feature points P1, P2, P3 and P4 respectively;

[0053] S46. Obtain the position of the device in the image to be processed according to the positions of the four target position points T1, T2, T3 and T4, and obtain the polygonal outline of the device in the image to be processed in combination with the polygonal outline of the device in the reference image.

[0054] From the above description, it can be seen that by performing coordinate calculations on the polygons of the device body outline in the real-time image based on the four feature points and calculating the minimum rectangle that can accommodate all the polygon vertices, the data used to mark the device position in the deep learning template can be obtained, thereby realizing automatic and rapid marking of the device position and outline. At the same time, the device's position data and outline data can also be used to continue training the deep learning template library, and further update the deep learning template library to improve the accuracy of intelligent discrimination of power equipment.

[0055] Please refer to Figure 5 , a power equipment surface defect detection terminal based on deep learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0056] S1. Obtain real-time images automatically and regularly collected by the power intelligent patrol system;

[0057] S2, using the deep learning template library to identify the real-time image, if the identification fails, proceed to step S3, otherwise proceed to step S4;

[0058] S3, using the image recognition mode to identify the real-time picture, if the identification fails, importing the real-time picture after manual marking into the deep learning template library for deep learning, after updating the deep learning template library, returning to S1 to identify the next real-time picture, otherwise entering step S4;

[0059] S4. Automatically mark the type, location and outline of the device on the real-time image, and import the marked real-time image into the deep learning template library for deep learning. After updating the deep learning template library, return to S1 to identify the next real-time image.

[0060] From the above description, it can be seen that the beneficial effects of the present invention are: based on the same technical concept, in conjunction with the above-mentioned deep learning-based surface defect detection method for power equipment, a deep learning-based surface defect detection terminal for power equipment is provided. First, based on the pre-obtained deep learning template library, the real-time pictures automatically and regularly collected by the power intelligent patrol system are recognized, and the unrecognizable pictures are judged by the image recognition mode. Both can automatically distinguish the type, position and contour of the marked equipment, and the pictures that cannot be judged by the image recognition mode are manually marked. After each marking, the marked real-time pictures are imported into the deep learning template library for training to update the deep learning template library, realize self-feedback image recognition processing method, make full use of the large number of pictures collected by the intelligent patrol system, complete the automatic maintenance and automatic update of the power equipment template library, and realize more accurate intelligent judgment of the working status of the power equipment.

[0061] Furthermore, in S3, the real-time image is identified by using an image recognition mode, specifically:

[0062] An image recognition template library is used to perform image recognition on the real-time pictures that cannot be recognized in the deep learning template library to determine the type, position and outline of the equipment in the real-time picture. The image recognition template library is composed of all historical pictures automatically and regularly collected by the electric power intelligent patrol system after identification, and each of the historical pictures includes the type, position and outline of the corresponding equipment.

[0063] From the above description, it can be seen that for real-time pictures that cannot be recognized by the deep learning template library, further recognition is performed through the image recognition template library, so that the type, location and outline of the equipment in the real-time picture can be marked, further realizing accurate intelligent judgment of the working status of the power equipment.

[0064] Furthermore, in S3, the real-time image after manual labeling is imported into the deep learning template library for deep learning, specifically:

[0065] Receive the type and position of the equipment manually marked on the real-time image, and the outline of the equipment obtained by manually copying the real-time image using polygons, and import the real-time image and its corresponding type, position and outline of the equipment into the image recognition template library and the deep learning template library for deep learning.

[0066] From the above description, it can be seen that for real-time pictures that the image recognition template library cannot recognize, the staff will manually mark the equipment type, location and outline in the real-time picture to further realize the intelligent judgment of the working status of the power equipment. At the same time, the real-time pictures after marking will be imported into the image recognition template library to further enrich the template materials in the image recognition template library, that is, to continuously improve the image recognition template library, so that the next time a similar real-time picture is encountered, there is no need to mark it through complex machine recognition, but it can be directly judged through the image recognition template library, which further improves the efficiency of intelligent judgment.

[0067] Furthermore, the step of automatically marking the type, location and outline of the device on the real-time image in S4 further includes:

[0068] The real-time picture and the type and outline of the corresponding device are imported into the image recognition template library.

[0069] From the above description, it can be seen that each time the real-time image is marked with the equipment type, position and outline, the marked real-time image is imported into the image recognition template library, which can further enrich the template materials in the image recognition template library, that is, continuously improve the image recognition template library to improve the efficiency of intelligent identification of power equipment.

[0070] Furthermore, the step of automatically marking the type, location and outline of the device on the real-time image in S4 specifically includes the following steps:

[0071] S41, obtaining the type of the device in the real-time picture by using the historical picture corresponding to the real-time picture in the image recognition template library identified in the image recognition mode, and taking the historical picture as a reference image and the real-time picture as an image to be processed;

[0072] S42, selecting four vertices of the polygonal outline of the device on the reference image as feature points P1, P2, P3, and P4;

[0073] S43, taking a certain feature point as the center, selecting a brightness data block T with a length and a width of M pixels as a data calculation basis for calculating similarity measurement, wherein the value of M is 34 to 40 pixels;

[0074] S44, using a global search method in the image to be processed to calculate the data block with the highest similarity to the data block T, the calculation formula is as follows:

[0075]

[0076] Among them, P i,jrepresents the image brightness data of M pixels in length and width centered at the coordinate position (i, j) in the image to be processed, where i is in the range of (M / 2) to (WM / 2), j is in the range of (M / 2) to (HM / 2), W is the width of the image to be processed, and H is the height of the image to be processed;

[0077] S45, counting the maximum values ​​of all R(i, j), where (i, j) is the position point on the image to be processed that matches the feature point in the reference image, and repeating steps S43 to S44 to obtain target position points T1, T2, T3 and T4 that match the feature points P1, P2, P3 and P4 respectively;

[0078] S46. Obtain the position of the device in the image to be processed according to the positions of the four target position points T1, T2, T3 and T4, and obtain the polygonal outline of the device in the image to be processed in combination with the polygonal outline of the device in the reference image.

[0079] From the above description, it can be seen that by performing coordinate calculations on the polygons of the device body outline in the real-time image based on the four feature points and calculating the minimum rectangle that can accommodate all the polygon vertices, the data used to mark the device position in the deep learning template can be obtained, thereby realizing automatic and rapid marking of the device position and outline. At the same time, the device's position data and outline data can also be used to continue training the deep learning template library, and further update the deep learning template library to improve the accuracy of intelligent discrimination of power equipment.

[0080] Please refer to Figure 1 , Embodiment 1 of the present invention is:

[0081] A method for detecting surface defects of power equipment based on deep learning, comprising the steps of:

[0082] S1. Obtain real-time images automatically and regularly collected by the power intelligent patrol system.

[0083] In this embodiment, a large number of intelligent patrol devices such as high-definition cameras, robots or drones are installed on the electric power intelligent patrol system. These intelligent patrol devices collect working images of various equipment at regular times and locations every day. In this embodiment, each real-time picture can have multiple devices or only one device, but a real-time picture contains at least one device.

[0084] S2. Use the deep learning template library to identify the real-time image. If the identification fails, go to step S3, otherwise go to step S4.

[0085] The deep learning template library first used in this embodiment is a pre-trained template. The real-time image is first analyzed and judged through the pre-trained deep learning template library to determine the location and outline of the device in the image and the type of device, so as to judge the defects of the equipment.

[0086] S3. Use image recognition mode to identify the real-time image. If the identification fails, import the real-time image after manual marking into the deep learning template library for deep learning. After updating the deep learning template library, return to step S1 to identify the next real-time image, otherwise go to step S4.

[0087] In this embodiment, for real-time images that cannot be identified by the pre-trained deep learning template library, the image recognition mode is used for identification, and for images that cannot be identified by the image recognition mode, manual labeling is used, that is, further intelligent judgment of the device is achieved as a whole.

[0088] S4: automatically mark the type, location and outline of the device on the real-time image, and import the marked real-time image into the deep learning template library for deep learning. After updating the deep learning template library, return to S1 to identify the next real-time image.

[0089] That is, in this embodiment, each time a real-time image is identified, the equipment type, location and contour will be marked, and it will be re-imported into the deep learning template library for further deep learning training, instead of the existing identified images only entering the historical image library for historical data query and not for other purposes, the deep learning template library can be continuously updated and improved to realize the self-feedback image recognition processing method. That is, the large number of images collected by the intelligent patrol system are fully utilized to complete the automatic maintenance and automatic update of the power equipment template library, and realize more accurate intelligent identification of the working status of the power equipment.

[0090] Please refer to Figures 2 to 4 , Embodiment 2 of the present invention is:

[0091] A method for detecting surface defects of power equipment based on deep learning, based on the above-mentioned embodiment 1, in this embodiment, as Figure 2 Shown is a specific flow chart of a method for detecting surface defects of power equipment based on deep learning.

[0092] Among them, in step S3, the image recognition mode is used to identify the real-time image, specifically:

[0093] The image recognition template library is used to perform image recognition on the real-time images that cannot be recognized in the deep learning template library to determine the type, location and outline of the equipment in the real-time image. In this embodiment, the image recognition template library is composed of all historical images automatically and regularly collected by the power intelligent patrol system after identification, and each historical image includes the type, location and outline of the corresponding equipment.

[0094] That is, in this embodiment, for real-time pictures that cannot be recognized by the deep learning template library, further recognition is performed through the image recognition template library, so that the type, location and outline of the equipment in the real-time picture can be marked, further realizing accurate intelligent judgment of the working status of the power equipment.

[0095] Among them, in step S3, the real-time image after manual labeling is imported into the deep learning template library for deep learning, specifically:

[0096] Receive the type and position of the equipment manually marked on the real-time image, as well as the outline of the equipment obtained by manually copying the real-time image using polygons, and import the real-time image and its corresponding type, position and outline of the equipment into the image recognition template library and the deep learning template library for deep learning.

[0097] That is, in this embodiment, for real-time images that cannot be recognized by the image recognition template library, the staff manually marks the device type, location and outline in the real-time image, such as Figure 5 As shown, the staff uses polygons to copy the outline of the equipment in the real-time picture, which further realizes the intelligent judgment of the working status of the power equipment. At the same time, the real-time picture after marking is imported into the image recognition template library to further enrich the template materials in the image recognition template library, that is, to continuously improve the image recognition template library, so that the next time a similar real-time picture is encountered, there is no need to mark it through complex machine recognition, but it can be directly judged through the image recognition template library, which further improves the efficiency of intelligent judgment.

[0098] The step S4 of automatically marking the type, location and outline of the device on the real-time image also includes:

[0099] Import the real-time images and their corresponding device types and outlines into the image recognition template library.

[0100] That is, in this embodiment, as with the marking process in the above step S3, the real-time picture after each equipment type, position and contour marking is imported into the image recognition template library to further enrich the template materials in the image recognition template library, that is, to continuously improve the image recognition template library and improve the efficiency of intelligent identification of power equipment.

[0101] Among them, the type, location and outline of the device are automatically marked on the real-time image in step S4, which specifically includes the following steps:

[0102] S41, obtaining the type of the device in the real-time image by using the historical image corresponding to the real-time image in the image recognition template library recognized in the image recognition mode, and taking the historical image as the reference image and the real-time image as the image to be processed.

[0103] S42 , selecting four vertices of the polygonal outline of the device on the reference image as feature points P1 , P2 , P3 , and P4 .

[0104] S43, taking a certain feature point as the center, selecting a brightness data block T with a length and a width of M pixels as a data calculation basis for calculating similarity measurement;

[0105] In this embodiment, the value of M affects the computing intensity. The larger the value of M, the more computing resources are consumed, the slower the computing speed, but the higher the recognition accuracy. The smaller the value of M, the faster the computing speed, but the lower the recognition accuracy. Therefore, in actual use, the value of M can be adjusted as needed. Generally, the value of M is 34 to 40 pixels. In this embodiment, the value of M is set to 37.

[0106] S44, using a global search method in the image to be processed to calculate the data block with the highest similarity to the data block T, the calculation formula is as follows:

[0107]

[0108] Among them, P i,j Represents the image brightness data of M pixels in length and width centered at the coordinate (i, j) in the image to be processed, where i ranges from (M / 2) to (WM / 2), j ranges from (M / 2) to (HM / 2), W is the width of the image to be processed, and H is the height of the image to be processed.

[0109] S45. Count the maximum values ​​of all R(i,j), where (i,j) is the position point on the image to be processed that matches the feature point in the reference image, and repeat steps S43 to S44 to obtain target position points T1, T2, T3 and T4 that match the feature points P1, P2, P3 and P4 respectively.

[0110] S46, obtaining the position of the device in the image to be processed according to the positions of the four target position points T1, T2, T3 and T4, and obtaining the polygonal outline of the device in the image to be processed in combination with the polygonal outline of the device in the reference image.

[0111] That is, in this embodiment, if Figure 3As shown, by performing coordinate calculations on the polygon of the device body outline in the real-time image based on four feature points and calculating the minimum rectangle that can accommodate all polygon vertices, the data used to mark the device position in the deep learning template can be obtained, thereby achieving automatic and rapid marking of the device position and outline. At the same time, the device position data and outline data can also be used to continue training the deep learning template library and further update the deep learning template library to improve the accuracy of intelligent identification of power equipment. And in this embodiment, Figure 2 As shown, for real-time images that have been manually marked, their marked contours, positions, etc. will also be converted into minimum rectangles through the above-mentioned coordinate calculation based on the four feature points to provide training data for deep learning, thereby further updating the deep learning template library.

[0112] Among them, the data imported into the deep learning template library for further deep learning training also includes the type data of the equipment. In this embodiment, by automatically and regularly acquiring image data for training, the regular updating and maintenance of the deep learning template library is achieved, thereby further improving the accuracy of intelligent judgment of the working status of power equipment.

[0113] Please refer to Figure 5 , Embodiment 3 of the present invention is:

[0114] A deep learning-based surface defect detection terminal 1 for electric equipment includes a memory 2, a processor 3, and a computer program stored in the memory 2 and executable on the processor 3. In this embodiment, the processor 3 implements the steps in the above-mentioned embodiment 1 or embodiment 2 when executing the computer program.

[0115] In summary, the present invention provides a method and terminal for detecting surface defects of power equipment based on deep learning. First, the real-time pictures automatically collected by the power intelligent patrol system are identified based on the deep learning template library obtained in advance. The image recognition template library is used to further identify the pictures that cannot be identified, so as to further realize the intelligent discrimination of the working state of the power equipment accurately; and the pictures that cannot be identified by the image recognition template library are processed by manual marking to further realize the intelligent discrimination of the working state of the power equipment. Among them, for each real-time picture marked, the image recognition template library will be imported to enrich the template materials in the image recognition template library to improve the efficiency of the intelligent discrimination of the power equipment; and the real-time picture that is judged each time will be re-imported into the deep learning template library for further deep learning training, replacing the existing judged pictures that only enter the historical image library for historical data query and not for other purposes, and can continuously update and improve the deep learning template library to realize the self-feedback image recognition processing method. That is, a large number of pictures collected by the intelligent patrol system are fully utilized to complete the automatic maintenance and automatic update of the template library of the power equipment, so as to realize more accurate intelligent discrimination of the working state of the power equipment.

[0116] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting surface defects of power equipment based on deep learning, characterized in that: Includes steps: S1. Obtain real-time images automatically and regularly collected by the power intelligent patrol system; S2, using the deep learning template library to identify the real-time image, if the identification fails, proceed to step S3, otherwise proceed to step S4; S3, using the image recognition mode to identify the real-time picture, if the identification fails, importing the real-time picture after manual marking into the deep learning template library for deep learning, after updating the deep learning template library, returning to S1 to identify the next real-time picture, otherwise entering step S4; S4, automatically marking the type, location and outline of the device on the real-time picture, and importing the marked real-time picture into the deep learning template library for deep learning, and after updating the deep learning template library, returning to S1 to identify the next real-time picture; In S3, the real-time image is identified by using an image recognition mode, specifically: An image recognition template library is used to perform image recognition on the real-time pictures that cannot be recognized in the deep learning template library to determine the type, position and outline of the equipment in the real-time picture. The image recognition template library is composed of all historical pictures automatically and regularly collected by the power intelligent patrol system after identification, and each of the historical pictures includes the type, position and outline of the corresponding equipment; The S4 automatically marks the type, location and outline of the device on the real-time image, specifically including the following steps: S41, obtaining the type of the device in the real-time picture by using the historical picture corresponding to the real-time picture in the image recognition template library identified in the image recognition mode, and taking the historical picture as a reference image and the real-time picture as an image to be processed; S42, selecting four vertices of the polygonal outline of the device on the reference image as feature points P1, P2, P3, and P4; S43, taking a certain feature point as the center, selecting a brightness data block T with a length and a width of M pixels as a data calculation basis for calculating similarity measurement, wherein the value of M is 34 to 40 pixels; S44, using a global search method in the image to be processed to calculate the data block with the highest similarity to the data block T, the calculation formula is as follows: Among them, P i,j represents the image brightness data of M pixels in length and width centered at the coordinate position (i, j) in the image to be processed, where i is in the range of (M / 2) to (WM / 2), j is in the range of (M / 2) to (HM / 2), W is the width of the image to be processed, and H is the height of the image to be processed; S45, counting the maximum values ​​of all R(i, j), where (i, j) is the position point on the image to be processed that matches the feature point in the reference image, and repeating steps S43 to S44 to obtain target position points T1, T2, T3 and T4 that match the feature points P1, P2, P3 and P4 respectively; S46. Obtain the position of the device in the image to be processed according to the positions of the four target position points T1, T2, T3 and T4, and obtain the polygonal outline of the device in the image to be processed in combination with the polygonal outline of the device in the reference image.

2. According to claim 1, a method for detecting surface defects of power equipment based on deep learning is characterized in that: In S3, the real-time image after manual labeling is imported into the deep learning template library for deep learning, specifically: Receive the type and position of the equipment manually marked on the real-time image, and the outline of the equipment obtained by manually copying the real-time image using polygons, and import the real-time image and its corresponding type, position and outline of the equipment into the image recognition template library and the deep learning template library for deep learning.

3. The method for detecting surface defects of power equipment based on deep learning according to claim 1, characterized in that: The step S4 of automatically marking the type, location and outline of the device on the real-time image further includes: The real-time picture and the type and outline of the corresponding device are imported into the image recognition template library.

4. A power equipment surface defect detection terminal based on deep learning, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: S1. Obtain real-time images automatically and regularly collected by the power intelligent patrol system; S2, using the deep learning template library to identify the real-time image, if the identification fails, proceed to step S3, otherwise proceed to step S4; S3, using the image recognition mode to identify the real-time picture, if the identification fails, importing the real-time picture after manual marking into the deep learning template library for deep learning, after updating the deep learning template library, returning to S1 to identify the next real-time picture, otherwise entering step S4; S4, automatically marking the type, location and outline of the device on the real-time picture, and importing the marked real-time picture into the deep learning template library for deep learning, and after updating the deep learning template library, returning to S1 to identify the next real-time picture; In S3, the real-time image is identified by using an image recognition mode, specifically: An image recognition template library is used to perform image recognition on the real-time pictures that cannot be recognized in the deep learning template library to determine the type, position and outline of the equipment in the real-time picture. The image recognition template library is composed of all historical pictures automatically and regularly collected by the power intelligent patrol system after identification, and each of the historical pictures includes the type, position and outline of the corresponding equipment; The S4 automatically marks the type, location and outline of the device on the real-time image, specifically including the following steps: S41, obtaining the type of the device in the real-time picture by using the historical picture corresponding to the real-time picture in the image recognition template library identified in the image recognition mode, and taking the historical picture as a reference image and the real-time picture as an image to be processed; S42, selecting four vertices of the polygonal outline of the device on the reference image as feature points P1, P2, P3, and P4; S43, taking a certain feature point as the center, selecting a brightness data block T with a length and a width of M pixels as a data calculation basis for calculating similarity measurement, wherein the value of M is 34 to 40 pixels; S44, using a global search method in the image to be processed to calculate the data block with the highest similarity to the data block T, the calculation formula is as follows: Among them, P i,j represents the image brightness data of M pixels in length and width centered at the coordinate position (i, j) in the image to be processed, where i is in the range of (M / 2) to (WM / 2), j is in the range of (M / 2) to (HM / 2), W is the width of the image to be processed, and H is the height of the image to be processed; S45, counting the maximum values ​​of all R(i, j), where (i, j) is the position point on the image to be processed that matches the feature point in the reference image, and repeating steps S43 to S44 to obtain target position points T1, T2, T3 and T4 that match the feature points P1, P2, P3 and P4 respectively; S46. Obtain the position of the device in the image to be processed according to the positions of the four target position points T1, T2, T3 and T4, and obtain the polygonal outline of the device in the image to be processed in combination with the polygonal outline of the device in the reference image.

5. The power equipment surface defect detection terminal based on deep learning according to claim 4 is characterized in that: In S3, the real-time image after manual labeling is imported into the deep learning template library for deep learning, specifically: Receive the type and position of the equipment manually marked on the real-time image, and the outline of the equipment obtained by manually copying the real-time image using polygons, and import the real-time image and its corresponding type, position and outline of the equipment into the image recognition template library and the deep learning template library for deep learning.

6. The power equipment surface defect detection terminal based on deep learning according to claim 4 is characterized in that: The step S4 of automatically marking the type, location and outline of the device on the real-time image further includes: The real-time picture and the type and outline of the corresponding device are imported into the image recognition template library.

Images