A method for eliminating duplicate images in equipment defect detection

Abstract

The present invention belongs to the field of image processing technology, and discloses a method for eliminating duplicate images for equipment defect detection, which is used for the preprocessing process of image data obtained by acquiring defective images of power transmission and transformation equipment, and the specific steps are as follows: 1) including data screening of external photos of the same target equipment, and pre-dividing the photo groups by a repetitive algorithm; 2) for each photo group, quickly distinguishing and determining the background area and the target area of ​​each external photo in the photo group according to a trained recognition model; 3) first selecting external photos that meet the background area distinction greater than a set threshold A; 4) then selecting external photos that meet the target area distinction greater than a set threshold B. The present invention sets multiple screening conditions, sequentially excludes multiple non-duplicate images, and then treats the retained images as duplicate images, so the efficiency of eliminating duplicate images is lower than the efficiency of eliminating non-duplicate images.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a method for eliminating duplicate images for equipment defect detection. Background Art

[0002] Defect detection usually refers to the detection of surface defects of objects. Surface defect detection uses advanced machine vision detection technology to detect spots, pits, scratches, color differences, defects, etc. on the surface or inside of the equipment. Defect detection includes detection during manufacturing and maintenance.

[0003] Among them, when performing defect detection on equipment in use, non-destructive testing is often used to detect defects in the equipment in use, so that the equipment can be tested smoothly without affecting its normal operation. Non-destructive testing uses the sound, light, magnetism and electricity characteristics of the material to detect whether there are defects or unevenness in the object being tested, and gives information such as defect size, location, nature and quantity, without damaging or affecting the performance of the object being tested. Compared with destructive testing, non-destructive testing has the following characteristics.

[0004] The first is that it is non-destructive, because it will not damage the performance of the object being tested during testing;

[0005] Second, it is comprehensive. Since the test is non-destructive, 100% comprehensive testing can be carried out on the object to be tested when necessary, which is impossible with destructive testing.

[0006] Third, it is full-process. Destructive testing is generally only applicable to testing of raw materials, such as stretching, compression, and bending commonly used in mechanical engineering.

[0007] Destructive testing is performed on raw materials used in manufacturing. For finished products and in-service products, destructive testing cannot be performed unless they are not intended to continue to be used. Nondestructive testing does not damage the performance of the object being tested. Therefore, it can not only perform full-process testing on raw materials used in manufacturing, various intermediate process links, and finally finished products, but also on equipment in service.

[0008] During nondestructive testing, the method will be selected based on the location of the equipment. If it is a place that is easy for people and equipment to reach, the standard test map can be obtained according to the standard test steps. The post-analysis process is relatively simple, and the existing machine learning methods are used to train the test algorithm to improve the accuracy and efficiency of the test. However, for some locations that are difficult for people and equipment to reach, special test equipment is usually used for testing, especially for power transmission line equipment suspended in the air. Existing means include drones, automatic traveling robots mounted on cables, and ground lifting test equipment. Since this type of equipment is remotely operated, in order to avoid duplication of work, as many photos and videos as possible will be taken after reaching the specified location once, and then the defects will be judged through post-screening. Using this method for defect identification will inevitably increase the difficulty of post-processing. If tens of thousands of images obtained for long transmission lines are processed, the processing difficulty can be imagined. Summary of the invention

[0009] In order to solve the problems existing in the prior art, the present invention provides a method for eliminating duplicate images in equipment defect detection, which aims to screen and eliminate defect photos with several duplicate images, thereby reducing the total amount of processed data and improving processing efficiency.

[0010] The technical solution adopted by the present invention is:

[0011] In a first aspect, the present invention discloses a method for removing duplicate images for equipment defect detection, which is used for preprocessing image data obtained by defect image acquisition of power transmission and transformation equipment, and the specific steps are as follows:

[0012] S1. It includes screening the external photos of the same target device, and pre-dividing a number of photo groups with simple repetitive features by a repetitive algorithm or according to the shooting parameters of the detection device;

[0013] S2. For each photo group, quickly distinguish and determine the background area and target area of ​​each external photo in the photo group based on the trained recognition model

[0014] S3. First, select the external photos that satisfy the background area distinction greater than the set threshold A;

[0015] S4. Then select the external photos that meet the target area distinction greater than the set threshold B, and retain one of the images as a repeated image group or synthesize one image through a multi-frame synthesis algorithm and retain it.

[0016] It is worth noting that the trained recognition model distinguishes the background from the target by combining the entire defect recognition method and using the prepared defect feature recognition model for preprocessing. At the same time, it will be combined with the existing edge detection algorithm and LSD straight line detection algorithm to accurately separate the target from the background. Because the shooting device will record the relevant information of each group of continuous photos in advance through program settings when storing photos, such as the information of the target object (tension clamps or insulators of transmission lines), location information, shooting information (frame, ISO, white balance, resolution, etc.). Then, during post-processing, the corresponding feature recognition module will be determined according to the information of the target object, and the preprocessing algorithm corresponding to each device will be preset to obtain the pixel data of the corresponding background area and target area.

[0017] It should also be noted that the repetition algorithm uses an existing algorithm to pre-process the images. Since the device will shoot at multiple angles when shooting the same target, and does not move continuously at a constant speed, the photos taken are likely to be repeated at a certain angle, and continue shooting after moving a certain distance. A simple repetition algorithm is used to distinguish photos taken at obviously different angles, thereby achieving a simple distinction effect.

[0018] The shooting parameters of the detection device are for some specific shooting devices and shooting methods, such as several fixed angle shooting methods that have been determined in advance, and the shooting device will record the angles at which some continuous photos are taken. The photos taken in this way have been simply classified when they were taken, and they only need to be divided according to the groups marked when they were stored.

[0019] In combination with the first aspect, the present invention provides a first implementation of the first aspect, in step S3, after determining the background area of ​​each external photo, edges of several objects in the background area are detected by an edge detection algorithm;

[0020] When calculating the degree of overlap, only the edges of the confirmed identical objects are checked for overlap.

[0021] In combination with the first implementation of the first aspect, the present invention provides a second implementation of the first aspect, in step S3, the overlap degree is calculated for a set of pixel points at the edges of several objects in the background area;

[0022] The calculation items include the number of pixels, pixel coordinates, and pixel grayscale values.

[0023] In combination with the second implementation of the first aspect, the present invention provides a third implementation of the first aspect, and the overlap degree calculation method in step S3 is as follows:

[0024] (3.1) Determine the edges of all objects in the background area of ​​each external photo according to the edge detection algorithm, and obtain the edge pixel points of each object;

[0025] (3.2) Determine the grayscale values ​​and coordinate values ​​of all edge pixels. First, determine the number of repeated objects in the photo group, and calculate the average edge grayscale value of the repeated objects in the background area of ​​the photo group and the geometric center point of the coordinate range;

[0026] (3.3) Thresholds A1, A2 and A3 are set, and images whose total number of objects in the background area is greater than the number of repeated objects A1, whose difference between the edge grayscale value of the repeated objects and the average edge grayscale value is greater than A2, and whose difference between the geometric center points of the repeated objects is greater than A3 are selected as non-repeating images, and the other images in the photo group are retained for the next step of screening.

[0027] In combination with the third implementation of the first aspect, the present invention provides a fourth implementation of the first aspect, wherein in step (3.3), a threshold value A4 is further provided, and photos having a background area with less than A4 repeated objects are also selected as non-repeating images.

[0028] In combination with the first aspect or the first to fourth embodiments of the first aspect, the present invention provides a fifth embodiment of the first aspect. In step S4, an edge algorithm is also used to obtain the edge of the object in the target area, and the number of repeated objects is calculated. Photos in which the number of repeated objects in the target area in each photo is less than a set threshold B1, and the total number of objects in the target area is greater than the number of repeated objects and exceeds B2 are selected as non-repeating images.

[0029] In combination with the first aspect or the first to fourth embodiments of the first aspect, the present invention provides a sixth embodiment of the first aspect. In step S4, a grayscale image of the target area is obtained for each photo retained in the photo group, and the grayscale images are detected sequentially in the order of shooting using the Pettitt mutation point detection algorithm, and pixel points with grayscale value mutations that appear in the subsequent grayscale image are recorded, and when the proportion of mutation pixels in any photo exceeds the set threshold B3, it is selected as a non-repetitive image.

[0030] In combination with the first aspect or the first to fourth embodiments of the first aspect, the present invention provides a seventh embodiment of the first aspect, wherein the repetition algorithm includes one or more of AHash, DHash, Phash, and WHash.

[0031] In combination with the first aspect or the first to fourth embodiments of the first aspect, the present invention provides an eighth embodiment of the first aspect, further comprising an internal perspective view obtained by an X-ray flaw detection device.

[0032] The beneficial effects of the present invention are:

[0033] (1) The present invention sets multiple screening conditions, sequentially excludes multiple non-repeating images, and then treats the retained images as duplicate images. Since the proportion of duplicate images for a single target object is relatively large, the efficiency of eliminating duplicate images is lower than the efficiency of eliminating non-repeating images.

[0034] (2) The method of the present invention innovatively utilizes the characteristics of outdoor areas or areas with relatively complex background features, and first separates the background from the target through an algorithm for screening, thereby utilizing the characteristics of the background image reflecting the difference in shooting angles, thereby excluding most non-repetitive images and improving detection efficiency;

[0035] (3) The present invention sets multiple thresholds to further eliminate images with non-repetitive features during the background area screening process, and utilizes the characteristic that the target object has similar feature values ​​when photographed at multiple angles, so as to retain images from different shooting angles that may be judged as repeated images in advance, thereby facilitating the subsequent defect feature detection of the target object from several different viewing angles, and avoiding the occurrence of defect detection accuracy affected by the elimination of repeated images due to the presence of large areas of identical features during the screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments.

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0039] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0040] Embodiment 1:

[0041] This embodiment discloses a method for removing duplicate images for device defect detection, such as Figure 1As shown, this method mainly aims at eliminating duplicate photos in the preprocessing stage as much as possible during the process of defect feature recognition through captured equipment pictures, so as to reduce the amount of calculation in the defect recognition process.

[0042] First of all, the method of this embodiment is mainly used in power transmission and transformation equipment, especially those installed at high altitudes or in areas that are difficult for people to reach. The detection equipment includes two types, one is a patrol drone, and the other is a handheld high-altitude self-hanging detection equipment.

[0043] Among them, the so-called high-altitude self-hanging detection equipment is a device that is moved and operated by a rod or a pull rope. It is moved from the ground to the corresponding transmission line through a handheld long rod or other equipment, and is hung on the tension cable of the transmission line through a hook, and then moved by a pull rope or a rod. When it moves to the corresponding device position, it completes a single detection task through its own mobile structure and a preset control program. In a single detection task, it is driven by its own mobile structure to rotate around the device within a certain range, and continuously takes several images at the corresponding node and transmits them back to the ground control device. The storage confidence of the control device is read by an external device, so that the detection image is transmitted for identification.

[0044] In this embodiment, when a single device is performing detection, the optical camera on the detection device is mainly used to take photos and videos, and its light source is the reflected image of natural light source on the corresponding device. The image obtained is a color image, and the background is an outdoor background.

[0045] The elimination method of this embodiment mainly includes the following steps:

[0046] First, the external photos of the same target device are screened, and several photo groups with simple repetitive features are pre-divided through the AHash repetition algorithm or according to the shooting parameters of the detection device; then, for each photo group, the background area and the target area of ​​each external photo in the photo group are quickly distinguished and determined according to the trained recognition model; first, the external photos that meet the background area distinction greater than the set threshold A are selected; then, the external photos that meet the target area distinction greater than the set threshold B are selected, and the retained images are retained as one of the repeated image groups or synthesized into one picture through a multi-frame synthesis algorithm.

[0047] In this embodiment, the existing edge detection algorithm and the LSD line detection algorithm are combined to assist in distinguishing the background and the target for each photo group. The so-called distinction is to further divide all the pixels into groups, namely, the background domain and the target domain.

[0048] When storing photos, the shooting device will record the relevant information of each group of continuous photos in advance through program settings. For example, if the target object in this embodiment is a tension clamp of a transmission line, it will be marked as an external photo of the tension clamp, and also include the location information and shooting information of the photo group. Then, during post-processing, the corresponding feature recognition module will be determined according to the information of the target object, and the pre-processing algorithm corresponding to each device will be preset to obtain the pixel point data of the corresponding background area and target area.

[0049] The repetition algorithm used is to pre-process the pictures using the existing algorithm. When the device takes pictures of the same target, it will shoot at different angles and not move continuously at a constant speed. As a result, the pictures taken are likely to be repeated at a certain angle and continue shooting after moving a certain distance. A simple repetition algorithm is used to distinguish the pictures taken at obviously different angles, thus achieving the effect of simple distinction.

[0050] The shooting parameters of the detection device are for some specific shooting devices and shooting methods, such as several fixed angle shooting methods that have been determined in advance, and the shooting device will record the angles at which some continuous photos are taken. The photos taken in this way have been simply classified when they were taken, and they only need to be divided according to the groups marked when they were stored.

[0051] Furthermore, the external photos of this embodiment are all taken from high altitude. The tensioning wire clamps photographed are close to the iron scaffolding, and there are many characteristic objects near the iron scaffolding, such as trees, protruding rocks, some buildings and a river. During the shooting process, the shooting equipment rotates around the axis of the tensioning wire clamp to shoot. Once the shooting angle changes, the objects in the background area will change significantly. In the picture taken at a fixed focal length, once the objects change, it means that the shooting angle has changed significantly. Once the shooting angle changes, even if the image features of the tensioning wire clamps are similar, possible defects will appear in the image and need to be retained.

[0052] When screening, it is necessary to first identify the objects in the background area, determine whether the shooting angle has changed by the change of the objects, and further determine whether the change exceeds the set threshold. After determining the background area of ​​each external photo, the edge detection algorithm is used to detect the edges of several objects in the background area; when calculating the overlap, only the edges of the confirmed identical objects are checked for overlap.

[0053] Since the difference in pixels in the background area of ​​each photo is directly compared, the amount of data is large. And since different objects will appear in the background area of ​​the two photos when the background area changes, but the interior of the object image will not change significantly when the focal length is fixed, it is only necessary to detect its outline. The edge detection algorithm can quickly extract the outline information of each object, including the proportion of its outline in the entire picture, the edge integrity and the coordinates in the photo.

[0054] Furthermore, the overlap degree is calculated for a set of pixels at the edges of several objects in the background area; the calculation items include the number of pixels, the coordinates of the pixels, and the grayscale values ​​of the pixels.

[0055] According to the edge detection algorithm, the edges of all objects in the background area of ​​each external photo are determined, and the edge pixels of each object are obtained; the grayscale values ​​and coordinate values ​​of all edge pixels are determined, and the number of repeated objects in the photo group is first determined, and the average edge grayscale value of the repeated objects in the background area of ​​the photo group and the geometric center point of the coordinate range are calculated.

[0056] Then, thresholds A1, A2, and A3 are set, and images with a total number of objects in the background area greater than A1 of repeated objects, a difference between the grayscale value of the edge of the repeated object and the average grayscale value of the edge of the repeated object greater than A2, and a difference between the geometric center points of the repeated objects greater than A3 are selected as non-repeating images, and other images in the photo group are retained for the next step of screening. A threshold A4 is also set, and photos with a number of repeated objects in the background area less than A4 are also selected as non-repeating images.

[0057] When calculating, the reference object will be calculated first. Since the entire photo group has been pre-processed to exclude images with large differences, the reference object is the object information with a repetition rate of more than 70% in all images of the photo group. Then this part of the object is regarded as a repeated object, and its corresponding area size and coordinates in each photo are calculated.

[0058] Among them, the coordinates are determined by the geometric center point of the object's outline. The coordinates of several edge pixel points of each object are brought into the calculation model to quickly obtain the coordinate value of the geometric center. Then, the distance value of the geometric center of the corresponding repeated object in a single photo is used to make a judgment based on the set threshold.

[0059] Furthermore, the edge algorithm is also used to obtain the edge of the object in the target area, and the number of repeated objects is calculated. The number of repeated objects in the target area in each photo is less than the set threshold B1, and the total number of objects in the target area is greater than the number of repeated objects and exceeds B2. The grayscale image of the target area is obtained for each photo retained in the photo group, and the grayscale images are detected in sequence by the Pettitt mutation point detection algorithm in the order of shooting, and the pixel points with grayscale value mutations in the next grayscale image are recorded, and any photo is selected as a non-repeating image after the proportion of mutation pixels exceeds the set threshold B3.

[0060] In some embodiments, the device for photographing defects is also provided with an x-ray detection device, which includes an x-ray emitter and a background plate. The images acquired by the device are all black and white perspective images, which are used to feedback the structural changes inside the device.

[0061] Since the background features are the same in these pictures, it is not necessary to separate the target and background during screening. And since the number of pictures taken by the X-ray detection equipment is much smaller than the number of external photos taken, and the shooting angle is constantly changing during shooting, it is only necessary to use the mutation point detection algorithm to capture the mutation pixel points.

Claims

1. A method for eliminating duplicate images for equipment defect detection, used for preprocessing image data obtained by defect image acquisition of power transmission and transformation equipment, characterized in that: The specific steps are as follows: S1. It includes screening the external photos of the same target device, and pre-dividing a number of photo groups with simple repetitive features by a repetitive algorithm or according to the shooting parameters of the detection device; S2. For each photo group, quickly distinguish and determine the background area and target area of ​​each external photo in the photo group based on the trained recognition model S3. First, select the external photos that satisfy the background area distinction greater than the set threshold A; S4. Then select the external photos that meet the target area distinction greater than the set threshold B, and retain one of the images as a repeated image group or synthesize one image through a multi-frame synthesis algorithm to retain; In step S3, after determining the background area of ​​each external photo, edges of several objects in the background area are detected by an edge detection algorithm; When calculating the degree of coincidence, only the edges of the confirmed identical objects are checked for coincidence; In step S3, the overlap degree is calculated for a set of pixel points at the edges of several objects in the background area; The calculation items include the number of pixels, pixel coordinates, and pixel grayscale values; The calculation method of the overlap degree in step S3 is as follows: (3.1) Determine the edges of all objects in the background area of ​​each external photo according to the edge detection algorithm, and obtain the edge pixel points of each object; (3.2) Determine the grayscale values ​​and coordinate values ​​of all edge pixels. First, determine the number of repeated objects in the photo group, and calculate the average edge grayscale value of the repeated objects in the background area of ​​the photo group and the geometric center point of the coordinate range; (3.3) Thresholds A1, A2 and A3 are set, and images whose total number of objects in the background area is greater than the number of repeated objects A1, whose difference between the edge grayscale value of the repeated objects and the average edge grayscale value is greater than A2, and whose difference between the geometric center points of the repeated objects is greater than A3 are selected as non-repeating images, and the other images in the photo group are retained for the next step of screening.

2. The method for eliminating duplicate images for equipment defect detection according to claim 1, characterized in that: In step (3.3), a threshold value A4 is also provided, and photos having a background area with a number of repeated objects less than A4 are also selected as non-repeating images.

3. A method for eliminating duplicate images for equipment defect detection according to claim 1 or 2, characterized in that: In step S4, the edge algorithm is also used to obtain the object edge of the target area, and the number of repeated objects is calculated. The photos in which the number of repeated objects in the target area in each photo is less than the set threshold B1, and the total number of objects in the target area is greater than the number of repeated objects and exceeds B2 are selected as non-repeating images.

4. A method for eliminating duplicate images for equipment defect detection according to any one of claims 1 or 2, characterized in that: In step S4, a grayscale image of the target area is obtained for each photo retained in the photo group, and the grayscale images are detected in sequence by the Pettitt mutation point detection algorithm in the order of shooting, and the pixel points with grayscale value mutations appearing in the subsequent grayscale image are recorded, and when the proportion of mutation pixels in any photo exceeds the set threshold B3, it is selected as a non-repetitive image.

5. A method for eliminating duplicate images for equipment defect detection according to claim 1 or 2, characterized in that: The repetition algorithm includes one or more of AHash, DHash, Phash, and WHash.

6. A method for eliminating duplicate images for equipment defect detection according to claim 1 or 2, characterized in that: Also included are internal perspective views obtained using X-ray flaw detection equipment.

Images