Insulation pull rod defect detection method and device based on image recognition and storage medium

Abstract

The application relates to an image recognition-based insulating pull rod defect detection method and device and a storage medium, wherein the method comprises the following steps: S1, acquiring all surface images of insulating pull rods, wherein the surface images are obtained by a camera, when the surface images are shot, light emitted by a linear light source parallel to the axis of the insulating pull rod is sequentially collected by the camera after passing through a light homogenizing plate and the insulating pull rod, and the visual axis of the camera is perpendicular to the axis of the insulating pull rod; S2, sequentially performing pre-diagnosis on all surface images of each insulating pull rod, if all surface images of any insulating pull rod pass the pre-diagnosis, the insulating pull rod is defect-free, otherwise, step S3 is performed; and S3, inputting each surface image of the insulating pull rod into a defect detection model to obtain a detection result by the defect detection model. Compared with the prior art, the application has the advantages of improving the detection rate and the like.

Description

Technical Field

[0001] This invention relates to the field of visual inspection of insulating tie rods, and in particular to a method, apparatus, and storage medium for detecting defects in insulating tie rods based on image recognition. Background Technology

[0002] Insulating rods are one of the core insulating components in GIS (Gas Insulated Switchgear), and their quality is crucial for the high reliability of GIS operation. Insulating rods with latent defects, once in service, will significantly reduce the reliability of GIS operation and threaten the safe operation of the power grid. Currently, defect detection of insulating rods mainly falls into two categories: visual inspection and mechanical methods. Due to their good transparency and hollow interior, insulating rods are often irradiated with lamps and directly observed by the human eye. Besides visual inspection, mechanical methods such as X-ray imaging, industrial CT imaging, and ultrasonic testing are also commonly used for defect detection. X-ray imaging utilizes the principle that normal areas and defective areas, as well as different defects, absorb X-rays differently. By measuring different X-ray intensities on the other side of the object under test, it can determine whether there are internal defects in the rod. Compared to X-ray imaging, industrial CT imaging provides 3D images. In addition, ultrasonic testing can also be used as a method for detecting defects in insulating tie rods. When ultrasonic waves encounter tiny defects such as cracks or impurities during propagation, they will be reflected at the sample-defect interface. The location and type of defect can be determined based on the characteristics of the reflected wave.

[0003] However, the existing technologies mentioned above are all labor-intensive, prone to misjudgment, or complex to operate, failing to meet the needs for rapid and accurate detection in the production process. The imperfections in existing tie rod defect detection methods increase the probability of defective insulated tie rods being introduced into the grid, posing a threat to the safe and reliable operation of the power grid.

[0004] In response, some existing technologies use machine vision to detect surface defects on insulating pulrods. For example, the paper "Intelligent identification method of insulation pulrods defects based on intentness-aware Mosaicdata augmentation and fusion of YOLOv5s" uses the YOLOv5s algorithm to detect defects on insulating pulrods. Specifically, it captures defect images by compiling a dataset containing photographs of insulating pulrods with defects such as white spots, cracks, impurities, and bubbles.

[0005] However, machine vision technologies, including the aforementioned existing technologies, all suffer from the problem of requiring a large amount of computation, which results in a relatively slow detection speed. Summary of the Invention

[0006] The purpose of this invention is to provide a method, device, and storage medium for detecting defects in insulating tie rods based on image recognition.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A method for detecting defects in insulating tie rods based on image recognition, comprising:

[0009] Step S1: Acquire all surface images of the insulating tie rod, wherein the surface images are captured by a camera. During the capture, light emitted by a line light source parallel to the axis of the insulating tie rod passes through a light diffuser and the insulating tie rod in sequence before being captured by the camera. The line of sight of the camera is perpendicular to the axis of the insulating tie rod.

[0010] Step S2: Perform a pre-diagnosis on all surface images of each insulating tie rod in sequence. If the pre-diagnosis of all surface images of any insulating tie rod passes, then the insulating tie rod is defect-free; otherwise, proceed to step S3.

[0011] Step S3: Input the images of each surface of the insulating tie rod into the defect detection model, and obtain the detection results from the defect detection model;

[0012] The detection process of a single surface image of the insulating tie rod in step S2 includes:

[0013] Step S2-1: Crop the image to obtain the image containing the lever region, and convert the image of the lever region from RGB three channels to a single channel for representing light intensity to obtain the target detection image;

[0014] Step S2-2: Based on the obtained target detection image, identify the direction of the pull rod axis and establish a coordinate system with the pull rod axis direction as the y-axis direction;

[0015] Step S2-3: Based on the obtained target detection image, determine the position of the x-axis based on the average value of the pixels in each row, wherein the y-coordinates of the pixels in each row are equal, and the camera's line of sight and the x-axis are in the same plane;

[0016] Step S2-4: For each column in the target detection image, determine whether there is a defect in the target image based on the difference between any non-x-axis pixel and its corresponding x-axis symmetrical pixel.

[0017] Step S2-4 includes:

[0018] Step S2-4-1: Generate a first sliding window and a second sliding window that are symmetrical about the x-axis;

[0019] Step S2-4-2: The first and second sliding windows slide synchronously in opposite directions along the y-axis, moving away from each other;

[0020] Step S2-4-3: Calculate the average error based on the pixel values ​​of the pixels within the first and second sliding windows during each sliding process:

[0021]

[0022] in: Let W be the average error of the j-th column at step t, and W be the length of the first and second sliding windows. Let be the maximum absolute value of the y-coordinate of the first sliding window at step t. I is the minimum absolute value of the y-coordinate of the first sliding window in step t. j,i Let i be the pixel value of the pixel with y-coordinate i in the j-th column of the target image;

[0023] Step S2-4-4: If the average error of pixel value calculation for each column in the first and second sliding windows is less than the pre-configured threshold during any sliding process, then the target image is defect-free; otherwise, the target image is defective.

[0024] The distance that the first and second sliding windows slide is less than the length of the first and second sliding windows.

[0025] In step S2-1, the process of converting the image of the lever area from RGB three channels to a single channel for representing light intensity is performed by grayscale conversion.

[0026] The pixels in the target detection image are:

[0027] I = a1·R + a2·G + a3·B

[0028] Where I is the pixel value in the target detection image, R is the value of the R channel in the original three-channel image, G is the value of the G channel in the original three-channel image, B is the value of the B channel in the original three-channel image, and a1, a2, and a3 are the weights of each channel.

[0029] The weight of each channel is determined by the material of the light source and the insulating tie rod.

[0030] The process of identifying the direction of the tie rod axis in step S2-2 includes:

[0031] The first convolution operator is used to convolve the target image to determine the edge of the lever region;

[0032] The axial direction of the tie rod is determined based on the edge of the tie rod region.

[0033] Step S2-3 includes:

[0034] Based on the obtained target detection image, determine the position of the horizontal midline;

[0035] Based on the obtained horizontal midline, the x-axis detection area is obtained by expanding to both sides along the y-axis direction;

[0036] In the x-axis detection area, the pixel values ​​of each row are sorted from smallest to largest, and the average value of all pixels in the order [20%, 80%] is calculated as the average light intensity of that row;

[0037] Choose the row with the highest average light intensity as the position of the x-axis.

[0038] An image recognition-based defect detection device for insulating tie rods includes a memory, a processor, and a program stored in the memory. When the processor executes the program, it implements the method described above.

[0039] A storage medium having a program stored thereon, which, when executed, implements the method described above.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. By introducing light intensity-based pre-diagnosis and combining it with the effect of a line light source parallel to the axis of the insulating tie rod, defect-free insulating tie rods can be quickly filtered out. Only insulating tie rods with problems in the pre-diagnosis are subjected to defect detection through the machine model, which can greatly improve the speed of defect detection of insulating tie rods.

[0042] 2. Using a sliding window method to calculate the average error has two advantages. First, it can avoid missed detections caused by the average error being too small due to the small area of ​​the defect being averaged, provided that the error threshold is set reasonably. Second, it can also avoid false detections caused by insufficient optical detection accuracy due to the error threshold being set too small.

[0043] 3. Using convolution to determine the edges can improve the detection accuracy of the axis.

[0044] 4. In the x-axis detection area, sort the pixel values ​​of each row from smallest to largest, and calculate the average value of all pixels in the order [20%, 80%] as the average light intensity of that row. This can avoid x-axis selection bias caused by some defective pixels raising the pixel value. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the main steps of the method of the present invention;

[0046] Figure 2 A schematic diagram of the sliding window process for calculating the average error;

[0047] Figure 3 This is a schematic diagram illustrating the principle of the defect detection model;

[0048] Among them: 101, the first sliding window; 102, the second sliding window. Detailed Implementation

[0049] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0050] Example 1

[0051] A method for detecting defects in insulating tie rods based on image recognition, such as Figure 1 As shown, it includes:

[0052] Step S1: Acquire all surface images of the insulating tie rod, wherein the surface images are captured by a camera. During the capture, light emitted by a line light source parallel to the axis of the insulating tie rod passes through the light diffuser and the insulating tie rod in sequence before being captured by the camera. The camera's line of sight is perpendicular to the axis of the insulating tie rod.

[0053] Based on this, in the obtained image, if the insulating rod is defect-free, the light intensity of points symmetrical about the camera's line of sight should be consistent.

[0054] Step S2: Perform a pre-diagnosis on all surface images of each insulating tie rod in sequence. If the pre-diagnosis of all surface images of any insulating tie rod passes, then the insulating tie rod is defect-free; otherwise, proceed to step S3.

[0055] The detection process of a single surface image of the insulating tie rod in step S2 includes:

[0056] Step S2-1: Crop the image to obtain the image containing the lever region, and convert the image of the lever region from RGB three channels to a single channel for representing light intensity to obtain the target detection image;

[0057] In some embodiments, the process of converting the image of the lever area from RGB three channels to a single channel for characterizing light intensity in step S2-1 is done by grayscale conversion. This is mainly for some five-color transparent insulating levers and white light source scenarios.

[0058] Of course, in other embodiments, in order to eliminate errors caused by the material of the insulating rod and the color temperature of the light source, the pixels in the target detection image are:

[0059] I = a1·R + a2·G + a3·B

[0060] Where I is the pixel value in the target detection image, R is the value of the R channel in the original three-channel image, G is the value of the G channel in the original three-channel image, B is the value of the B channel in the original three-channel image, and a1, a2, and a3 are the weights of each channel.

[0061] The weight of each channel is determined by the material of the light source and the insulating rod. Specifically, a set of weights can be determined based on experience or experiments, depending on different combinations of light source and insulating rod materials.

[0062] Step S2-2: Based on the obtained target detection image, identify the direction of the pull rod axis and establish a coordinate system with the pull rod axis direction as the y-axis direction;

[0063] The process of identifying the direction of the tie rod axis includes:

[0064] The first convolution operator is used to convolve the target image to determine the edge of the lever region;

[0065] The axial direction of the tie rod is determined based on the edge of the tie rod region.

[0066] Step S2-3: Based on the obtained target detection image, determine the x-axis position based on the average pixel value of each row, where the y-coordinates of the pixels in each row are equal, and the camera's line of sight and the x-axis are in the same plane, including:

[0067] Based on the obtained target detection image, determine the position of the horizontal midline;

[0068] Based on the obtained horizontal midline, the x-axis detection area is obtained by expanding to both sides along the y-axis direction;

[0069] In the x-axis detection area, the pixel values ​​of each row are sorted from smallest to largest, and the average value of all pixels in the order [20%, 80%] is calculated as the average light intensity of that row. This can avoid x-axis selection bias caused by some defective pixels raising the pixel value.

[0070] Choose the row with the highest average light intensity as the position of the x-axis.

[0071] Step S2-4: For each column in the target detection image, based on the difference between any non-x-axis pixel and its corresponding x-axis symmetrical pixel, determine whether the target image has defects, including:

[0072] Step S2-4-1: Generate a first sliding window and a second sliding window that are symmetrical about the x-axis;

[0073] Step S2-4-2: As Figure 2 As shown, the first and second sliding windows slide synchronously in opposite directions along the y-axis, moving away from each other.

[0074] Step S2-4-3: Calculate the average error based on the pixel values ​​of the pixels within the first and second sliding windows during each sliding process:

[0075]

[0076] in: Let W be the average error of the j-th column at step t, and W be the length of the first and second sliding windows. Let be the maximum absolute value of the y-coordinate of the first sliding window at step t. I is the minimum absolute value of the y-coordinate of the first sliding window in step t. j,i Let i be the pixel value of the pixel with y-coordinate i in the j-th column of the target image;

[0077] In some embodiments, the distance that the first sliding window and the second sliding window slide are less than the length of the first sliding window and the second sliding window, so that the pixels in the sliding window overlap in two adjacent steps, thereby avoiding missed detections caused by defects being located at the boundary.

[0078] Step S2-4-4: If the average error of pixel value calculation for each column in the first and second sliding windows is less than the pre-configured threshold during any sliding process, then the target image is defect-free; otherwise, the target image is defective.

[0079] Step S3: Input the images of each surface of the insulating tie rod into the defect detection model, and obtain the detection results from the defect detection model.

[0080] For defect detection models, existing technologies can be referenced in the design. For example, the main body can be divided into three parts: a backbone network responsible for feature extraction, a neck network responsible for feature fusion, and a prediction network, such as... Figure 3 As shown, the details are as follows:

[0081] (1) Backbone Network: Responsible for extracting image features and transforming the input raw image into a multi-layer feature map. It consists of the Conv module, the C3_X module, and the SPPF (Spatial Pyramid Pooling Fusion) module. The Conv module consists of convolutional layers, batch normalization (BN) layers, and activation functions. After the input features pass through the convolutional layers, the network extracts local information. The BN layers normalize the distribution of the output features, and the activation function enables the network to have non-linear transformation capabilities. An SPPF module is added at the end of the backbone to apply pooling processes of different sizes to the convolutional feature maps. Then, the pooling outputs of different sizes are combined to obtain a consistent output size, thereby increasing the receptive field of the feature maps.

[0082] (2) Neck Feature Fusion Network: The neck region adopts an FPN+PAN structure. The FPN upsamples the feature map output from the last layer of the backbone network in a top-down manner and fuses it with the feature map of the same scale in the backbone network to obtain a richer feature representation. This operation is repeated, resulting in fused feature maps with dimensions of 40×40×512 and 80×80×256, respectively. The PAN performs two-step convolution on the lower-level feature maps in the FPN from the bottom up and fuses the convolutional feature maps with the feature maps of the same scale in the FPN. This operation is repeated until the top layer of the FPN structure is reached, resulting in fused feature maps of sizes 40×40×256 and 20×20×512. The FPN propagates high-level semantic information from top to bottom, while the PAN propagates positional information from bottom to top. The two aggregated features are combined in different detection layers.

[0083] (3) The Head outputs feature maps of three different scales, corresponding to the detection of large, medium, and small objects respectively. The YOLOv5s network divides the input image of size 640×640×3 into an N×N grid and predicts information for all grid cells. The prediction information for each grid cell includes the center coordinates, width, and height of the predicted bounding box, as well as the classification probability and confidence of the insulation rod defect. The coordinates, length, and width of the predicted box represent the precise location and size of the predicted object. The confidence indicates whether the predicted target exists in the grid cell. The higher the confidence value, the higher the probability that the predicted target exists. The classification probability determines the classification information of the predicted target. First, the confidence of each predicted bounding box is compared with a set threshold. If the confidence exceeds the threshold, it is considered that the target exists in the bounding box, and the location and size of the detected target are obtained. Then, the non-maximum suppression (NMS) algorithm is used to filter the repeated predicted bounding boxes of the same target; finally, the target's index and category are determined according to the maximum classification probability.

[0084] The defect detection model operates as follows: First, the input image is processed by resizing it to the network's preset size; in the patent, the insulated pull rod photo is 640×640 pixels. Then, the scaled image is divided into RGB channels, and based on the color information of each pixel, it is normalized to a floating-point number between 0 and 1. Next, the built-in Mosaic data augmentation method is used to expand the dataset, enhancing its robustness and diversity. Furthermore, during dataset preprocessing, the model performs adaptive anchor box calculations based on the pre-labeled data to obtain the optimal anchor boxes for that dataset. After preprocessing, the model undergoes several rounds of training. In each round, the YOLO algorithm uses the original anchor boxes as a reference, calculates the difference between the predicted anchor box and the actual target position, and updates the weight matrix using backpropagation. After training, the model provides the weight matrix corresponding to the optimal result for that round, used for target detection.

[0085] Example 2:

[0086] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An image recognition-based insulating pull rod defect detection method, characterized by, include: Step S1: Acquire all surface images of the insulating tie rod, wherein the surface images are captured by a camera. During the capture, light emitted by a line light source parallel to the axis of the insulating tie rod passes through a light diffuser and the insulating tie rod in sequence before being captured by the camera. The line of sight of the camera is perpendicular to the axis of the insulating tie rod. Step S2: Perform a pre-diagnosis on all surface images of each insulating tie rod in sequence. If the pre-diagnosis of all surface images of any insulating tie rod passes, then the insulating tie rod is defect-free; otherwise, proceed to step S3. Step S3: Input the images of each surface of the insulating tie rod into the defect detection model, and obtain the detection results from the defect detection model; The detection process of a single surface image of the insulating tie rod in step S2 includes: Step S2-1: Crop the image to obtain the image containing the lever region, and convert the image of the lever region from RGB three channels to a single channel for characterizing light intensity to obtain the target detection image; Step S2-2: Based on the obtained target detection image, identify the direction of the pull rod axis and establish a coordinate system with the pull rod axis direction as the y-axis direction; Step S2-3: Based on the obtained target detection image, determine the position of the x-axis based on the average value of the pixels in each row, wherein the y-coordinates of the pixels in each row are equal, and the camera's line of sight and the x-axis are in the same plane; Step S2-4: For each column in the target detection image, based on the difference between any non-x-axis pixel and its corresponding x-axis symmetrical pixel, determine whether the target image has a defect; Step S2-4 includes: Step S2-4-1: generating a first sliding window and a second sliding window that are symmetrical with respect to x axis-symmetrical first and second sliding windows; Step S2-4-2: The first and second sliding windows slide synchronously in opposite directions along the y-axis, moving away from each other; Step S2-4-3: Calculate the average error based on the pixel values ​​of the pixels within the first and second sliding windows during each sliding process: , in: For the first j The average error at step t, The lengths of the first and second sliding windows, For the first t Step 1 sliding window y The maximum absolute value of the coordinates. For the first t Step 1 sliding window y The minimum absolute value of the coordinates. For the target image, the first j List y The axis coordinates are i The pixel value of the pixel; Step S2-4-4: If the average error of pixel value calculation of pixels in the first and second sliding windows in each column is less than the pre-configured threshold during any sliding process, then the target image is defect-free; otherwise, the target image is defective. Step S2-3 includes: Based on the obtained target detection image, determine the position of the horizontal midline; Based on the obtained horizontal midline, the x-axis detection area is obtained by expanding to both sides along the y-axis direction; In the x-axis detection area, the pixel values ​​of each row are sorted from smallest to largest, and the average value of all pixels in the order [20%, 80%] is calculated as the average light intensity of that row; Choose the row with the highest average light intensity as the position of the x-axis.

2. The image recognition-based insulating pull rod defect detection method according to claim 1, characterized in that, The distance that the first and second sliding windows slide is less than the length of the first and second sliding windows.

3. The image recognition-based insulating pull rod defect detection method according to claim 1, characterized in that, In step S2-1, the process of converting the image of the lever area from RGB three channels to a single channel for representing light intensity is performed by grayscale conversion.

4. The image recognition-based insulating pull rod defect detection method according to claim 1, characterized in that, The pixels in the target detection image are: I = a1·R + a2·G + a3·B Where I is the pixel value in the target detection image, R is the value of the R channel in the original three-channel image, G is the value of the G channel in the original three-channel image, B is the value of the B channel in the original three-channel image, and a1, a2, and a3 are the weights of each channel.

5. The image recognition-based insulating pull rod defect detection method according to claim 4, characterized in that, The weight of each channel is determined by the material of the light source and the insulating tie rod.

6. The image recognition-based insulating pull rod defect detection method according to claim 1, wherein The process of identifying the direction of the tie rod axis in step S2-2 includes: The first convolution operator is used to convolve the target image to determine the edge of the lever region; The axial direction of the tie rod is determined based on the edge of the tie rod region.

7. An image recognition-based insulating pull rod defect detection device comprising a memory, a processor, and a program stored in the memory, wherein When the processor executes the program, it implements the method as described in any one of claims 1-6.

8. A storage medium having stored thereon a program, characterized by When the program is executed, it implements the method as described in any one of claims 1-6.

Images