Insulation pull rod defect detection method and device based on image recognition and storage medium

Abstract

The invention relates to an insulation pull rod defect detection method and device based on image recognition and a storage medium, and the method comprises the steps: S1, obtaining all surface images of an insulation pull rod, enabling the surface images to be shot by a camera, and enabling light rays emitted by a line light source parallel to the axis of the insulation pull rod to be reflected by the camera during shooting; the light is collected by the camera after passing through the light equalizing plate and the insulating pull rod in sequence, and the visual axis of the camera is perpendicular to the axis of the insulating pull rod; s2, pre-diagnosing all the surface images of each insulating pull rod in sequence, if the pre-diagnosis of all the surface images of any insulating pull rod is passed, determining that the insulating pull rod has no defect, otherwise, executing a step S3; and S3, inputting each surface image of the insulating pull rod into the defect detection model, and obtaining a detection result through the defect detection model. Compared with the prior art, the method has the advantages of improving the detection rate and the like.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection of insulating pull rods, and in particular to an insulating pull rod defect inspection method, device and storage medium based on image recognition. Background Art

[0002] The insulating rod is one of the core insulating parts in GIS (gas insulated switchgear), and its product quality is of great significance to the high-reliability operation of GIS. Once the insulating rod with latent defects is put into service, it will greatly reduce the reliability of GIS operation and threaten the safe operation of the power grid. At present, there are two main types of insulating rod defect detection: visual inspection and mechanical inspection. Insulating rods are often placed inside to illuminate and directly observe by human eyes due to their good permeability and hollow interior for easy observation. In addition to the visual inspection method, mechanical methods such as X-ray imaging inspection, industrial CT imaging inspection and ultrasonic flaw detection are also commonly used methods for insulating rod defect detection. X-ray imaging inspection uses the principle that normal areas and defective areas and different defects have different absorption effects on radiation. By measuring different radiation intensities on the other side of the object to be tested, it is determined whether there are defects inside the rod. Compared with X-ray imaging inspection, the image detected by industrial CT is 3D. In addition, ultrasonic testing can also be used as a method for detecting defects in insulating pull rods. When ultrasonic waves encounter tiny defects such as cracks and impurities during propagation, they will be reflected at the sample-defect interface, and the location and type of the defect can be determined based on the characteristics of the reflected wave.

[0003] However, the above existing technologies all have the problems of heavy workload, easy misjudgment, or complex operation, and fail to meet the requirements of fast and accurate detection in the production process. The imperfection of existing rod defect detection methods leads to an increased probability of insulating rods entering the grid with defects, posing a threat to the safe and reliable operation of the power grid.

[0004] In this regard, some existing technologies detect surface defects of insulating rods based on machine vision technology. For example, in "Intelligent identification method of insulation pulrods defects based on integrity-aware Mosaic data augmentation and fusion of YOLOv5s", the YOLOv5s algorithm is used to detect defects on insulating rods. Specifically, a data set of insulating rod photos containing white spots, cracks, impurities and bubble defects is compiled to capture defect images.

[0005] However, machine vision technologies including the above-mentioned existing technologies all have the problem of requiring a large amount of calculation, which results in a slow detection speed. Summary of the invention

[0006] The purpose of the present invention is to provide an insulating pull rod defect detection method, device and storage medium based on image recognition.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] An insulating pull rod defect detection method based on image recognition, comprising:

[0009] Step S1: Acquire all surface images of the insulating rod, wherein the surface images are obtained by photographing with a camera, wherein light emitted by a line light source parallel to the axis of the insulating rod passes through a light averaging plate and the insulating rod in sequence and is then collected by the camera, and the visual axis of the camera is perpendicular to the axis of the insulating rod;

[0010] Step S2: pre-diagnose all surface images of each insulating rod in turn. If all surface images of any insulating rod pass the pre-diagnosis, the insulating rod has no defects. Otherwise, execute step S3;

[0011] Step S3: inputting each surface image of the insulating pull rod into a defect detection model, and obtaining a detection result from the defect detection model;

[0012] The detection process of a single surface image of the insulating pull rod in step S2 includes:

[0013] Step S2-1: cropping an image including the pull rod area, and converting the image of the pull rod area from RGB three channels to a single channel for representing light intensity, to obtain a target detection image;

[0014] Step S2-2: Based on the obtained target detection image, identify the axis direction of the pull rod, and establish a coordinate system with the axis direction of the pull rod as the y-axis direction;

[0015] Step S2-3: Based on the target detection image, determine the position of the x-axis based on the average value of pixels in each row, wherein the y-coordinates of pixels in each row are equal, and the visual axis of the camera is in the same plane as the x-axis;

[0016] Step S2-4: For each column in the target detection image, determine whether the target image has defects based on the difference between any pixel point not on the x-axis and another pixel symmetrical to the x-axis.

[0017] The step S2-4 comprises:

[0018] Step S2-4-1: Generate a first sliding window and a second sliding window that are symmetrical with respect to the x-axis;

[0019] Step S2-4-2: The first sliding window and the second sliding window slide synchronously along the y-axis in opposite directions away from each other;

[0020] Step S2-4-3: Calculate the average error based on the pixel values ​​of the pixels in the first sliding window and the second sliding window in each sliding process:

[0021]

[0022] in: is the average error of the jth column and the tth step, W is the length of the first sliding window and the second sliding window, is the absolute maximum value of the y coordinate of the first sliding window in step t, is the absolute minimum value of the y coordinate of the first sliding window in step t, I j,i is the pixel value of the pixel with y-axis coordinate i in the j-th column of the target image;

[0023] Step S2-4-4: If the average error of the pixel values ​​of the pixels in the first sliding window and the second sliding window in each column during any sliding step is less than a preconfigured threshold, the target image has no defects; otherwise, the target image has defects.

[0024] The sliding distance of the first sliding window and the second sliding window is smaller than the length of the first sliding window and the second sliding window.

[0025] In the process of converting the image of the pull rod area from the RGB three-channel to a single channel for representing the light intensity in the step S2-1, the method adopted is grayscale.

[0026] The pixels in the target detection image are:

[0027] I=a1·R+a2·G+a3·B

[0028] Among them, I is the pixel value in the target detection image, R is the value of the R channel in the original three-channel image, G is the value of the G channel in the original three-channel image, B is the value of the B channel in the original three-channel image, and a1, a2, and a3 are the weights of each channel respectively.

[0029] The weight of each channel is determined by the material of the light source and the insulating rod.

[0030] The process of identifying the axis direction of the tie rod in step S2-2 includes:

[0031] The first convolution operator is used to convolve the target image to determine the edge of the pull rod area;

[0032] The axis direction of the tie rod is determined based on the edge of the tie rod area.

[0033] The step S2-3 comprises:

[0034] Based on the obtained target detection image, determine the position of the horizontal center line;

[0035] Based on the obtained horizontal center line, the x-axis detection area is expanded to both sides along the y-axis direction;

[0036] In the x-axis detection area, the pixel values ​​of each row are sorted from small to large, and the average of all pixel points in the order [20%, 80%] is calculated as the average light intensity of the row;

[0037] Select the row with the largest average light intensity as the position of the x-axis.

[0038] An insulating pull rod defect detection device based on image recognition comprises a memory, a processor, and a program stored in the memory, wherein the processor implements the above method when executing the program.

[0039] A storage medium stores a program, which implements the above method when executed.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. By introducing pre-diagnosis based on light intensity and combining it with the role of a line light source parallel to the axis of the insulating rod, defect-free insulating rods can be quickly filtered out, and only insulating rods with problems in pre-diagnosis are tested for defects through the machine model, which can greatly improve the speed of insulating rod defect detection.

[0042] 2. The sliding window method is used to calculate the average error. Compared with the overall calculation error, on the one hand, when the error threshold is set reasonably, it can avoid missed detection due to the average error being too small after the defect area is averaged. On the other hand, it can also avoid false detection caused by insufficient optical detection accuracy when the error threshold is set too small.

[0043] 3. Using convolution to determine the edge can improve the detection accuracy of the axis.

[0044] 4. In the x-axis detection area, the pixel values ​​of each row are sorted from small to high, and the average of all pixels in the order of [20%, 80%] is calculated as the average light intensity of the row, which can avoid the x-axis selection bias caused by the high pixel values ​​of some defective pixels. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic flow chart of the main steps of the method of the present invention;

[0046] Figure 2 Schematic diagram of the sliding window process for calculating the average error;

[0047] Figure 3 It is the principle schematic diagram of the defect detection model;

[0048] Wherein: 101, a first sliding window, 102, a second sliding window. DETAILED DESCRIPTION

[0049] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0050] Example 1

[0051] A defect detection method for insulating rods based on image recognition, such as Figure 1 As shown, including:

[0052] Step S1: Acquire all surface images of the insulating rod, wherein the surface images are obtained by photographing with a camera. During photographing, light emitted by a line light source parallel to the axis of the insulating rod passes through a light emitting plate and the insulating rod in sequence and is then collected by the camera. The visual axis of the camera is perpendicular to the axis of the insulating rod.

[0053] Based on this, in the obtained image, if the insulating rod has no defects, the light intensity of the points symmetrical with respect to the center of the camera's visual axis should be consistent.

[0054] Step S2: pre-diagnose all surface images of each insulating rod in turn. If all surface images of any insulating rod pass the pre-diagnosis, the insulating rod has no defects. Otherwise, execute step S3;

[0055] The detection process of a single surface image of the insulating pull rod in step S2 includes:

[0056] Step S2-1: cropping an image including the pull rod area, and converting the image of the pull rod area from RGB three channels to a single channel for representing light intensity, to obtain a target detection image;

[0057] Among them, in some embodiments, in the process of converting the image of the pull rod area from RGB three channels to a single channel for characterizing light intensity in step S2-1, the method adopted is grayscale. This is mainly for some five-color transparent insulating pull rods and white light source scenes.

[0058] Of course, in other embodiments, in order to eliminate the error caused by the material of the insulating rod and the color temperature of the light source, the pixels in the target detection image are:

[0059] I=a1·R+a2·G+a3·B

[0060] Among them, I is the pixel value in the target detection image, R is the value of the R channel in the original three-channel image, G is the value of the G channel in the original three-channel image, B is the value of the B channel in the original three-channel image, and a1, a2, and a3 are the weights of each channel respectively.

[0061] The weight of each channel is determined by the materials of the light source and the insulating rod. Specifically, a set of weights can be determined based on different combinations of light sources and insulating rod materials, based on experience or experiments.

[0062] Step S2-2: Based on the obtained target detection image, identify the axis direction of the pull rod, and establish a coordinate system with the axis direction of the pull rod as the y-axis direction;

[0063] The process of identifying the axis direction of the tie rod includes:

[0064] The first convolution operator is used to convolve the target image to determine the edge of the pull rod area;

[0065] The axis direction of the tie rod is determined based on the edge of the tie rod area.

[0066] Step S2-3: Based on the target detection image, the position of the x-axis is determined based on the average value of the pixels in each row, wherein the y-coordinates of the pixels in each row are equal, and the visual axis of the camera is in the same plane as the x-axis, including:

[0067] Based on the obtained target detection image, determine the position of the horizontal center line;

[0068] Based on the obtained horizontal center line, the x-axis detection area is expanded to both sides along the y-axis direction;

[0069] In the x-axis detection area, the pixel values ​​of each row are sorted from small to high, and the mean of all pixels in the order of [20%, 80%] is calculated as the average light intensity of the row, which can avoid the x-axis selection bias caused by the high pixel values ​​of some defective pixels.

[0070] Select the row with the largest average light intensity as the position of the x-axis.

[0071] Step S2-4: for each column in the target detection image, based on the difference between any pixel point not on the x-axis and another pixel symmetrical to the x-axis, determine whether the target image has defects, including:

[0072] Step S2-4-1: Generate a first sliding window and a second sliding window that are symmetrical with respect to the x-axis;

[0073] Step S2-4-2: Figure 2 As shown, the first sliding window and the second sliding window slide synchronously along the y-axis in opposite directions away from each other;

[0074] Step S2-4-3: Calculate the average error based on the pixel values ​​of the pixels in the first sliding window and the second sliding window in each sliding process:

[0075]

[0076] in: is the average error of the jth column and the tth step, W is the length of the first sliding window and the second sliding window, is the absolute maximum value of the y coordinate of the first sliding window in step t, is the absolute minimum value of the y coordinate of the first sliding window in step t, I j,i is the pixel value of the pixel with y-axis coordinate i in the j-th column of the target image;

[0077] In some embodiments, the sliding distance between the first sliding window and the second sliding window is less than the length of the first sliding window and the second sliding window. In this way, pixels in the sliding windows may overlap in two adjacent steps, thereby avoiding missed detection due to defects being located at the boundary.

[0078] Step S2-4-4: If the average error of the pixel values ​​of the pixels in the first sliding window and the second sliding window in each column during any sliding step is less than a preconfigured threshold, the target image has no defects; otherwise, the target image has defects.

[0079] Step S3: inputting each surface image of the insulating pull rod into the defect detection model, and obtaining the detection result from the defect detection model.

[0080] For the defect detection model, the existing technology design can be referred to. For example, the main body can be divided into three parts: the backbone network responsible for feature extraction, the neck network responsible for feature fusion, and the prediction network. Figure 3 As shown, the details are as follows:

[0081] (1) Backbone network: responsible for extracting image features and converting the input raw image into a multi-layer feature map. It consists of a Conv module, a C3_X module, and an SPPF (Spatial Pyramid Pooling Fusion) module. The Conv module consists of a convolutional layer, a batch normalization (BN) layer, and an activation function. After the input features pass through the convolutional layer, the network extracts local information, the BN layer normalizes the distribution of the output features, and the activation function enables the network to have nonlinear transformation capabilities. An SPPF module is added at the end of the backbone to apply pooling processes of different sizes to the convolutional feature map, and then the pooling outputs of different sizes are combined to obtain a consistent output size, thereby increasing the receptive field of the feature map.

[0082] (2) Neck feature fusion network: The neck adopts the structure of FPN+PAN. FPN upsamples the feature map output by the last layer of the backbone network in a top-down manner, and fuses the upsampled feature map with the feature map of the same scale in the backbone network to obtain a richer feature representation. Repeating this operation, the dimensions of the fused feature maps are 40×40×512 and 80×80×256 respectively. PAN performs 2-step convolution on the underlying feature map in FPN from bottom to top, and fuses the convolved feature map with the feature map of the same scale in FPN. Repeat this operation until the top layer of the FPN structure is reached, and fused feature maps of size 40×40×256 and 20×20×512 are obtained. FPN propagates high-level semantic information from top to bottom, while PAN propagates position information from bottom to top. The two aggregated features are combined at different detection layers.

[0083] (3)Head outputs three feature maps of different scales, corresponding to large, medium and small object detection. The YOLOv5s network divides the input image of size 640×640×3 into N×N grids and predicts information for all grid cells. The prediction information of each grid cell includes the center coordinates, width and height of the predicted bounding box, as well as the classification probability and confidence of the insulating pull rod defect. The coordinates, length and width of the predicted box represent the precise position and size of the predicted object. The confidence indicates whether the predicted target exists in the grid cell. The higher the confidence value, the higher the possibility that the predicted target exists. The classification probability determines the classification information of the predicted target. First, the confidence of each predicted bounding box is compared with the set threshold. If the confidence exceeds the threshold, it is considered that there is an object in the bounding box, and the position and size of the detected object are obtained. Then, the non-maximum suppression (NMS) algorithm is used to filter the repeated predicted bounding boxes of the same target; finally, the index and category of the target are determined according to the maximum classification probability.

[0084] The execution process of the defect detection model is as follows: First, the input image is processed and the image size is rescaled to the preset size of the network. The size of the insulating pull rod photo in the patent is 640×640 pixels. Then the scaled image is divided into three channels of RGB, and it is normalized to a floating point number between 0 and 1 according to the color information of each pixel in the image. After that, the Mosaic data enhancement method built into the model is selected to expand the data set to enhance the robustness and diversity of the data set. In addition, during the data set preprocessing process, the model will perform adaptive anchor box calculation based on the labeled labels of the data set itself to obtain the best anchor box for the data set. After the above preprocessing, the model starts several rounds of training. In each round of training, the YOLO algorithm uses the original anchor box as a reference, calculates the difference between the predicted anchor box and the actual position of the target, and updates the weight matrix according to the back propagation algorithm. After the training is completed, the model will give the weight matrix of the round corresponding to the optimal result for target detection.

[0085] Embodiment 2:

[0086] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

Claims

1. A method for detecting defects in insulating pull rods based on image recognition, characterized in that: include: Step S1: Acquire all surface images of the insulating rod, wherein the surface images are obtained by photographing with a camera, wherein light emitted by a line light source parallel to the axis of the insulating rod passes through a light averaging plate and the insulating rod in sequence and is then collected by the camera, and the visual axis of the camera is perpendicular to the axis of the insulating rod; Step S2: pre-diagnose all surface images of each insulating rod in turn. If all surface images of any insulating rod pass the pre-diagnosis, the insulating rod has no defects. Otherwise, execute step S3; Step S3: inputting each surface image of the insulating pull rod into a defect detection model, and obtaining a detection result from the defect detection model; The detection process of a single surface image of the insulating pull rod in step S2 includes: Step S2-1: cropping an image including the pull rod area, and converting the image of the pull rod area from RGB three channels to a single channel for representing light intensity, to obtain a target detection image; Step S2-2: Based on the obtained target detection image, identify the axis direction of the pull rod, and establish a coordinate system with the axis direction of the pull rod as the y-axis direction; Step S2-3: Based on the target detection image, determine the position of the x-axis based on the average value of pixels in each row, wherein the y-coordinates of pixels in each row are equal, and the visual axis of the camera is in the same plane as the x-axis; Step S2-4: For each column in the target detection image, determine whether the target image has defects based on the difference between any pixel point not on the x-axis and another pixel symmetrical to the x-axis.

2. The insulating rod defect detection method based on image recognition according to claim 1 is characterized in that: The step S2-4 comprises: Step S2-4-1: Generate a first sliding window and a second sliding window that are symmetrical with respect to the x-axis; Step S2-4-2: The first sliding window and the second sliding window slide synchronously along the y-axis in opposite directions away from each other; Step S2-4-3: Calculate the average error based on the pixel values ​​of the pixels in the first sliding window and the second sliding window in each sliding process: in: is the average error of the jth column and the tth step, W is the length of the first sliding window and the second sliding window, is the absolute maximum value of the y coordinate of the first sliding window in step t, is the absolute minimum value of the y coordinate of the first sliding window in step t, I j,i is the pixel value of the pixel with y-axis coordinate i in the j-th column of the target image; Step S2-4-4: If the average error of the pixel values ​​of the pixels in the first sliding window and the second sliding window in each column during any sliding step is less than a preconfigured threshold, the target image has no defects; otherwise, the target image has defects.

3. The method for detecting defects of insulating pull rods based on image recognition according to claim 2, characterized in that: The sliding distance of the first sliding window and the second sliding window is smaller than the length of the first sliding window and the second sliding window.

4. The method for detecting defects of insulating pull rods based on image recognition according to claim 1, characterized in that: In the process of converting the image of the pull rod area from the RGB three-channel to a single channel for representing the light intensity in the step S2-1, the method adopted is grayscale.

5. The method for detecting defects of insulating pull rods based on image recognition according to claim 1, characterized in that: The pixels in the target detection image are: I=a1·R+a2·G+a3·B Among them, I is the pixel value in the target detection image, R is the value of the R channel in the original three-channel image, G is the value of the G channel in the original three-channel image, B is the value of the B channel in the original three-channel image, and a1, a2, and a3 are the weights of each channel respectively.

6. The method for detecting defects of insulating pull rods based on image recognition according to claim 5, characterized in that: The weight of each channel is determined by the material of the light source and the insulating rod.

7. The method for detecting defects of insulating pull rods based on image recognition according to claim 1, characterized in that: The process of identifying the axis direction of the tie rod in step S2-2 includes: The first convolution operator is used to convolve the target image to determine the edge of the pull rod area; The axis direction of the tie rod is determined based on the edge of the tie rod area.

8. The method for detecting defects of insulating pull rods based on image recognition according to claim 1, characterized in that: The step S2-3 comprises: Based on the obtained target detection image, determine the position of the horizontal center line; Based on the obtained horizontal center line, the x-axis detection area is expanded to both sides along the y-axis direction; In the x-axis detection area, the pixel values ​​of each row are sorted from small to large, and the average of all pixel points in the order [20%, 80%] is calculated as the average light intensity of the row; Select the row with the largest average light intensity as the position of the x-axis.

9. An insulating pull rod defect detection device based on image recognition, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A storage medium having a program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 8 is implemented.

Images