Power tool detection method based on machine vision

Abstract

The present invention relates to the field of power tool detection, and more specifically, to a power tool detection method based on machine vision. The method comprises: continuously acquiring multiple frames of surface grayscale images of power lines, performing mixed Gaussian statistics on the surface grayscale images to obtain a pixel value distribution curve, calculating a clarity index according to the pixel value distribution curve, and retaining surface grayscale images with a clarity greater than a preset clarity threshold; calculating the overlap coefficient between a target image and a reference image, traversing to obtain the overlap coefficient between the target image and the reference image at different positions, and in response to the maximum overlap coefficient, obtaining the relative position of the target image and the reference image, traversing to obtain the relative position of any two adjacent frames of surface grayscale images; splicing the surface grayscale images according to the relative position to obtain a power line map, inputting the power line map into a preset neural network, outputting a defect probability value, and completing the detection. Through the technical solution of the present invention, the accuracy and efficiency of the power tool detection results can be improved.

Description

Technical Field

[0001] The present invention relates to the field of power tool detection, and more specifically, to a power tool detection method based on machine vision. Background Art

[0002] The connection of overhead transmission line conductors and ground wires generally uses crimping-type power hardware - tension clamps and straight joint pipes, which must bear all the tension of the conductors and ground wires, and are also electrical conductors. They will not be disassembled after installation. During the process of materials, processes, production, construction, and acceptance, if the conductors and ground wires are not firmly and precisely fitted in the crimping pipe or are not fully penetrated, or the steel core crimping does not meet the requirements, it will bury a major hidden danger for the safe operation of the line.

[0003] The X-ray detection technology, which has been applied to the fields of human body, security inspection, aerospace, precision manufacturing, etc., is introduced into the professional field of power transmission lines, realizing real-time digital imaging and intuitive perspective detection without destructive dissection of the tension clamp, filling the gap in digital imaging perspective detection technology for the crimping quality of tension clamps and joint pipes for power transmission lines at home and abroad. The X-ray detection method can intuitively display the crimping state of the wire in the aluminum tube, making it easy to make an intuitive judgment, and the negative film can be used as the original record of the detection for multiple research and long-term preservation.

[0004] The existing Chinese patent application document with publication number CN118172242A discloses a method, medium and system for stitching drone water surface images, which stitches each image in the pre-processed water surface image group to obtain a stitched image and output it. However, when using a drone or a drone equipped with an X-ray flaw detector to shoot, it will be affected by wind or the operator, and the captured image may have spots or missing parts, resulting in inaccurate detection results. Summary of the invention

[0005] In order to solve the problem of inaccurate detection results, the present invention proposes a power tool detection method based on machine vision.

[0006] The present invention discloses a power tool detection method based on machine vision, comprising: continuously acquiring surface grayscale images of multiple frames of power transmission lines, performing mixed Gaussian statistics on the surface grayscale images to obtain pixel value distribution curves, one surface grayscale image corresponds to one pixel value distribution curve, calculating a clarity index according to the pixel value distribution curve, and retaining surface grayscale images with a clarity greater than a preset clarity threshold; taking any retained surface grayscale image as a target image, taking a surface grayscale image of a historical frame adjacent to the target image as a reference image, calculating an overlap coefficient between the target image and the reference image, traversing to obtain the overlap coefficient between the target image and the reference image at different positions, and in response to the maximum overlap coefficient, obtaining the relative position of the target image and the reference image, traversing to obtain the relative position of any two adjacent frames of surface grayscale images; splicing the surface grayscale images according to the relative positions to obtain a power transmission line map, inputting the power transmission line map into a preset neural network, outputting a defect probability value, and completing the detection.

[0007] It not only improves the image quality and stitching accuracy, but also minimizes image missing and errors by combining historical frame information and clarity assessment, thereby enhancing the reliability and accuracy of the transmission line inspection process and improving the accuracy and efficiency of defect detection.

[0008] Preferably, the clarity index satisfies the relationship: for the same pixel value distribution curve, the distribution probability of any maximum value point within the three times standard deviation interval is calculated to construct a distribution probability sequence; the number of pixels within the three times standard deviation interval corresponding to any maximum value point in the pixel value distribution curve is taken as the normal number, and the first maximum value and the second maximum value are respectively obtained, the first maximum value is the maximum value of the normal number of the pixel value distribution curve, and the second maximum value is the maximum value of the normal number of continuous multiple frames of surface grayscale images;

[0009] , Indicates the clarity index, represents the first maximum value, represents the second maximum value, represents the standard deviation of the distribution probability sequence, represents the hyperbolic tangent function.

[0010] By calculating the distribution probability of the maximum points and the number of normal pixels in different frames, the clarity index of each frame is further determined. This method can directly associate the pixel value distribution characteristics of the image with the blurriness of the image, use the standard deviation to measure the fluctuation of the normal pixel distribution in the image, and adjust the clarity index through the hyperbolic tangent function.

[0011] Preferably, the clarity index also satisfies the relationship:

[0012] , Indicates the clarity index, Indicates the pixel value distribution curve The gray value corresponding to the maximum point is Indicates the pixel value distribution curve The gray value corresponding to the maximum point is represents the total number of maximum points, represents the hyperbolic tangent function.

[0013] The maximum points represent the key features of different gray levels in the image, and the grayscale differences between them reflect the fluctuations in image clarity.

[0014] Preferably, the overlap coefficient includes: taking the pixel points of the overlapping part of the target image and the reference image as the overlapping pixel points; the overlap coefficient satisfies the relationship:

[0015] , represents the overlap coefficient, Indicates the number of overlapping pixels. Represents the total number of pixels in the target image. Indicates overlapping pixels The grayscale value in the reference image, Indicates overlapping pixels Grayscale value in the target image.

[0016] The overlapping pixel areas between the target image and the reference image are identified, and the degree of match between the two images is quantified by comparing the grayscale value differences of these overlapping pixels. This matching evaluation can deeply analyze the differences between the two images in terms of imaging quality, perspective consistency, spatial alignment, etc., thereby achieving more accurate image alignment.

[0017] Preferably, the neural network is a convolutional neural network, which extracts features from the transmission line diagram to obtain image features, and classifies the image features to obtain defect probability values.

[0018] Preferably, the training process of the preset neural network includes: using the transmission line diagram in the history as input information, and the true value of the defect probability value of the transmission line diagram as a label to obtain a set of training data, inputting the training data into the preset neural network to obtain an output result; calculating the loss value of the preset neural network through the output result and the label, the loss function uses the cross entropy loss function, back-propagates the error signal according to the loss value, updates the network parameters of the preset neural network, and makes the loss value smaller; iteratively updates the network parameters of the preset neural network, and when the preset neural network reaches the set maximum number of training times or the loss value is less than the set loss value, stops updating to obtain a trained preset neural network.

[0019] Preferably, it also includes: taking the pixel points of the overlapping part of the target image and the reference image as overlapping pixel points; calculating the ratio of the overlapping pixel points to the total number of pixel points in the target image, traversing to obtain the ratio of each surface grayscale image in a preset sampling period, calculating the mean of all ratios, taking the difference between 1 and the mean as the update ratio, and taking the inverse of the update ratio as the maximum allowable retake time; in response to the overlapping coefficients of the surface grayscale images within the maximum allowable retake time being less than the overlapping threshold, the drone returns to the starting position of the maximum allowable retake time at the same speed for retake.

[0020] By analyzing the ratio of the overlap coefficient of each surface grayscale image to the total number of pixels in the target image, possible deviations in the image acquisition process, such as insufficient image overlap or image blur, can be monitored in real time. Based on the mean of these ratios, the update ratio can be calculated, and the maximum allowable retake time can be derived from it to ensure that the drone can re-collect images that cover the target area within this time period.

[0021] Preferably, the drone returns to the starting position of the maximum allowed reshoot time at the same speed for reshooting, which also includes: when an overlapping coefficient of a surface grayscale image corresponding to a certain frame within the maximum allowed reshoot time is less than an overlapping threshold, the drone returns to the position where the overlapping coefficient is less than the overlapping threshold at the same speed for reshooting.

[0022] Beneficial effects of the present invention:

[0023] The present invention continuously obtains grayscale images of the transmission line surface, combines pixel value distribution and clarity index to screen out high-quality images, optimizes the positioning of the target image through the overlap coefficient between historical frames, and finally performs defect detection on the spliced ​​transmission line map through a convolutional neural network. It not only ensures the clarity and quality of the image and avoids misjudgment caused by blurred or unclear images, but also ensures the integrity of the image and the reliability of detection through a dynamic reshoot strategy. Through an adaptive update mechanism, the drone can automatically return to the appropriate position within the reshoot time to perform accurate image acquisition, minimize human intervention, improve detection efficiency, and be able to update the training model in real time to adapt to the detection needs of different power tools and transmission lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0025] Figure 1 It is a flow chart of a power tool detection method based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0027] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0028] The present invention provides a method for detecting electric tools based on machine vision. Figure 1 As shown, the power tool detection method based on machine vision includes steps S1 to S3, which are described in detail below.

[0029] S1, continuously obtain multiple frames of surface grayscale images of the power transmission line, perform mixed Gaussian statistics on the surface grayscale images to obtain a pixel value distribution curve, calculate a clarity index based on the pixel value distribution curve, and retain surface grayscale images with a clarity greater than a preset clarity threshold.

[0030] It should be noted that transmission lines are a type of power tool, mainly used to transmit the electricity generated by power plants to distant users or substations through high voltage. It is made of conductive materials and laid in elevated or underground forms to ensure that the loss of electricity is minimized during long-distance transmission. Transmission lines are not only an important part of the power system, but also play a key role in ensuring a stable supply of electricity and supporting various types of industrial, household and commercial electricity use.

[0031] It should be noted that using drones equipped with X-ray flaw detectors to inspect transmission lines is an efficient and non-destructive detection method. Through the flexibility of drones and the high penetration of X-ray flaw detectors, potential defects on the surface and inside of transmission lines can be accurately detected.

[0032] During the execution process, the drone will continuously acquire multiple frames of X-ray images, which can reflect the status of different angles and positions of the transmission line. Subsequently, through image processing technology, these X-ray images are converted into surface grayscale images. Different grayscale values ​​in the grayscale image represent the density changes of different materials or structures, which can clearly show abnormal conditions such as surface cracks, corrosion, and wear.

[0033] Furthermore, with the help of image analysis algorithms, such as edge detection and pattern recognition, more accurate structural information can be extracted from the surface grayscale image.

[0034] In one embodiment, multiple frames of surface grayscale images of power lines are continuously acquired and mixed Gaussian statistics are performed on the surface grayscale images, which can effectively model the probability distribution characteristics of pixel values ​​in the image. After each surface grayscale image is fitted by the Gaussian mixture model, a corresponding pixel value distribution curve is generated, which reflects the distribution of each grayscale level in the image, wherein one surface grayscale image corresponds to one pixel value distribution curve.

[0035] It should be noted that the purpose of Gaussian mixture is to identify and quantify the change pattern of pixel values ​​in an image. In particular, during the detection process, the distribution characteristics of pixel values ​​can reveal the change in the blur level of the image. When the pixel value distribution of a frame of image changes significantly compared with the previous frame, it usually means that the image quality of the frame is different, which may be caused by factors such as deviation in the image shooting angle, motion blur, and changes in the surface of the object.

[0036] Calculating the clarity index includes: for the same pixel value distribution curve, calculating the distribution probability of any maximum value point within the three-times standard deviation interval, and constructing a distribution probability sequence. Taking the number of pixels within the three-times standard deviation interval corresponding to any maximum value point in the pixel value distribution curve as the normal number, obtaining the first maximum value and the second maximum value respectively, the first maximum value is the maximum value of the normal number of the pixel value distribution curve, and the second maximum value is the maximum value of the normal number of continuous multiple-frame surface grayscale images.

[0037] , Indicates the clarity index, represents the first maximum value, represents the second maximum value, represents the standard deviation of the distribution probability sequence, represents the hyperbolic tangent function.

[0038] By establishing a statistical model of the pixel value distribution curve and combining it with the distribution probability within the three-times standard deviation interval, the clarity and quality changes of the image can be accurately quantified. By calculating the distribution probability of the maximum value points and the number of normal pixels in different frames of images, the clarity index of each frame of the image can be further determined. This method can directly associate the pixel value distribution characteristics of the image with the degree of blur of the image, use the standard deviation to measure the degree of fluctuation of the normal pixel distribution in the image, and adjust the clarity index through the hyperbolic tangent function. The clarity index reflects the change in image quality. When the clarity of the image decreases or blurs, the clarity index will decrease accordingly, indicating that the image is abnormal.

[0039] This calculation method can monitor changes in image quality in real time in multiple frames of images, help the detection system accurately determine the status of the transmission line, and promptly discover image quality degradation caused by changes in shooting conditions, equipment problems or external interference, thereby providing more reliable data support for subsequent image processing, anomaly detection and equipment maintenance.

[0040] Exemplarily, the preset clarity threshold is set to 0.7, which can be specifically set by those skilled in the art. The preset clarity threshold is for preliminarily screening the captured surface grayscale image to remove petals or blurred images.

[0041] In one embodiment, the clarity index also satisfies the relationship:

[0042] , Indicates the clarity index, Indicates the pixel value distribution curve The gray value corresponding to the maximum point is Indicates the pixel value distribution curve The gray value corresponding to the maximum point is represents the total number of maximum points, represents the hyperbolic tangent function.

[0043] The maximum points represent the key features of different grayscale levels in the image, and the grayscale differences between them reflect the fluctuations in image clarity. This calculation method can more accurately capture the subtle blur or clarity changes caused by changes in focal length, lighting conditions, or object surfaces during the shooting process. The application of the hyperbolic tangent function further smoothes these fluctuations and avoids the instability caused by overly drastic changes, so that the clarity index can more robustly reflect the true state of image quality.

[0044] S2, calculate the overlap coefficient between the target image and the reference image, traverse to obtain the overlap coefficient between the target image and the reference image at different positions, and in response to the maximum overlap coefficient, obtain the relative position of the target image and the reference image, and traverse to obtain the relative position of any two adjacent frames of surface grayscale images.

[0045] It should be noted that since the drone or image acquisition structure is affected by external factors such as wind during flight, it often causes the acquisition structure to vibrate or deviate from the predetermined track, which in turn causes the image to shift or distort. Especially in continuous image acquisition, if the drone is not stable enough, some image areas will not be able to cover the transmission line, resulting in missing or blurred areas of the image. This situation will seriously affect the accuracy of subsequent defect identification, because in defect detection based on image analysis, any omission or incompleteness of the image may lead to incorrect diagnosis or missed detection. Missing areas in the image may be misjudged as defect-free or structurally intact, missing important information such as potential damage, corrosion or cracks, further affecting the health assessment and maintenance decisions of the transmission line.

[0046] In one embodiment, any reserved surface grayscale image is used as a target image, and a surface grayscale image of a historical frame adjacent to the target image is used as a reference image.

[0047] The overlap coefficients of the target image and the reference image at different positions are obtained by traversing, and the relative position of the target image and the reference image is obtained in response to the maximum overlap coefficient. It should be explained that although the target image and the reference image are obtained by shooting, the relative position of the target image and the reference image is not clear, that is, how much overlap there is between the target image and the reference image, so it is necessary to fix the position of the reference image unchanged, traverse the overlapped parts of the target image and the reference image at different positions, and determine the case where the overlapped part is the largest.

[0048] The pixels of the overlapping part of the target image and the reference image are regarded as overlapping pixels. The overlap coefficient satisfies the relationship:

[0049] , represents the overlap coefficient, Indicates the number of overlapping pixels. Represents the total number of pixels in the target image. Indicates overlapping pixels The gray value in the reference image, Indicates overlapping pixels Grayscale value in the target image.

[0050] The overlapping pixel areas between the target image and the reference image are identified, and the degree of match between the two images is quantified by comparing the grayscale value differences of these overlapping pixels. This matching evaluation can deeply analyze the differences between the two images in terms of imaging quality, perspective consistency, spatial alignment, etc., thereby achieving more accurate image alignment.

[0051] The relative positions of any two adjacent surface grayscale images can be obtained according to the calculation method of the overlap coefficient.

[0052] In one embodiment, it also includes: after obtaining the relative position, it is necessary to determine whether a reshoot is needed. The pixel points of the overlapping part of the target image and the reference image are taken as overlapping pixel points; the ratio of the overlapping pixel points to the total number of pixel points in the target image is calculated, and the ratio of each surface grayscale image in the preset sampling period is traversed to obtain the mean of all ratios, and the difference between 1 and the mean is used as the update ratio, and the inverse of the update ratio is used as the maximum allowable reshoot time; in response to the overlap coefficient of the surface grayscale image within the maximum allowable reshoot time being less than the overlap threshold, the drone returns to the starting position of the maximum allowable reshoot time at the same speed for reshooting.

[0053] In one embodiment, when the overlap coefficient of a certain frame of the surface grayscale image corresponding to the maximum allowed reshoot time is less than the overlap threshold, the drone returns to the position where the overlap coefficient is less than the overlap threshold at the same speed for reshooting.

[0054] S3, splicing the surface grayscale image according to the relative position to obtain the transmission line map, inputting the transmission line map into the preset neural network, outputting the defect probability value, and completing the detection.

[0055] In one embodiment, the relative positions of any two adjacent surface grayscale images are obtained according to step S2, and all surface grayscale images are spliced ​​to obtain a complete transmission line map.

[0056] The transmission line diagram is input into a preset neural network, which is a convolutional neural network. The convolutional neural network extracts features from the transmission line diagram to obtain image features, and classifies the image features to obtain defect probability values.

[0057] The training process of the neural network includes: taking the transmission line diagram in the history as input information and the true value of the defect probability value of the transmission line diagram as a label to obtain a set of training data, inputting the training data into the preset neural network to obtain the output result; calculating the loss value of the preset neural network through the output result and the label, the loss function uses the cross entropy loss function, back-propagates the error signal according to the loss value, updates the network parameters of the preset neural network to make the loss value smaller; iteratively updates the network parameters of the preset neural network, and stops updating when the preset neural network reaches the set maximum number of training times or the loss value is less than the set loss value to obtain a trained preset neural network.

[0058] According to the defect probability value output by the neural network, it needs to be explained that the output probability values ​​are 0 and 1, that is, the output of the location where there is a defect in the transmission line is 1, and the output of the location where there is no defect in the transmission line is 0, thereby completing the detection of the transmission line.

[0059] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

[0060] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A power tool detection method based on machine vision, characterized in that: include: Continuously obtain multiple frames of surface grayscale images of the power transmission line, perform mixed Gaussian statistics on the surface grayscale images to obtain a pixel value distribution curve, one surface grayscale image corresponds to one pixel value distribution curve, calculate the clarity index according to the pixel value distribution curve, and retain the surface grayscale image with a clarity index greater than a preset clarity threshold; The clarity index satisfies the relationship: for the same pixel value distribution curve, the distribution probability of any maximum value point within the three times standard deviation interval is calculated to construct a distribution probability sequence; the number of pixels within the three times standard deviation interval corresponding to any maximum value point in the pixel value distribution curve is taken as the normal number, and the first maximum value and the second maximum value are respectively obtained, the first maximum value is the maximum value of the normal number of the pixel value distribution curve, and the second maximum value is the maximum value of the normal number of continuous multiple frames of surface grayscale images; , Indicates the clarity index, represents the first maximum value, represents the second maximum value, represents the standard deviation of the distribution probability sequence, represents the hyperbolic tangent function; Any reserved surface grayscale image is used as a target image, and the surface grayscale image of the historical frame adjacent to the target image is used as a reference image; Calculating the overlap coefficient between the target image and the reference image includes: taking the pixel points of the overlapping part of the target image and the reference image as the overlapping pixel points; the overlap coefficient satisfies the relationship: , represents the overlap coefficient, Indicates the number of overlapping pixels. Represents the total number of pixels in the target image. Indicates overlapping pixels The gray value in the reference image, Indicates overlapping pixels Grayscale value in the target image; Obtain the overlap coefficients of the target image and the reference image at different positions by traversing, obtain the relative position of the target image and the reference image in response to the maximum overlap coefficient, and obtain the relative position of any two adjacent surface grayscale images by traversing; The surface grayscale images are spliced ​​according to the relative positions to obtain the transmission line map, which is then input into a preset neural network, and the defect probability value is output to complete the detection.

2. The power tool detection method based on machine vision according to claim 1, characterized in that: The clarity index also satisfies the relationship: , Indicates the clarity index, Indicates the pixel value distribution curve The gray value corresponding to the maximum point is Indicates the pixel value distribution curve The gray value corresponding to the maximum point is represents the total number of maximum points, represents the hyperbolic tangent function.

3. The power tool detection method based on machine vision according to claim 1, characterized in that: The neural network is a convolutional neural network, which extracts features from the transmission line diagram to obtain image features, and classifies the image features to obtain defect probability values.

4. The power tool detection method based on machine vision according to claim 1, characterized in that: The training process of the preset neural network includes: The transmission line diagram in history is used as input information, and the true value of the defect probability value of the transmission line diagram is used as a label to obtain a set of training data, and the training data is input into a preset neural network to obtain an output result; The loss value of the preset neural network is calculated by outputting the results and labels. The loss function uses the cross entropy loss function. The error signal is back-propagated according to the loss value, and the network parameters of the preset neural network are updated to reduce the loss value. Iteratively update the network parameters of the preset neural network. When the preset neural network reaches the set maximum number of training times or the loss value is less than the set loss value, stop updating to obtain a trained preset neural network.

5. The power tool detection method based on machine vision according to claim 1, characterized in that: Also includes: The pixels of the overlapping part of the target image and the reference image are regarded as overlapping pixels; Calculate the ratio of the overlapping pixels to the total number of pixels in the target image, traverse to obtain the ratio of each surface grayscale image within the preset sampling period, calculate the mean of all ratios, take the difference between 1 and the mean as the update ratio, and take the inverse of the update ratio as the maximum allowable retake time; In response to the overlap coefficients of the surface grayscale images within the maximum allowable retake time being less than the overlap threshold, the UAV returns to the starting position of the maximum allowable retake time at the same speed for retake.

6. The power tool detection method based on machine vision according to claim 5, characterized in that: The drone returns to the starting position of the maximum allowable retake time at the same speed to retake the video, and the retake also includes: When the overlap coefficient of the surface grayscale image corresponding to a certain frame is less than the overlap threshold within the maximum allowed reshoot time, the UAV returns to the position where the overlap coefficient is less than the overlap threshold at the same speed for reshoot.

Images