Insulator multispectral image instance segmentation method

Abstract

The application discloses a kind of insulator multispectral image instance segmentation methods, belong to transmission line external insulation state on-line detection technical field;Construct multi-dimensional feature threshold set to insulator multispectral image is binarized and removes irrelevant background information;In insulator binary image, through image tilt correction and image segmentation method, determine insulator positioning process and parameter;In original insulator multispectral image, insulator positioning process and parameter are used to carry out insulator positioning and image segmentation;In segmented image, through multi-dimensional feature threshold set, generate insulator mask and then complete instance segmentation, this method can realize the extraction of efficient, accurate insulator multispectral image insulator area spectral information, facilitate power system operation and maintenance personnel to use instance segmented insulator multispectral image to carry out subsequent spectral data research.

Description

Technical Field

[0001] This invention belongs to the field of online detection technology of external insulation status of transmission lines, and particularly relates to a method for segmenting insulator multispectral images. Background Technology

[0002] Insulators are the most numerous and diverse electrical components in power transmission and transformation equipment. Operating under constant exposure to air, they are inevitably affected by agricultural and industrial pollution, leading to contamination, aging, and damage. This significantly reduces their external insulation strength, ultimately causing large-scale, prolonged power outages and seriously threatening the safe and stable operation of the power system. Therefore, accurately detecting the condition of insulators is crucial for ensuring their safe and stable operation.

[0003] Currently, there are limited methods for detecting the energized condition of insulators, with visible light, infrared, and ultraviolet imaging being the main methods. However, visible light images have low accuracy in diagnostic models in practical applications. The detection results of infrared and ultraviolet images are affected not only by the condition of the insulator itself but also by its operating voltage and operating load. Moreover, these methods can only reflect abnormalities in the late stages of fault development and are difficult to detect potential fault hazards such as contamination, aging, and damage in a timely manner.

[0004] Online insulator condition detection based on spectral imaging technology is a novel remote sensing method proposed in recent years. It boasts advantages such as high accuracy and result visualization, making it significant for achieving rapid, accurate, non-contact, and non-destructive detection of external insulation conditions. Multispectral imaging devices, with their compact size, simple structure, and high image resolution, are well-suited for use as UAV payloads to acquire spectral image information of line insulators. Since the images acquired by multispectral imaging devices contain a large amount of irrelevant background information besides the insulator itself, instance segmentation of multispectral images of insulators becomes a crucial foundation and prerequisite for spectral analysis of line insulator conditions. However, most existing instance segmentation methods apply to grayscale or color images, which are poorly adapted to multispectral insulator images. Therefore, optimization of image instance segmentation methods using insulator spectral features is necessary. Summary of the Invention

[0005] The purpose of this invention is to provide a method for segmenting instances of multispectral images of insulators, so as to solve the technical problem of low efficiency and accuracy of current multispectral image segmentation of insulators.

[0006] To achieve the above objectives, the specific technical solution of the insulator multispectral image instance segmentation method of the present invention is as follows:

[0007] A method for segmenting instances of a multispectral image of an insulator, comprising the following steps:

[0008] Step S01: Construct a multi-dimensional feature threshold set to perform irrelevant background information threshold removal operation on the original insulator multispectral image to obtain a background-removed binarized image of the insulator.

[0009] Step S02: On the insulator binarized image from step S01, the insulator positioning process and parameters are determined by image tilt correction and image segmentation methods.

[0010] Step S03: On the original insulator multispectral image from step S01, insulator positioning and image segmentation are performed using the insulator positioning process and parameters determined in step S02.

[0011] Step S04: On the image segmented in step S03, an insulator mask is generated using the multidimensional feature threshold set to complete the insulator multispectral image instance segmentation.

[0012] Furthermore, in step S01, the original multispectral images of the insulators are collected by capturing images of double-stranded suspension insulator strings suspended on the crossarm of the tower using a multispectral camera. The insulator regions in each band image are in a registered state in the spatial dimension.

[0013] Furthermore, in step S01, the irrelevant background information threshold removal operation includes the following steps:

[0014] Step S01-1, Construct a multidimensional feature set:

[0015] K = {k1, k2, k3, k4}

[0016] in, x1 to x n These represent the spectral reflectance of each band at the same location in the original multispectral image of the insulator, from smallest to largest, in order of increasing center wavelength.

[0017] Step S01-2: Set the initial threshold of the multidimensional feature set K and construct the multidimensional feature threshold set V = {v1, v2, v3, v4}. Use V to binarize the original insulator multispectral image. With the goal of removing background interference after binarization, iterate and optimize the threshold using the grid search method.

[0018] Step S01-3: Use the current best multidimensional feature threshold set V to binarize the original insulator multispectral image, and complete the threshold removal operation for irrelevant background information.

[0019] Furthermore, in step S02, determining the insulator positioning process and parameters includes the following steps:

[0020] Step S02-1, introduce the plane transformation formula:

[0021]

[0022] Where x and y are the abscissa and ordinate of a point on the binary image plane of the insulator, respectively; f(x,y) is the gray value of the point (x,y) on the binary image plane of the insulator; θ is the inclination angle of the line along which the current integration is performed; s is the distance from the origin to the line along which the current integration is performed; and δ(z) = 0 (z ≠ 0). The gray value f(x,y) of point (x,y) on the binary image plane of the insulator is integrated along the straight line xcosθ+ysinθ=s with a distance s from the origin and a tilt angle of θ, and then projected onto the corresponding point on the transformation plane. The tilt angle θ corresponding to the maximum value on the transformation plane is selected, and the image is rotated counterclockwise by 90°-θ to complete one tilt correction. The tilt angle θ is recorded.

[0023] Step S02-2: Starting from the leftmost edge of the image after tilt correction, search column by column to the right. When the percentage of non-zero pixel values ​​in a column exceeds 20%, it is considered that the region containing the left-end insulator string has been entered. When the percentage of non-zero pixel values ​​in a column is less than 10%, it is considered that the region containing the left-end insulator string has ended, and the current column e is marked. l ;

[0024] Step S02-3: Starting from the rightmost end of the image after tilt correction, search column by column to the left. The judgment logic is the same as in step S02-2. Mark the end column e of the region of the right-end insulator string. r Select column value m = (e l +e r The image after one tilt correction is roughly segmented using ) / 2 as the segmentation position, and the coarse segmentation column value m is recorded.

[0025] Step S02-4: Using the tilt correction method in step S02-1, perform secondary tilt correction on the two roughly segmented images respectively, and record the tilt angle θ of the left insulator string image. l The image of the right-end insulator string is tilted at an angle θ. r ;

[0026] Steps S02-5 involve operating on the two images after secondary tilt correction, searching column by column from the leftmost edge of the image to the right. When the percentage of non-zero pixel values ​​in a column exceeds 30%, it is considered that the region containing the insulator string has been entered, and the current column c is marked. l When the percentage of non-zero pixel values ​​in a column is less than 15%, the insulator string region is considered to have ended and the current column c is marked. r According to c l and c r Perform column-wise image segmentation, then search row by row from the top of the image downwards. When the percentage of non-zero pixel values ​​in a row exceeds 30%, it is considered to have entered the region containing the insulator string, and the current row is marked.u When the percentage of non-zero pixel values ​​in a row is less than 30%, the current row is considered the last row of the insulator string region and the current row is marked as r. d Record the precise segmentation parameters A = {c l ,c r ,r u ,r d}

[0027] Furthermore, in step S03, the insulator positioning process includes the first tilt correction, coarse image segmentation, second tilt correction, and precise image segmentation performed sequentially in step S02. The insulator positioning parameters include the tilt angle θ, the coarse segmentation column value m, and the tilt angle θ of the left-end insulator string image recorded sequentially in step S02. l The tilt angle θ of the right-end insulator string image r Precise segmentation parameter A.

[0028] Furthermore, in step S04, generating the insulator mask includes the following steps:

[0029] Step S04-1: The multidimensional feature threshold set constructed in step S01 is used to perform binarization processing on the precisely segmented insulator multispectral image in step S03 to generate a binarization matrix of the same size as the original precisely segmented insulator multispectral image.

[0030] Step S04-2: The binarized matrix in step S04-1 is directly multiplied by the corresponding element of the precisely segmented insulator multispectral image in step S03, so that the image retains only the original information of the insulator string region.

[0031] The insulator multispectral image instance segmentation method of the present invention has the following advantages: The present invention relates to an insulator multispectral image instance segmentation method, which constructs a multidimensional feature threshold set to binarize the insulator multispectral image and remove irrelevant background information; on the binarized insulator image, the insulator positioning process and parameters are determined by image tilt correction and image segmentation methods; on the original insulator multispectral image, image segmentation is performed using the insulator positioning process and parameters; on the segmented image, an insulator mask is generated by using a multidimensional feature threshold set to complete instance segmentation.

[0032] This invention fully utilizes the scientific nature and robustness of multidimensional feature threshold sets, and combines a stable image tilt correction algorithm and reasonable image segmentation logic to achieve efficient and accurate extraction of spectral information of insulator regions from multispectral images of insulators. This facilitates power system operation and maintenance personnel to conduct subsequent spectral data research using multispectral images of insulators segmented from examples. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the insulator multispectral image instance segmentation method of the present invention.

[0034] Figure 2 This is a multispectral image of the original insulator from Embodiment 1 of the present invention.

[0035] Figure 3 This is a background-removed binarized image of an insulator according to Embodiment 1 of the present invention.

[0036] Figure 4 This is a multispectral image of the segmented left-end insulator string according to Embodiment 1 of the present invention.

[0037] Figure 5 The image shows the segmentation visualization results of the left-end insulator string instance in Embodiment 1 of the present invention. Detailed Implementation

[0038] To better understand the purpose, structure, and function of this invention, the following detailed description of a multispectral image instance segmentation method for insulators, in conjunction with the accompanying drawings, is provided.

[0039] Example 1:

[0040] like Figure 1 The flowchart shown is a method for segmenting insulator multispectral images according to this application. It can be seen that this application provides a method for segmenting insulator multispectral images, specifically including the following steps:

[0041] Step S01: Construct a multi-dimensional feature threshold set, which identifies insulator regions in spatially registered states in images of double-stranded suspension insulator strings suspended on the crossarm of a tower, obtained by capturing images of these insulators in each band using a multispectral camera. Figure 2 The original multispectral image of the insulator shown is subjected to an irrelevant background information thresholding removal operation to obtain, as shown below. Figure 3 The image shown is a binarized image of an insulator with the background removed.

[0042] Step S01-1: Construct a multidimensional feature set K = {k1,k2,k3,k4}, where x1 to x n These represent the spectral reflectance of each band at the same location in the original multispectral image of the insulator, from smallest to largest wavelength.

[0043] Step S01-2: Set the initial threshold of the multidimensional feature set K and construct the multidimensional feature threshold set V = {v1, v2, v3, v4}. Use V to binarize the original insulator multispectral image. With the goal of removing background interference after binarization, iterate and optimize the threshold using the grid search method.

[0044] Step S01-3: Use the current best multidimensional feature threshold set V to binarize the original insulator multispectral image, and complete the threshold removal operation for irrelevant background information.

[0045] Step S02: Determine the insulator positioning process and parameters on the binary image of the insulator using image tilt correction and image segmentation methods.

[0046] Step S02-1, introduce the plane transformation formula:

[0047]

[0048] Where x and y are the abscissa and ordinate of a point on the binary image plane of the insulator, respectively; f(x,y) is the gray value of the point (x,y) on the binary image plane of the insulator; θ is the inclination angle of the line along which the current integration is performed; s is the distance from the origin to the line along which the current integration is performed; and δ(z) = 0 (z ≠ 0). The gray value f(x,y) of point (x,y) on the binary image plane of the insulator is integrated along a straight line with a distance s from the origin and a tilt angle of θ, and then projected onto the corresponding point on the transformation plane. The tilt angle θ corresponding to the maximum value on the transformation plane is selected, and the image is rotated counterclockwise by 90°-θ to complete one tilt correction. The tilt angle θ is recorded.

[0049] Step S02-2: Starting from the leftmost edge of the image after tilt correction, search column by column to the right. When the percentage of non-zero pixel values ​​in a column exceeds 20%, it is considered that the region containing the left-end insulator string has been entered. When the percentage of non-zero pixel values ​​in a column is less than 10%, it is considered that the region containing the left-end insulator string has ended, and the current column e is marked. l .

[0050] Step S02-3: Starting from the rightmost end of the image after tilt correction, search column by column to the left. The judgment logic is the same as in step S02-2. Mark the end column e of the region of the right-end insulator string. r Select column value m = (e l +e r The image after tilt correction is roughly segmented using ) / 2 as the segmentation position, and the coarse segmentation column value m is recorded.

[0051] Step S02-4: Using the tilt correction method in step S02-1, perform secondary tilt correction on the two roughly segmented images respectively, and record the tilt angle θ of the left insulator string image. l The image of the right-end insulator string is tilted at an angle θ. r .

[0052] Steps S02-5 involve operating on the two images after secondary tilt correction, searching column by column from the leftmost edge of the image to the right. When the percentage of non-zero pixel values ​​in a column exceeds 30%, it is considered that the region containing the insulator string has been entered, and the current column c is marked. l When the percentage of non-zero pixel values ​​in a column is less than 15%, the insulator string region is considered to have ended and the current column c is marked. r According to c l and c r Perform column-wise image segmentation, then search row by row from the top of the image downwards. When the percentage of non-zero pixel values ​​in a row exceeds 30%, it is considered to have entered the region containing the insulator string, and the current row is marked. u When the percentage of non-zero pixel values ​​in a row is less than 30%, the current row is considered the last row of the insulator string region and the current row is marked as r. d Record the precise segmentation parameters A = {c l ,c r ,r u ,r d}

[0053] Step S03: On the original insulator multispectral image from Step S01, the insulator positioning process performed sequentially in Step S02—namely, primary tilt correction, coarse image segmentation, secondary tilt correction, and precise image segmentation—and the insulator positioning parameters recorded sequentially in Step S02—namely, tilt angle θ, coarse segmentation column value m, and tilt angle θ of the left-end insulator string image—are applied. l The tilt angle θ of the right-end insulator string image r Insulator localization and image segmentation are performed using precise segmentation parameter A, where the segmented multispectral image of the left-end insulator string is shown below. Figure 4 As shown.

[0054] Step S04: On the image segmented in step S03, an insulator mask is generated using the multidimensional feature threshold set to complete the multispectral image instance segmentation of the insulator. The visualization result of the left-end insulator string instance segmentation is shown below. Figure 5 As shown.

[0055] Step S04-1: The multidimensional feature threshold set constructed in step S01 is used to perform binarization processing on the precisely segmented insulator multispectral image in step S03 to generate a binarization matrix of the same size as the original precisely segmented insulator multispectral image.

[0056] Step S04-2: The binarized matrix in step S04-1 is directly multiplied by the corresponding element of the precisely segmented insulator multispectral image in step S03, so that the image retains only the original information of the insulator string region.

[0057] Depend on Figure 5It can be seen that after instance segmentation of the original insulator multispectral image, the spectral information of the insulator string in the image is completely preserved, and the original complex background interference information is removed to a great extent, which facilitates subsequent research and analysis of the insulator target.

[0058] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for segmenting instances of a multispectral image of an insulator, characterized in that, Includes the following steps: Step S01: Construct a multi-dimensional feature threshold set to perform irrelevant background information threshold removal operation on the original insulator multispectral image to obtain a background-removed binarized image of the insulator. Step S02: On the insulator binarized image from step S01, the insulator positioning process and parameters are determined by image tilt correction and image segmentation methods. In step S02, determining the insulator positioning process and parameters includes the following steps: Step S02-1, introduce the plane transformation formula: ； Where x and y are the abscissa and ordinate of a point on the binary image plane of the insulator, respectively; f(x,y) is the gray value of the point (x,y) on the binary image plane of the insulator; θ is the inclination angle of the line along which the current integration is performed; s is the distance from the origin to the line along which the current integration is performed; when z≠0, δ(z)=0, and The gray value f(x,y) of a point (x,y) on the binary image plane of the insulator is drawn along a straight line with a distance s from the origin and an inclination angle of θ. Integrate and project onto the corresponding point on the transformation plane. Select the tilt angle θ corresponding to the maximum value on the transformation plane and rotate the image counterclockwise by 90°-θ to complete one tilt correction. Record the tilt angle θ. Step S02-2: Starting from the leftmost edge of the image after tilt correction, search column by column to the right. When the percentage of non-zero pixel values ​​in a column exceeds 20%, it is considered that the region containing the left-end insulator string has been entered. When the percentage of non-zero pixel values ​​in a column is less than 10%, it is considered that the region containing the left-end insulator string has ended, and the current column e is marked. l ; Step S02-3: Starting from the rightmost end of the image after tilt correction, search column by column to the left. The judgment logic is the same as in step S02-2. Mark the end column e of the region of the right-end insulator string. r Select column value m=(e l +e r The image after one tilt correction is roughly segmented using ) / 2 as the segmentation position, and the coarse segmentation column value m is recorded. Step S02-4: Using the tilt correction method in step S02-1, perform secondary tilt correction on the two roughly segmented images respectively, and record the tilt angle θ of the left insulator string image. l The image of the right-end insulator string is tilted at an angle θ. r ; Steps S02-5 involve operating on the two images after secondary tilt correction, searching column by column from the leftmost edge of the image to the right. When the percentage of non-zero pixel values ​​in a column exceeds 30%, it is considered that the region containing the insulator string has been entered, and the current column c is marked. l When the percentage of non-zero pixel values ​​in a column is less than 15%, the insulator string region is considered to end and the current column c is marked. r According to c l and c r Perform column-wise image segmentation, then search row by row from the top of the image downwards. When the proportion of non-zero pixel values ​​in a row exceeds 30%, it is considered to have entered the region containing the insulator string, and the current row is marked. u When the percentage of non-zero pixel values ​​in a row is less than 30%, the current row is considered the last row of the insulator string region and the current row is marked as r. d Record the precise segmentation parameters A={c l ,c r ,r u ,r d }; Step S03: On the original insulator multispectral image from step S01, insulator positioning and image segmentation are performed using the insulator positioning process and parameters determined in step S02. Step S04: On the image segmented in step S03, an insulator mask is generated using the multidimensional feature threshold set to complete the insulator multispectral image instance segmentation.

2. The insulator multispectral image instance segmentation method according to claim 1, characterized in that, In step S01, the original multispectral images of the insulators are collected by taking pictures of the double-stranded suspension insulator strings suspended on the crossarm of the tower using a multispectral camera. The insulator regions in each band image are in a registered state in spatial dimension.

3. The insulator multispectral image instance segmentation method according to claim 1, characterized in that, In step S01, the irrelevant background information threshold removal operation includes the following steps: Step S01-1, construct a multidimensional feature set: K = {k1, k2, k3, k4}; in, , , , x1 to x n k1 represents the spectral reflectance of each band at the same location in the original multispectral image of the insulator, from smallest to largest center wavelength; k2 represents the reflectance distribution trend; k3 represents the standard deviation of reflectance; k4 represents the root mean square value of reflectance; k5 represents the average reflectance. Step S01-2: Set the initial threshold of the multidimensional feature set K and construct the multidimensional feature threshold set V={v1,v2,v3,v4}. Use V to binarize the original insulator multispectral image. With the goal of removing background interference after binarization, the threshold is iterated and optimized by the grid search method. Step S01-3: Use the current best multidimensional feature threshold set V to binarize the original insulator multispectral image, and complete the threshold removal operation for irrelevant background information.

4. The insulator multispectral image instance segmentation method according to claim 1, characterized in that, In step S03, the insulator positioning process includes the first tilt correction, coarse image segmentation, second tilt correction, and precise image segmentation performed sequentially in step S02. The insulator positioning parameters include the tilt angle θ, coarse segmentation column value m, and tilt angle θ of the left-end insulator string image recorded sequentially in step S02. l The tilt angle θ of the right-end insulator string image r Precise segmentation parameter A.

5. The insulator multispectral image instance segmentation method according to claim 1, characterized in that, In step S04, generating the insulator mask includes the following steps: Step S04-1: The multidimensional feature threshold set constructed in step S01 is used to perform binarization processing on the precisely segmented insulator multispectral image in step S03 to generate a binarization matrix of the same size as the original precisely segmented insulator multispectral image. Step S04-2: The binarized matrix in step S04-1 is directly multiplied by the corresponding element of the precisely segmented insulator multispectral image in step S03, so that the image retains only the original information of the insulator string region.

Images