Synthetic power line segmentation data set generation algorithm based on pixel fusion

Abstract

The invention discloses a synthetic power line segmentation data set generation algorithm based on pixel fusion. The method comprises the following steps: generating a virtual power tower, a power line and segmentation true values thereof based on a virtual engine; segmenting a sky region from the background image; obtaining an offline enhancement parameter; and carrying out pixel-level fusion on the virtual power line and the background image and generating a new segmentation truth value. According to the method, computer vision and digital image processing technologies are combined, power line segmentation data sets can be generated in batches, the synthesized segmentation truth values can be automatically generated, and extra manual annotation is not needed. The method has the advantages of low cost, high efficiency, automatic labeling and the like.

Description

Technical Field

[0001] The present invention relates to the field of digital image processing, and in particular to an algorithm for generating a synthetic power line segmentation data set based on pixel fusion. Background Art

[0002] With the increasing demand for electricity in modern society, the safe operation of transmission lines has become a key factor in ensuring the stable development of social economy. Traditional transmission line inspection mainly relies on manual methods, which is not only labor-intensive and inefficient, but also has certain safety hazards and management difficulties. In order to improve the efficiency and accuracy of transmission line inspection, automated inspection methods based on computer vision and image processing technology have emerged.

[0003] Semantic segmentation technology plays a vital role in the automated inspection of power lines. Through semantic segmentation, the power lines and background in the image can be accurately separated, thereby realizing real-time monitoring of the power line status. Specifically, semantic segmentation can identify and distinguish different elements in the image, such as power lines, sky, trees, etc., providing basic data for subsequent analysis and processing. However, in practical applications, it is a challenge to obtain a large number of high-quality power line segmentation annotation datasets. Traditional dataset annotation methods are time-consuming and labor-intensive, and it is difficult to collect power line image data in different environments. To solve this problem, a synthetic power line segmentation dataset generation algorithm based on pixel fusion is proposed. The algorithm uses a virtual engine to generate virtual power towers and power lines, and combines real background images to generate realistic synthetic images and their segmentation truth values ​​through pixel-level fusion, thereby greatly reducing the annotation cost and time of the dataset. The synthetic power line segmentation dataset generation algorithm based on pixel fusion can not only generate a large amount of high-quality training data, but also simulate power line images under different weather, lighting and viewing angles, and improve the robustness and accuracy of the segmentation model in various complex environments. The introduction of this algorithm provides a reliable data foundation for the power line automated inspection system, and is expected to significantly improve the intelligence and automation level of transmission line inspection. Summary of the invention

[0004] To improve the accuracy and anti-interference ability of tree barrier intrusion detection, the present invention provides a synthetic power line segmentation dataset generation algorithm based on pixel fusion. This method uses virtual power towers, power lines and their segmentation truth values ​​generated by a virtual engine, combined with the sky area of ​​the background image extracted based on the dark channel and HLS color space, to generate new segmentation truth values ​​through pixel-level fusion, thereby greatly reducing the annotation cost and time of the dataset and improving the robustness and accuracy of the segmentation model.

[0005] To achieve the above object, the present invention discloses an algorithm for generating a synthetic power line segmentation data set based on pixel fusion, comprising the following steps:

[0006] Step S10: Generate virtual power towers, power lines and their segmentation truth values ​​based on the virtual engine. Automatically generate batches of different virtual power towers and power lines through the virtual engine, and provide their corresponding segmentation truth value images;

[0007] Step S101: using a virtual engine to generate synthetic virtual power lines and power towers with different numbers of strands and sags;

[0008] Step S20: Extract the sky area of ​​the background image based on the dark channel and HLS color space. Convert the image to the HLS color space, generate a dark channel image using the dark channel prior, and segment the dark channel using the Otsu threshold method to generate the sky area;

[0009] Step S201: First, check whether there is a sky area in the image, use the dark channel prior to generate a dark channel image, perform threshold segmentation on the dark channel through the Otsu threshold method, generate a binary image, perform connected region analysis on the binary image, and determine whether there is a sky area that meets the conditions.

[0010] Step S202: Convert the image with sky area into HLS color space, use the dark channel prior of large window to generate dark channel image, reduce the influence of artificial light source at night, use Otsu threshold method to perform threshold segmentation on dark channel, generate rough sky area mask, and calculate HLS threshold range and sky condition according to the rough sky area mask.

[0011] Step S203: generating an HLS-based sky mask according to the estimated HLS threshold range, filtering the grayscale image using a sliding window variance filter, and generating a variance-based sky mask;

[0012] Step S204: performing a logical AND operation on the variance-based sky mask and the HLS-based sky mask to generate an initial sky mask, excluding the determined non-sky area to refine the initial sky mask, performing an opening operation on the refined sky mask to remove small noise, performing a closing operation on the non-sky area to fill the small gaps in the non-sky area, finding the largest connected area through connected area analysis, and generating a final sky mask;

[0013] Step S30: Randomly generate offline enhancement parameters for transformations such as rotation and translation. By randomly generating parameters such as rotation angle and translation distance, data enhancement is performed on the virtual power lines to increase the diversity and complexity of the data;

[0014] Step S40: Fusing the virtual power lines with the background image at the pixel level and generating a new segmentation truth value. The virtual power lines are seamlessly merged with the background image using the pixel fusion method to ensure the consistency of the virtual power lines with the background image in terms of color, brightness, etc., thereby generating a realistic synthetic image and its segmentation truth value. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Flowchart of the steps for generating a synthetic dataset for the present invention.

[0016] Figure 2 This is an effect diagram of extracting the sky area of ​​the background image in the present invention.

[0017] Figure 3 This is a synthetic data image effect diagram generated by the present invention.

[0018] Figure 4 The segmentation truth value of the synthetic data image generated by the present invention. DETAILED DESCRIPTION

[0019] The following describes the implementation of the present invention through specific examples and in conjunction with the accompanying drawings. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific examples, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0020] Step S10: Generate virtual power towers, power lines and their segmentation truth values ​​based on the virtual engine. Automatically generate batches of different virtual power towers and power lines through the virtual engine, and provide their corresponding segmentation truth value images;

[0021] Step S101: using a virtual engine to generate synthetic virtual power lines and power towers with different numbers of strands and sags;

[0022] Step S20: Extract the sky area of ​​the background image based on the dark channel and HLS color space. Convert the image to the HLS color space, generate a dark channel image using the dark channel prior, and segment the dark channel using the Otsu threshold method to generate the sky area;

[0023] Step S201: First, check whether there is a sky area in the image, and use the dark channel prior to generate a dark channel image. The formula of the objective function is:

[0024]

[0025] Where DarkChannel(x) represents the dark channel value at pixel position x, y is a pixel position within the window Ω(x), and I R (y), I G (y), I B (y) are the intensity values ​​of the red, green and blue channels of the image at the pixel position y, respectively. All pixel positions y within the window Ω(x) are traversed, and the minimum value at these positions is the dark channel value.

[0026] The dark channel is threshold segmented by the Otsu threshold method to generate a binary image, and the connected region analysis is performed on the binary image to determine whether there is a sky area that meets the conditions. The formula of the objective function is:

[0027] First generate a normalized histogram of the image:

[0028]

[0029] Among them, p(i) is the probability of gray value i, n i is the number of pixels with grayscale value i, and N is the total number of pixels in the image.

[0030] Then calculate the between-class variance:

[0031]

[0032] Where t is the current threshold, is the inter-class variance, and ω0(t) is the proportion of pixels below the threshold t to the total pixels (the weight of the foreground):

[0033]

[0034] ω1(t) is the ratio of pixels above the threshold t to the total pixels (the weight of the background):

[0035]

[0036] μ0(t) is the average grayscale value of pixels below threshold t:

[0037]

[0038] μ1(t) is the average grayscale value of pixels above the threshold t:

[0039]

[0040] Calculate the inter-class variance of all possible thresholds t and select the threshold t that maximizes the inter-class variance * :

[0041]

[0042] According to the calculated threshold t * Segment the image:

[0043]

[0044] Among them, f(x,y) is the grayscale value of the original image, and g(x,y) is the binarized image.

[0045] Step S202: Convert the image with sky area into HLS color space, use the dark channel prior of large window to generate dark channel image, reduce the influence of artificial light source at night, use Otsu threshold method to perform threshold segmentation on dark channel, generate rough sky area mask, and calculate HLS threshold range and sky condition according to the rough sky area mask.

[0046] Step S203: Generate an HLS-based sky mask according to the estimated HLS threshold range, filter the grayscale image using a sliding window variance filter, and generate a variance-based sky mask. The formula of the objective function is:

[0047]

[0048] Among them, μ(x,y) is the mean value in the window, (x,y) is the current pixel position, is the window radius, w is the window size, I is the grayscale image, and I(x+i,y+i) is the pixel value at the relative position (i,j) in the window;

[0049]

[0050] Among them, v(x,y) is the square mean in the window, I 2 (x+i,y+i) is the square of the pixel value at the relative position (i,j) within the window;

[0051] σ 2 (x,y)=v(x,y)-μ 2 (x,y)

[0052] Among them, σ 2 (x,y) is the variance of the current pixel position (x,y);

[0053]

[0054] Among them, Mask(x,y) is the generated sky mask based on variance, and T is the set variance threshold.

[0055] Step S204: performing a logical AND operation on the variance-based sky mask and the HLS-based sky mask to generate an initial sky mask, excluding the determined non-sky area to refine the initial sky mask, performing an opening operation on the refined sky mask to remove small noise, performing a closing operation on the non-sky area to fill the small gaps in the non-sky area, finding the largest connected area through connected area analysis, and generating a final sky mask;

[0056] Step S30: Randomly generate offline enhancement parameters for transformations such as rotation and translation. By randomly generating parameters such as rotation angle and translation distance, data enhancement is performed on the virtual power lines to increase the diversity and complexity of the data;

[0057] Step S40: Fusing the virtual power lines with the background image at the pixel level and generating a new segmentation truth value. The virtual power lines are seamlessly merged with the background image using the pixel fusion method to ensure the consistency of the virtual power lines with the background image in terms of color, brightness, etc., thereby generating a realistic synthetic image and its segmentation truth value.

Claims

1. A pixel fusion based synthetic power line segmentation dataset generation algorithm, characterized in that: The following steps are involved: S10: Generate virtual power towers, power lines and their segmentation truth values ​​based on the virtual engine; S20: extract the sky area of ​​the background image based on the dark channel and HLS color space; S30: Randomly generate offline enhancement parameters for transformations such as rotation and translation; S40: Fusing the virtual power lines with the background image at pixel level and generating new segmentation truth values.

2. The method for obtaining initial virtual power towers, power lines and their segmentation true values ​​according to claim 1, characterized in that: The virtual engine is used to automatically generate batches of different virtual power towers, power lines and their segmentation truth values.

3. The method for extracting the sky area of ​​the background image based on the dark channel and the HLS color space according to claim 1, characterized in that: The image is converted to HLS color space, the dark channel prior is used to generate a dark channel image, and the dark channel is segmented using the Otsu thresholding method to generate the sky area.

4. The method for fusing virtual power lines with background images at pixel level and generating new segmentation truth values ​​according to claim 1, characterized in that: Pixel fusion is used to seamlessly merge the virtual power lines with the background image and generate new segmentation truth values.

Images