A power line image defect region segmentation method, system, device and medium

Abstract

The present application belongs to the field of image defect region segmentation, and discloses a power line image defect region segmentation method, system, device and medium. The method comprises the following steps: acquiring power line image data, analyzing the power line image data to obtain original image data; generating a gradient response map based on the original image data, obtaining a preliminary geometric contour of the power equipment, matching the preliminary geometric contour with a power equipment template, and establishing line structure information; selecting power equipment pixel coordinates based on the line structure information, and delimiting a target attention region according to the power equipment pixel coordinates; extracting local features of the target attention region to obtain a preliminary defect segmentation map, and generating a defect region mask according to the preliminary defect segmentation map. The present application delimits the target attention region to reduce background interference, integrates various information in the attention region to determine the defect mode, segments the defect region, and optimizes the result through connected region analysis, so that the accuracy, robustness and recognition ability for defect types of defect detection can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image defect region segmentation, in particular to a power line image defect region segmentation method, system, device and medium. BACKGROUND

[0002] The power line image defect region segmentation method is a specific application of the above-mentioned image defect region segmentation technology in the field of power system inspection and maintenance. It specifically refers to using computer vision algorithms to analyze the images of power lines and their attached devices obtained by unmanned aerial vehicles, robots or artificial shooting.

[0003] In the existing power line image analysis technology, analysis resources are often wasted on irrelevant background areas, for example, the texture of complex background such as tree branches adjacent to the power transmission line in the image is misjudged as a defect of the line component, which not only reduces the operation efficiency, but also increases the possibility of false positives. At the same time, most existing technologies rely on feature analysis of single-frame static images, such as judgment based on only the gray scale, color or simple texture of the local area of the image. This way is difficult to effectively deal with environmental factors such as light changes and shadow obstructions, and it is also difficult to distinguish different abnormalities, for example, it is difficult to distinguish between stable dirt attached to the surface of the insulator and instantaneous corona discharge phenomenon, because both may exhibit similar visual features in a single image. Therefore, improvement is needed. SUMMARY

[0004] In view of the above-mentioned existing problems, the present application is proposed.

[0005] Therefore, the present application provides a power line image defect region segmentation method to solve the problems that the existing power line image analysis technology is susceptible to background interference, difficult to cope with environmental factors and unable to effectively distinguish different abnormalities.

[0006] To solve the above technical problems, the present application provides the following technical scheme: a power line image defect region segmentation method, comprising the following steps:

[0007] Obtaining power line image data, analyzing the power line image data to obtain original image data;

[0008] Generating a gradient response map based on the original image data, obtaining a preliminary geometric contour of the power equipment, matching the preliminary geometric contour with a power equipment template, and establishing line structure information;

[0009] Based on the line structure information, selecting power equipment pixel coordinates, and delimiting a target attention area according to the power equipment pixel coordinates;

[0010] Extracting local features of the target attention region obtains a preliminary defect segmentation map, and generating a defect region mask according to the preliminary defect segmentation map.

[0011] As a preferred scheme of the power line image defect region segmentation method, the gradient response map is generated based on the original image data, including:

[0012] Based on the original image data, the intensity difference between the horizontal adjacent pixels and the vertical adjacent pixels is calculated by using the difference operator to obtain the horizontal difference component and the vertical difference component.

[0013] The gradient amplitude is calculated according to the horizontal difference component and the vertical difference component, and the gradient direction is calculated by using the arctangent function to generate the gradient response map.

[0014] As a preferred scheme of the power line image defect region segmentation method, the preliminary geometric contour of the power equipment is obtained, the preliminary geometric contour is matched with the power equipment template, and the line structure information is established, including:

[0015] Based on the gradient response map, the non-edge points are filtered by setting the lower limit of the gradient amplitude, the optimal straight line parameters and the curve equation are obtained by grouping the remaining edge points, the power equipment position is labeled, and the preliminary geometric contour is obtained.

[0016] According to the power equipment template image, the maximum value position is calculated pixel by pixel within the range of the preliminary geometric contour, and the line structure information is established.

[0017] The beneficial effects of the preferred technical scheme are that by setting the lower limit of the gradient amplitude to filter the non-edge points, the noise and non-key information in the image can be effectively removed, and the data amount of subsequent processing can be reduced. The search area is reduced by performing template matching within the range of the preliminary geometric contour, which can improve the matching efficiency and make the established line structure information more rapid and accurate.

[0018] As a preferred scheme of the power line image defect region segmentation method, the power equipment pixel coordinates are selected based on the line structure information, and the target attention region is delineated according to the power equipment pixel coordinates, including:

[0019] According to the line structure information, the geometric position and shape data of the power equipment are obtained, a two-dimensional coordinate list of all pixel points constituting the shape boundary is extracted, and a component boundary coordinate set is obtained.

[0020] The component boundary coordinate set is traversed, the normal vector direction of each point is calculated, the new coordinates are obtained by extending the preset pixel distance according to the normal vector direction, the new coordinates are connected into a polygon or an external rectangle, and the region mask is defined to delineate the target attention region.

[0021] The beneficial effects of the preferred technical scheme are: the boundary range of the power equipment is accurately determined, the interference of the background area on the subsequent defect detection is reduced, the calculation resources are concentrated in the key area, the processing efficiency and the pertinence of detection are improved, and the subsequent analysis of the defects is more efficient and accurate.

[0022] As a preferred scheme of the power line image defect region segmentation method, the local feature of the target attention region is extracted to obtain a preliminary defect segmentation map, including:

[0023] In the target attention region, a neighborhood window is defined according to the line structure information, the average value of the gradient amplitude of all pixels in the window, the dispersion of the gradient direction, the energy and contrast value of the gray level co-occurrence matrix, and the mean and variance of the gray value are calculated, and the local feature statistics are obtained;

[0024] The local feature statistics and the dynamic change region indication are combined and input into a preset decision logic or a lookup table to determine an adaptive structure unit, the pixel value is updated according to the adaptive structure unit, and a preliminary defect segmentation map is obtained.

[0025] The beneficial effects of the preferred technical scheme are: the local feature statistics are calculated in the target attention region, which can comprehensively describe the characteristics of the pixels in the region and provide rich data for defect judgment; the pixel value is updated according to the adaptive structure unit to obtain a preliminary defect segmentation map, which can improve the accuracy and fineness of defect segmentation, and the preliminary segmentation result is closer to the actual defect situation.

[0026] As a preferred scheme of the power line image defect region segmentation method, the dynamic change region indication includes:

[0027] The corner points and patch regions of intensity change in the detection pixel neighborhood in each image in the original image data are detected, the statistical descriptor vectors of the gray level distribution or the gradient direction are calculated around the detected regions, the nearest neighbor matching search is performed in the continuous frame descriptor set, and the inter-frame feature matching pair is obtained;

[0028] According to the inter-frame feature matching pair, the inlier matching pairs with consistent coordinate transformation relationship are screened, the over-determined equation set is established by using the inlier pair coordinates, the optimal affine transformation or perspective transformation matrix parameters are solved, and the matrix transformation is applied to adjust all pixel coordinates of the subsequent frames to obtain the aligned pixel coordinates;

[0029] Based on the aligned pixel coordinates and the registered original image data sequence, the pixel intensity values at the same aligned coordinate positions in the two images are accessed, the absolute values of the difference values are calculated, the difference values are compared with the preset threshold, all pixel positions with the difference values exceeding the threshold are marked, and the dynamic change region indication is obtained.

[0030] As a preferred scheme of the power line image defect region segmentation method, wherein: generating a defect region mask according to the preliminary defect segmentation map comprises:

[0031] Setting a threshold for the preliminary defect segmentation map to a binary image, initializing a label map, sequentially scanning the binary image pixels, if an unmarked foreground pixel is encountered, assigning a new label, and pushing all adjacent foreground pixels into a stack or queue, and looping until the stack or queue is empty to obtain a labeled connected region;

[0032] Traversing the labeled connected region through the label map, using a counter to count the total number of pixels for each unique label, establishing a label and pixel number mapping, recording labels with a pixel number below a preset area threshold, and setting the pixel value corresponding to the label to a background value to obtain a size filtered connected region;

[0033] Based on the size filtered connected region, detecting all foreground pixels adjacent to the background pixels as boundary points, and replacing the original value with the average or median value of the neighborhood pixel value for each boundary point through the mean or median filter kernel, and outputting a defect region mask.

[0034] Another object of the present application is to provide a power line image defect region segmentation system.

[0035] To solve the above technical problems, the present application provides the following technical scheme: a power line image defect region segmentation system, comprising: a data acquisition module for acquiring power line image data, and analyzing the power line image data to obtain original image data;

[0036] A data processing module for generating a gradient response map based on the original image data, obtaining a preliminary geometric contour of the power equipment, matching the preliminary geometric contour with a power equipment template, and establishing line structure information; selecting power equipment pixel coordinates based on the line structure information, and defining a target attention region according to the power equipment pixel coordinates;

[0037] A defect recognition module for extracting local features of the target attention region to obtain a preliminary defect segmentation map, and generating a defect region mask according to the preliminary defect segmentation map.

[0038] The present application provides a computer device comprising a memory and a processor, the memory being used to store computer executable instructions, and the processor being used to execute the computer executable instructions, which when executed by the processor implement the steps of the power line image defect region segmentation method.

[0039] In a fourth aspect, the present application provides a computer readable storage medium storing computer executable instructions, which, when executed by a processor, implement the steps of the power line image defect region segmentation method.

[0040] Compared with the prior art, the present application has the following advantages: the present application acquires power line image data, establishes line structure information, delimits a target attention region to reduce background interference, identifies a dynamic change region through inter-frame feature matching, determines a defect mode in the attention region by comprehensively considering multiple information, segments a defect region, and optimizes the result through connected region analysis, thereby improving the accuracy, robustness and defect type recognition ability of defect detection. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 The overall flowchart of a power line image defect region segmentation method according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0044] Embodiment 1, refer to Figure 1 According to an embodiment of the present application, a power line image defect region segmentation method is provided, which comprises:

[0045] S100: acquiring power line image data, and analyzing the power line image data to obtain original image data;

[0046] S101: generating a gradient response map based on the original image data, obtaining a preliminary geometric contour of the power equipment, matching the preliminary geometric contour with a power equipment template, and establishing line structure information;

[0047] S102: selecting a power equipment pixel coordinate based on the line structure information, and delimiting a target attention region according to the power equipment pixel coordinate;

[0048] S103: Extract the local features of the target attention region to obtain a preliminary defect segmentation map, and generate a defect region mask according to the preliminary defect segmentation map.

[0049] It should be noted that, after the power line image data is acquired, the power equipment is identified and located, and the line structure information is established, which lays a foundation for subsequent targeted analysis. The target attention region is demarcated, the computing resources can be concentrated in the core part where potential defects occur, the interference of the background region is effectively reduced, the processing efficiency and detection directionality can be improved. In the target attention region, different types of defects can be finely and accurately segmented to obtain a preliminary segmentation result, the regularity and reliability of the segmentation result are optimized, and the defect region mask generated finally has higher precision and practical value, and the overall accuracy, robustness and recognition ability of specific defect types of the defect detection are improved.

[0050] Embodiment 2, refer to Figure 1 For an embodiment of the present application, based on the above embodiment, a power line image defect region segmentation method is provided.

[0051] In the embodiment of the present application, the specific process of obtaining the power line image data in step S100 includes:

[0052] The power line image data including power equipment such as insulators and conductors is acquired, the image file path is specified or the video capture device parameters are configured through the file system interface, the data reading operation instruction is started, the returned binary byte sequence data is received, and the image input stream is obtained.

[0053] Based on the image input stream, the file header identifier of the starting part of the byte stream is read, the encoding type, pixel depth and image width and height are determined by comparing the image format library, the byte stream is converted into a pixel value array, and the pixel matrix data is established; based on the pixel matrix data, a two-dimensional array is created, the pixel values are filled in the row and column order, and the row, column index and corresponding intensity value of each element are recorded to obtain the original image data.

[0054] In an alternative embodiment, when acquiring power line image data containing insulators and conductors, a system file interface function can be called through an API. In Windows systems, the CreateFile function or in Linux systems, the open system call, is used to pass in the image file path to obtain a file handle. If a video stream is being processed, the cv::VideoCapture class interface of OpenCV is used to configure device ID, frame rate, resolution, and other parameters, and connect the video source. Data reading instructions are started for the file or video stream. When reading a file, the number of bytes to be read is specified based on a preset buffer size (such as 4 KB), and the ReadFile or read function is used to read in a loop until the end of the file; when reading a video stream, the cv::VideoCapture::read() function is called to capture frame data and store it in the buffer. The system or library function receives the returned raw binary byte sequence data, fills the buffer, and the data block constitutes a complete image input stream.

[0055] In an alternative embodiment, when processing data based on the image input stream, the first data block start byte can be used to identify the file header identifier. JPEG, PNG, BMP files all have their own specific start byte sequence. By maintaining a predefined image format library hash table or dictionary structure, the input stream start byte can be compared with the library key to determine the image encoding type. After determining the encoding type, the subsequent bytes are read according to the format specification from a specific offset to obtain metadata, such as the JFIF or EXIF segment for JPEG or the IHDR block for PNG. After obtaining the encoding type, pixel depth, and image width and height information, the corresponding decoding library is called, such as libjpeg, libpng, or the built-in decoder of OpenCV, to process the remaining byte stream, convert the compressed data into raw pixel values, and form a data stream in raster scan order. Finally, the pixel values are rearranged according to the image width and height and pixel depth to construct a pixel matrix data.

[0056] In an alternative embodiment, based on the width W and height H parsed from the image metadata, a two-dimensional array or a three-dimensional array matching the width and height is declared or allocated in memory, and the array element type matches the pixel depth. The pixel matrix data is traversed through nested loops, and the pixel values at the corresponding positions are assigned to the new array. For grayscale images, one value is read and assigned each time; for RGB images, three values are read. During the assignment process, a record containing the row index, column index, and intensity value is created for each pixel. These records can be a data structure parallel to the two-dimensional array, or the two-dimensional array elements can be designed as a composite type containing coordinates and intensity values, such as defining a PixelInfo structure.

[0057] In the embodiment of the present application, the step S101 generates a gradient response map based on the original image data, obtains a preliminary geometric contour of the power equipment, matches the preliminary geometric contour with a power equipment template, and establishes line structure information, further comprising sub-steps A1-A2:

[0058] A1: Based on the original image data, the intensity difference with the horizontal and vertical adjacent pixels is calculated by using a difference operator to obtain the horizontal and vertical difference components; the gradient amplitude is calculated according to the horizontal and vertical difference components, and the gradient direction is calculated by using an arctangent function to generate a gradient response map.

[0059] A2: Based on the gradient response map, non-edge points are filtered by setting a gradient amplitude lower limit, the remaining edge points are grouped to obtain optimal straight line parameters and curve equations, the power equipment position is labeled, and a preliminary geometric contour is obtained; the line structure information is established by calculating the maximum value position pixel by pixel in the preliminary geometric contour range according to the power equipment template image.

[0060] In the embodiment of the present application, the line structure information is obtained specifically as follows:

[0061] Based on the original image data, that is, a structured two-dimensional array containing the mapping relationship between the pixel coordinates and intensity values, each pixel point in the array is traversed , and a specific difference operator is applied to it, for example, a Sobel operator is selected , the Sobel operator contains two convolution kernels, a horizontal direction kernel and a vertical direction kernel , and the two kernels are respectively convolved with the pixel intensity values in the neighborhood of the pixel point , to calculate the horizontal intensity difference component and the vertical intensity difference component , wherein * represents convolution operation, and the two difference components and reflect the change rate of the image intensity of the pixel point in the horizontal and vertical directions.

[0062] According to the two orthogonal difference components, the gradient amplitude of the pixel point is calculated, and the gradient direction is calculated simultaneously; the process is repeated for all pixel points in the original image data, and finally two new matrices with the same size as the original image are obtained, one stores the gradient amplitude of each pixel point, and the other stores the corresponding gradient direction , and the two matrices together constitute the gradient response map.

[0063] ​Based on the gradient response map, in particular the gradient magnitude A lower threshold of gradient magnitude is set to filter out pixels with insignificant intensity change, which usually correspond to flat regions or noise in the image, and keep pixels with gradient magnitude as potential edge points.

[0064] In an alternative embodiment, the threshold can be set by statistical analysis, for example, calculating the histogram of the entire gradient magnitude and then applying Otsu algorithm to automatically calculate an optimal threshold that can divide the pixels into two classes: edge and non-edge.

[0065] In another alternative embodiment, based on experience setting, for 8-bit grayscale images, by observing the gradient magnitude range of typical edges in a large number of power line image samples, a fixed threshold is set, such as all pixels with gradient magnitude lower than are marked as non-edge points and do not participate in subsequent processing.

[0066] For the edge point set, a parameter space accumulation peak detection method is used, a straight line is fitted by Hough transform, a curve is fitted by generalized Hough transform or random sample consensus algorithm, and by analyzing the spatial position, direction and length of the fitted straight line and curve, combined with the general structure prior knowledge of power lines, it is marked which geometric structures correspond to conductors and which correspond to towers, to obtain the preliminary geometric contour.

[0067] Based on the original image data and the preliminary geometric contour, load the grayscale or binary pattern image used to identify insulators and fittings from the pre-set template library, and use the preliminary geometric contour information to limit the search range of template matching;

[0068] For example, the insulator template searches in the vicinity of the identified tower contour, and the fitting template searches in the vicinity of the end of the position where the insulator template matching is successful or the end point of the conductor contour. This limitation avoids time-consuming global search on the entire image. Within the limited search range, for each template to be matched , slide the template window and move it pixel by pixel on the original image data , calculate the similarity between the current image sub-region and the template , and select the normalized cross-correlation as the similarity measure, whose calculation formula is:

[0069] ,

[0070] wherein is the current image sub-region Similarity between template and image sub-region, is the top-left corner coordinate of the image sub-region, is the pixel index within the template, is the average gray level of the image sub-region, is the average gray level of the template, the calculated NCC value ranges between , and the value closer to 1 indicates a higher matching degree.

[0071] In an alternative embodiment, the sum of squared differences (SSD) can also be selected for calculation:

[0072] ,

[0073] wherein, is the top-left corner coordinate of the image sub-region, is the pixel index within the template.

[0074] If , it is considered that the template corresponding power equipment is successfully matched at the position , and the type and position coordinates of the component are recorded. The process is repeated for all pre-stored insulator and fitting templates, and the type and position information of all successfully matched components are summarized to establish the line structure information.

[0075] It should be noted that the present application calculates the pixel gradient response to generate a gradient response map, which can highlight the areas with obvious changes in pixel intensity in the image, effectively capture potential edge information, and provide a basis for subsequent identification of power line component contours. Setting the gradient amplitude lower limit to filter non-edge points reduces noise and irrelevant information interference, and improves the accuracy of edge extraction. Grouping the remaining edge points and fitting straight line and curve equations accurately mark the position of the conductor and the tower, and preliminarily construct the geometric framework of the power line. Template matching within the preliminary geometric contour limit reduces the search space and improves the matching efficiency. Pixel-by-pixel calculation of similarity and marking of the maximum value position can accurately identify the position of the insulator and the fitting, thereby comprehensively and accurately establishing the line structure information, providing a reliable basis for subsequent targeted defect detection and analysis, and enhancing the accuracy and reliability of the entire power line image analysis system.

[0076] In the embodiment of the present application, based on the line structure information in step S102, the power equipment pixel coordinates are selected, and the target attention region is determined according to the power equipment pixel coordinates, which further includes sub-steps B1-B2:

[0077] B1: According to the line structure information, the geometric position and shape data of the power equipment are obtained, the two-dimensional coordinate list of all pixel points constituting the shape boundary is extracted, and the component boundary coordinate set is obtained;

[0078] B2: Traverse the component boundary coordinate set, calculate the normal vector direction of each point, expand the preset pixel distance according to the normal vector direction to obtain new coordinates, connect the new coordinates to form a polygon or circumscribed rectangle, and define the region mask to delineate the target area of ​​interest.

[0079] Specifically, based on the line structure information, for each identified component, a precise boundary description is extracted. If the component is identified through template matching, its boundary can be obtained by translating the non-background pixel contour of the matched template image according to the matching position. If the component is a conductor or tower, its boundary is determined by the fitted straight line or curve equation. By sampling or calculating the analytical boundary on the curve at a certain step size, the two-dimensional coordinates of all pixels constituting the outer boundary of the component's shape are obtained. The coordinates of the boundary pixels extracted from each component are collected into a list or array;

[0080] For example, an insulator string may correspond to a string containing the coordinates of hundreds of boundary points. A list of all these lists is compiled together to obtain the component boundary coordinate set.

[0081] In one alternative implementation, based on the component boundary coordinate set, the coordinate list of each component is processed, and each boundary point in the list is traversed. Calculation points Local boundary normal vector at the location First, determine the points adjacent to this point on the boundary. and Calculate the approximate value of the tangent vector. Then the tangent vector Rotate 90 degrees to obtain the normal vector or Choose the direction pointing outwards from the component as the correct normal vector direction, and then... Normalization is performed to obtain the unit normal vector. ,in It is the magnitude of the normal vector, along the unit normal vector. Direction, the original boundary point coordinates Add a preset pixel distance value The new coordinates are calculated, and the preset pixel distance value is used here. It is a positive integer representing the outward expansion boundary width for all points in the component boundary coordinate list. All perform this offset calculation, form a new coordinate point list, get the extended boundary coordinates. Based on the extended boundary coordinates, the new coordinate list corresponding to each power equipment is processed, and the points in the list are connected in order according to their order on the original boundary to generate a closed polygon contour, and the new polygon surrounds the original component and an external width of about The region of the pixel, and the target attention region is demarcated.

[0082] In another optional embodiment, a minimum circumscribed rectangle containing all the extended boundary coordinate points is directly calculated, and the boundary of the rectangle is taken as the boundary of the target region. Whether the generated polygon contour or the circumscribed rectangle boundary is adopted, the next step is to define a region mask composed of all the pixel points surrounded by the boundary. For the polygon contour, a standard graphics filling algorithm is used to create a binary mask image with the same size as the original image, set all pixel positions marked as internal to 1 and other pixel positions to 0, and the binary mask image clearly identifies the pixel region that needs to be analyzed in the subsequent process, and demarcates the target attention region.

[0083] It should be noted that the component boundary coordinate set is obtained based on the line structure information, the boundary range of each component is accurately determined, and accurate basic data is provided for subsequent processing. The coordinate traversal calculation vector direction and the extension of the preset pixel distance fully consider the area where defects may occur around the component, and avoid missing potential defect positions. The polygon contour is generated by connecting the extended boundary coordinates, or the circumscribed rectangle boundary is calculated, and then the region mask is defined to demarcate the target attention region, so that the subsequent image analysis and calculation resources can be concentrated in these key regions, the interference of the background region is reduced, the processing efficiency is improved, the potential defect region is focused, the pertinence and accuracy of the power line component defect detection are enhanced, and defects can be more quickly and accurately identified and analyzed.

[0084] In the embodiment of the application, the local features of the target attention region are extracted in step S103 to obtain a preliminary defect segmentation map, and a defect region mask is generated according to the preliminary defect segmentation map, and further comprising sub-steps C1-C6:

[0085] C1: Detect the corner points and patch regions of intensity changes in the detection pixel neighborhood in each image in the original image data, calculate the statistical descriptor vectors of the gray distribution or gradient direction around the detected regions, and perform nearest neighbor matching search in the continuous frame descriptor set to obtain inter-frame feature matching pairs;

[0086] C2: According to the inter-frame feature matching pair, the inner point matching pair with consistent coordinate transformation relationship is screened, an over-determined equation set is established by using the inner point coordinates, the optimal affine transformation or perspective transformation matrix parameters are solved, and all pixel coordinates of subsequent frames are adjusted by applying the matrix transformation to obtain aligned pixel coordinates;

[0087] C3: Based on the aligned pixel coordinates and the registered original image data sequence, the pixel intensity values of the same aligned coordinate positions in the two frames are accessed, the absolute values of the difference values are calculated, the difference values are compared with a preset threshold value, all pixel positions with the difference values exceeding the threshold value are marked to obtain a dynamic change area indication.

[0088] C4: In the target attention area, a neighborhood window is defined according to the line structure information, taking each pixel as the center, the average value of the gradient amplitude of all pixels in the window, the dispersion of the gradient direction, the energy and contrast value of the gray level co-occurrence matrix, and the mean and variance of the gray value are calculated to obtain local feature statistics; the local feature statistics and the dynamic change area indication are input into a preset decision logic or lookup table to determine an adaptive structure unit, the pixel value is updated according to the adaptive structure unit, and a preliminary defect segmentation map is obtained.

[0089] C5: The preliminary defect segmentation map is set to a binary image, a label map is initialized, and the binary image pixels are sequentially scanned; if an unmarked foreground pixel is encountered, a new label is assigned, and all adjacent foreground pixels are pushed into a stack or a queue; the cycle is processed until the stack or the queue is empty to obtain a labeled connected region; the labeled connected region is traversed through the label map, a counter is used to count the total number of pixels of each unique label, a label and pixel number mapping is established, labels with a pixel number lower than a preset area threshold value are recorded, and the pixel values corresponding to the labels are set to background values to obtain a size-filtered connected region.

[0090] C6: Based on the size-filtered connected region, all foreground pixels adjacent to the background pixels are detected as boundary points, and each boundary point is replaced by the average value or the median value of the neighborhood pixel values through the mean or median filter kernel to output a defect region mask.

[0091] In the embodiment of the present application, the preliminary defect segmentation map is obtained by the following steps:

[0092] In the target attention area, a neighborhood window is defined according to the line structure information, taking each pixel as the center, the average value of the gradient amplitude of all pixels in the window, the dispersion of the gradient direction, the energy and contrast value of the gray level co-occurrence matrix, and the mean and variance of the gray value are calculated to obtain local feature statistics;

[0093] ​Based on the local feature statistics and the dynamic change region indication, the input statistics value and the change indication mark are sent to a preset decision logic or lookup table, and the corresponding structure unit type is output. If an elongated high-contrast region is detected, a linear unit is selected. If a sheet-shaped texture change region is detected, a square unit is selected, and the length or side length is determined to obtain an adaptive structure unit.

[0094] Based on the adaptive structure unit, in each pixel of the target attention region, the neighborhood range is determined according to the selected unit shape and size, the intensity values of all pixels in the neighborhood are read, the maximum value is selected for expansion or the minimum value is selected for corrosion, the center pixel value is updated, and a preliminary defect segmentation map is obtained.

[0095] Specifically, in the target attention region, according to the previously established line structure information, the feature of each pixel point in the target attention region is extracted ;

[0096] A square neighborhood window is defined with as the center, and the size is , where is a positive odd number, for example, set . The size is selected according to the relationship between the typical scale of the texture or defect expected to be captured and the image resolution. By evaluating the feature extraction effect of different sizes on sample images, the arithmetic mean value of the gradient amplitudes of all pixels in the window is calculated using the previously calculated gradient response map, the dispersion of the gradient direction is calculated, for example, using the variance of the gradient direction vector or calculating the entropy of the gradient direction histogram;

[0097] In an optional embodiment, the average vector of the direction vector in the window is calculated, and then the dispersion is calculated. Based on the original gray values in the window, the gray co-occurrence matrix is calculated; based on , the texture feature is calculated, and finally the statistics of the original gray values in the window, including the mean and variance, are calculated. These calculated feature values are combined into a feature vector associated with the pixel point . This process is repeated for all pixels in the target attention region to obtain the local feature statistics.

[0098] Based on the local feature statistics and the dynamic change region indication, for each pixel point in the target attention region, its feature vector and the corresponding dynamic change indication mark are sent as inputs into a preset decision logic or lookup table.

[0099] In one alternative implementation, the decision logic or lookup table is established based on prior knowledge of common defect patterns in power lines and analysis of image features. The rules can be formulated by domain experts or learned from sample data labeled with defect types and corresponding structural units through machine learning methods.

[0100] This decision process is performed on each pixel in the target region of interest to obtain a mapping map where each pixel location stores the selected structural unit type, size, and orientation, resulting in an adaptive structural unit.

[0101] Based on the adaptive structural unit graph, for each pixel in the target region of interest Morphological operations are performed on the current pixel. For example, if the decision logic determines that the current pixel needs to be processed using linear structural units, and the goal is to extract cracks that are brighter than the background, then a dilation operation is selected, based on... The specified linear unit length and direction determine the... The neighborhood of the center point includes the center point and points extending 2 pixels along the 45-degree and 225-degree directions, for a total of 5 pixels: Read the intensity values ​​of these 5 pixels in the original image data (or the preprocessed image, such as a grayscale image), and select the maximum value among these intensity values. and this maximum value As the output image If the pixel value of the location is determined to require processing with square structural units (possibly corresponding to corrosion detection), and the goal is to extract corrosion that is darker than the background (e.g., the corrosion color is darker), then the erosion operation is selected, based on... The specified square cell side length, for example, 3 pixels, determines the... Centered Within a neighborhood, read the intensity values ​​of these 9 pixels and select the minimum value among them. and this minimum value As the output image The pixel value of the location, for all pixels within the ROI. Based on its corresponding adaptive structural unit and preset morphological operations (dilation or erosion, depending on whether the target to be extracted is a bright feature or a dark feature, which is usually determined together in the design decision logic), the maximum or minimum value in the neighborhood is calculated, and the pixel value at the corresponding position in the output image is updated. After traversing all pixels in the ROI, the generated output image is the preliminary defect segmentation map.

[0102] In this embodiment of the invention, the step of obtaining the defect region mask is as follows:

[0103] A threshold is set for the preliminary defect segmentation image to convert it into a binary image, a label map is initialized, and the binary image pixels are sequentially scanned. If an unmarked foreground pixel is encountered, a new label is assigned, and all adjacent foreground pixels are pushed into a stack or queue. The process is repeated until the stack or queue is empty, resulting in a labeled connected region.

[0104] Based on the labeled connected region, the label map is traversed, and a counter is used to count the total number of pixels for each unique label. A label-pixel number mapping is established, and labels with pixel numbers below a pre-set area threshold are recorded. The pixel values corresponding to these labels are set to background values, resulting in a size-filtered connected region.

[0105] Based on the size-filtered connected region, all foreground pixels adjacent to background pixels are detected as boundary points. A 3x3 or 5x5 mean or median filter kernel is applied to each boundary point, replacing the original value with the average or median value of the neighboring pixels. The defect region mask is output.

[0106] Specifically, for the preliminary defect segmentation image, a global or local adaptive threshold is first set to convert it into a binary image. A label map with the same size as the binary image and all elements set to 0 is initialized. A label counter current_label is set to 1. Then, each pixel point of the binary image is traversed in raster scan order If the current pixel point is a foreground pixel and its corresponding value in the label map is 0, indicating that it has not been labeled, a connected region labeling process is started. The label counter current_label is increased by 1, and the next unprocessed pixel of the binary image is scanned. After traversing all pixels, each independent connected foreground region in the label map is assigned a unique positive integer label, resulting in a labeled connected region.

[0107] Based on the labeled connected region, the size of each connected region needs to be counted and small regions need to be filtered out. First, a counter data structure is created to store the number of pixels corresponding to each label, such as a hash map with label values as keys and the number of times the label appears as values. Then, each pixel point of the label map is traversed again , and its label value is read . If , the entry with the key is found in the hash map, and its corresponding value is increased by 1. If the key does not exist, a new entry is created in the map , and the value is initialized to 1. After the traversal is complete, the hash map stores the unique label of each connected region and its corresponding total number of pixels. A pre-set area threshold is set to remove connected regions that are too small and are likely to be caused by noise or image artifacts.

[0108] Based on the size-filtered connected components, identify the boundary pixels of these retained regions. Traverse the filtered image. For each foreground pixel (i.e., value 1 or label > 0), check the pixels in its 4-neighborhood or 8-neighborhood. If the foreground pixel has at least one neighbor that is a background pixel, then the foreground pixel is determined to be a boundary point. Collect a list of coordinates of all such boundary points.

[0109] Smoothing filters are applied to these detected boundary points to reduce jagged edges caused by pixelation or previous steps. Optional applications may be used. or The filtering kernel, and the mean filtering or median filtering methods.

[0110] In one of the alternative factual approaches, if the following is selected The median filter works by creating a copy of the filtered connected components as the output mask image, and then iterating through each point in the previously collected list of boundary point coordinates. Define a around it The neighborhood window is used to read the values ​​of 9 pixels within the window in the size-filtered connected component image. These 9 values ​​are then sorted, and the median value is selected. Finally, the corresponding value in the output mask image is used. The pixel values ​​at each location are updated to the calculated median. This operation is performed only on boundary points, while the pixel values ​​inside the region remain unchanged. After processing all boundary points, the output mask image is the final defect region mask, which has relatively smooth boundaries and maintains the filtered connectivity of the internal regions.

[0111] It should be noted that by detecting corner points and patchy areas and performing feature matching, inter-frame correspondences can be found. Pixels are aligned through coordinate transformation, and intensity differences are calculated to mark dynamically changing areas, providing a basis for distinguishing between defects and interference. Multiple local features are calculated in the target area of ​​interest, and combined with dynamic change indicators to determine adaptive structural units to obtain a preliminary defect segmentation map, which can accurately identify different types of defects. Binarizing the preliminary segmentation map, marking connected regions, and size filtering remove small noise areas. Boundary points are then filtered and smoothed, resulting in a more regular defect area mask boundary, effectively reducing false positives and improving the accuracy of defect detection, thus providing support for the maintenance and repair of power lines.

[0112] Embodiment 3, the above is a schematic scheme of a power line image defect region segmentation method. It should be noted that the technical scheme of the power line image defect region segmentation system belongs to the same concept as the technical scheme of the power line image defect region segmentation method described above. The technical details of the power line image defect region segmentation system in this embodiment are not described in detail, and can be referred to the description of the technical scheme of the power line image defect region segmentation method described above.

[0113] The embodiment also provides a power line image defect region segmentation system, comprising:

[0114] A data acquisition module is configured to acquire power line image data, analyze the power line image data to obtain original image data.

[0115] A data processing module is configured to generate a gradient response map based on the original image data, acquire a preliminary geometric contour of a power device, match the preliminary geometric contour with a power device template, and establish line structure information; select a power device pixel coordinate based on the line structure information, and demarcate a target attention region according to the power device pixel coordinate.

[0116] A defect recognition module is configured to extract local features of the target attention region to obtain a preliminary defect segmentation map, and generate a defect region mask based on the preliminary defect segmentation map.

[0117] The embodiment also provides an electronic device suitable for power line image defect region segmentation, comprising a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to implement the power line image defect region segmentation method proposed in the above embodiment.

[0118] The embodiment also provides a storage medium having a computer program stored thereon, and the program is executed by a processor to implement the power line image defect region segmentation method proposed in the above embodiment.

[0119] The storage medium proposed in the embodiment and the power line image defect region segmentation method proposed in the above embodiment belong to the same inventive concept, and the technical details not described in detail in the embodiment can be referred to the above embodiment, and the embodiment has the same beneficial effects as the above embodiment.

[0120] Those skilled in the art can clearly understand the present application by the above description of the embodiments, and the present application can be realized by software and necessary general hardware, and of course, can also be realized by hardware. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH, a hard disk, or an optical disc, and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods of various embodiments of the present application.

[0121] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application, and although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all should be covered in the scope of the claims of the present application.

Claims

1. A method for segmenting a defect region of a power line image, characterized by, The method comprises the following steps: acquiring power line image data, and parsing the power line image data to obtain original image data; generating a gradient response map based on the original image data, acquiring a preliminary geometric contour of a power device, matching the preliminary geometric contour with a power device template, and establishing line structure information; based on the line structure information, selecting a power device pixel coordinate, and demarcating a target attention region according to the power device pixel coordinate; extracting local features of the target attention region to obtain a preliminary defect segmentation map, and generating a defect region mask according to the preliminary defect segmentation map; extracting local features of the target attention region to obtain a preliminary defect segmentation map, including: in the target attention region, defining a neighborhood window centered on each pixel according to the line structure information, calculating the average value of the gradient amplitude of all pixels in the window, the dispersion of the gradient direction, the energy and contrast value of the gray level co-occurrence matrix, and the mean and variance of the gray value, and obtaining local feature statistics; combining the local feature statistics and the dynamic change region indication, inputting into a preset decision logic or lookup table to determine an adaptive structure unit, updating the pixel value according to the adaptive structure unit to obtain a preliminary defect segmentation map; the dynamic change region indication includes: detecting the corner and patch region of the intensity change in the detection pixel neighborhood in each image in the original image data, calculating the statistical descriptor vector of the gray distribution or the gradient direction around the detected region, and performing nearest neighbor matching search in the continuous frame descriptor set to obtain the inter-frame feature matching pair; according to the inter-frame feature matching pair, screening the inlier matching pair with consistent coordinate transformation relationship, using the inlier pair coordinates to establish an over-determined equation set, and solving to obtain the optimal affine transformation or perspective transformation matrix parameters, and applying the matrix transformation to adjust all pixel coordinates of the subsequent frames to obtain aligned pixel coordinates; based on the aligned pixel coordinates and the registered original image data sequence, accessing the pixel intensity values at the same aligned coordinate position in two images, calculating the absolute value of the difference value, comparing the difference value with a preset threshold, marking all pixel positions with the difference value exceeding the threshold, and obtaining the dynamic change region indication; the preliminary defect segmentation map generates a defect region mask, including: setting a threshold for the preliminary defect segmentation map to a binary image, initializing a label map, sequentially scanning the binary image pixels, if a foreground pixel is encountered, assigning a new label, and pressing all adjacent foreground pixels into a stack or queue, and processing in a loop until the stack or queue is empty, to obtain a labeled connected region; traverse the label map through the labeled connected region, use a counter to count the total number of pixels of each unique label, establish a label and pixel number mapping, record the label whose pixel number is lower than the preset area threshold, and set the pixel value corresponding to the label to a background value, to obtain a size filtered connected region; based on the size filtered connected region, detect all foreground pixels adjacent to the background pixels as boundary points, replace the original value of each boundary point with the average value or median value of the neighborhood pixel value through mean or median filtering kernel, and output a defect region mask.

2. The power line image defect region segmentation method of claim 1, wherein, the method for generating a gradient response map based on the original image data comprises the following steps: Based on the original image data, the intensity difference with horizontal adjacent pixels and vertical adjacent pixels is calculated by using difference operator to obtain horizontal difference component and vertical difference component; The gradient amplitude is calculated according to the horizontal difference component and the vertical difference component, and the gradient direction is calculated by using arctangent function to generate a gradient response map.

3. The power line image defect region segmentation method of claim 2, wherein, The preliminary geometric contour of the power equipment is obtained, the preliminary geometric contour is matched with the power equipment template, and the line structure information is established, including: Based on the gradient response map, the non-edge points are filtered by setting the lower limit of the gradient amplitude, the remaining edge points are grouped to obtain the optimal straight line parameters and curve equation, the power equipment position is labeled, and the preliminary geometric contour is obtained; According to the power equipment template image, the maximum value position is calculated pixel by pixel within the range of the preliminary geometric contour to establish the line structure information.

4. The power line image defect region segmentation method of claim 3, wherein, Based on the line structure information, the power equipment pixel coordinates are selected, and the target attention region is delimited according to the power equipment pixel coordinates, including: obtaining the geometric position and shape data of the power equipment according to the line structure information, extracting the two-dimensional coordinate list of all pixel points constituting the shape boundary, and obtaining the component boundary coordinate set; The normal vector direction of each point is calculated by traversing the component boundary coordinate set, the new coordinates are obtained by extending the preset pixel distance according to the normal vector direction, the new coordinates are connected into a polygon or an external rectangle, and the target attention region is delimited by defining the region mask.

5. A power line image defect region segmentation system applying the power line image defect region segmentation method according to any one of claims 1 to 4, characterized by, It includes: A data acquisition module is configured to acquire power line image data and analyze the power line image data to obtain original image data; A data processing module is configured to generate a gradient response map based on the original image data, obtain a preliminary geometric contour of power equipment, match the preliminary geometric contour with a power equipment template, and establish line structure information; Based on the line structure information, the power equipment pixel coordinates are selected, and the target attention region is delimited according to the power equipment pixel coordinates; A defect identification module is configured to extract local features of the target attention region to obtain a preliminary defect segmentation map, and generate a defect region mask based on the preliminary defect segmentation map; Extracting local features of the target attention region to obtain a preliminary defect segmentation map includes: in the target attention region, defining a neighborhood window centered on each pixel according to the line structure information, calculating the average value of the gradient amplitude, the dispersion of the gradient direction, the energy and contrast value of the gray level co-occurrence matrix, and the mean and variance of the gray value of all pixels in the window, and obtaining local feature statistics; The dynamic change area indication is combined with the local feature statistics and input into a preset decision logic or lookup table to determine an adaptive structure unit, the pixel value is updated according to the adaptive structure unit, and a preliminary defect segmentation map is obtained; The dynamic change area indication includes: detecting the corner and patch area of intensity change in the detection pixel neighborhood in each image in the original image data, calculating the statistical descriptor vector of the gray distribution or the gradient direction around the detected area, and performing nearest neighbor matching search in the continuous frame descriptor set to obtain the inter-frame feature matching pair; According to the inter-frame feature matching pairs, the inlier matching pairs with consistent coordinate transformation relationship are screened, an over-determined equation set is established by using the inlier pairs, optimal affine transformation or perspective transformation matrix parameters are solved, and all pixel coordinates of subsequent frames are adjusted by using the matrix transformation to obtain aligned pixel coordinates; Based on the aligned pixel coordinates and the registered original image data sequence, pixel intensity values at the same aligned coordinate positions in two frames are accessed, absolute values of difference values are calculated, the difference values are compared with a preset threshold, all pixel positions with the difference values exceeding the threshold are marked, and a dynamic change area indication is obtained; The preliminary defect segmentation map generates a defect region mask, including: setting a threshold for the preliminary defect segmentation map to a binary image, initializing a label map, sequentially scanning pixels of the binary image, if an unmarked foreground pixel is encountered, assigning a new label, and pushing all adjacent foreground pixels into a stack or a queue, and performing a loop until the stack or the queue is empty, to obtain a labeled connected region; The labeled connected region is used to traverse the label map, a counter is used to count the total number of pixels of each unique label, a label and pixel number mapping is established, labels with pixel numbers below a preset area threshold are recorded, and pixel values corresponding to the labels are set to background values, to obtain a size-filtered connected region; Based on the size-filtered connected region, all foreground pixels adjacent to background pixels are detected as boundary points, each boundary point is replaced by the average or median value of the neighborhood pixel values by using mean or median filtering kernel, and a defect region mask is output. 6.A computer device, comprising a memory and a processor, the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, when the computer executable instructions are executed by the processor, the steps of a power line image defect region segmentation method in any one of claims 1 to 4 are implemented. 7.A computer readable storage medium, storing computer executable instructions, when the computer executable instructions are executed by a processor, the steps of a power line image defect region segmentation method in any one of claims 1 to 4 are implemented.

Images