Image segmentation method and device for small-scale component, equipment and storage medium

Abstract

The embodiment of the invention discloses an image segmentation method and device for a small-scale component, equipment and a storage medium. The method comprises the steps of obtaining a to-be-segmented image with a to-be-segmented small-scale component; identifying and positioning a small-scale component in the to-be-segmented image by using a multi-scale analysis technology to obtain a target image area of the small-scale component; performing boundary optimization processing on the target image area by using a preset adaptive threshold adjustment mechanism to obtain a preliminary image segmentation result; correcting the preliminary image segmentation result by using pre-collected priori knowledge and context information of the small-scale component to obtain a corrected image segmentation result; performing morphological operation processing on the corrected image segmentation result to obtain an optimized image segmentation result; and re-projecting the optimized image segmentation result into the original image coordinate system to generate a final image segmentation result of the small-scale component. Through the above mode, the precision and reliability of image segmentation of small-scale components are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a method and device, equipment and storage medium for image segmentation of a scale component. Background Art

[0002] Although existing image segmentation methods can meet the needs of power transmission line inspection to a certain extent, they still have some shortcomings. Traditional image segmentation methods mainly rely on fixed thresholds and simple edge detection techniques. These methods are prone to mis-segmentation and missed segmentation when processing images with uneven lighting and complex backgrounds, resulting in low recognition accuracy. In addition, these methods often ignore the prior knowledge and contextual information of components, and cannot effectively reduce the impact of noise, further reducing the reliability of the segmentation results.

[0003] Therefore, there is currently a problem of poor effect on image segmentation of small-scale components. Summary of the invention

[0004] The main purpose of the present invention is to provide a method and apparatus, device and storage medium for image segmentation of scale components, which can solve the problem of poor image segmentation effect of small-scale components in the prior art.

[0005] To achieve the above object, the present invention provides a method for image segmentation of small-scale components in a first aspect, the method comprising:

[0006] Acquire an image to be segmented, where the image to be segmented is an inspection image of a power transmission line, and there are small-scale components to be segmented in the inspection image;

[0007] Using multi-scale analysis technology to identify and locate small-scale components in the image to be segmented, and obtaining a target image area of ​​the small-scale components;

[0008] Utilizing a preset adaptive threshold adjustment mechanism to perform boundary optimization processing on the target image region to obtain a preliminary image segmentation result;

[0009] Using the prior knowledge and context information of the small-scale components collected in advance, the preliminary image segmentation result is corrected to obtain a corrected image segmentation result;

[0010] Performing morphological operation processing on the corrected image segmentation result to obtain an optimized image segmentation result;

[0011] The optimized image segmentation result is reprojected into the original image coordinate system to generate the final image segmentation result of the small-scale component.

[0012] In a feasible implementation, the multi-scale analysis technology is used to identify and locate small-scale components in the image to be segmented to obtain a target image region of the small-scale components, and the method also includes:

[0013] Defuzzifying the image to be segmented by using image stabilization technology to obtain a processed image to be segmented;

[0014] The detail expression in the processed image to be segmented is enhanced based on the dynamic range expansion technology to obtain an enhanced image to be segmented.

[0015] In a feasible implementation, the using of multi-scale analysis technology to identify and locate small-scale components in the image to be segmented to obtain a target image region of the small-scale components includes:

[0016] Using multi-scale analysis technology, the image to be segmented is decomposed at multiple levels to obtain a multi-scale decomposition result;

[0017] In the multi-scale space, a local contrast enhancement technique is used to highlight the difference between the small-scale components and the background on the multi-scale decomposition result, so as to obtain a contrast-enhanced multi-scale decomposition result;

[0018] The multi-scale decomposition result is processed by using a pre-trained machine learning algorithm to identify small-scale components, so as to obtain a target image region of the small-scale components. The machine learning model is trained using a number of image samples of small-scale components.

[0019] In one feasible implementation, the machine learning algorithm includes a convolutional neural network, and the convolution operation of the convolutional neural network can be expressed as:

[0020]

[0021] In the formula, (F smooth ) i,j,k is the value of the smoothed feature map at position (i, j) and channel k; K 1x1 (1,1,c,k) is the weight of the 1x1 convolution kernel at position (1,1), input channel c, and output channel k; F fuse ) i,j,c is the value of the fused feature map at position (i, j) and channel c; b k is the bias term on output channel k;

[0022] F fuse =F up +F high

[0023] In the formula, F up is the upsampled feature map, Fhigh It is a high-resolution feature map;

[0024] The upsampling operation can be expressed as:

[0025] F up =Upsample(F low )

[0026] In the formula, F low is a low-resolution feature map, and the upsampling operation is used to convert the low-resolution feature map F low Zoom in to the high-resolution feature map F high Same size H high ×W high .

[0027] In a feasible implementation, the boundary optimization processing of the target image region is performed by using a preset adaptive threshold adjustment mechanism to obtain a preliminary image segmentation result, including:

[0028] An adaptive segmentation algorithm and the target image region are used to perform boundary recognition to obtain a preliminary image segmentation result, and the adaptive threshold adjustment mechanism includes an adaptive segmentation algorithm.

[0029] In one feasible implementation, the adaptive segmentation algorithm includes:

[0030] Multiscale Gaussian kernel function:

[0031]

[0032] In the formula, F context (x,y) i is the eigenvector of the i-th scale, σ i is the standard deviation of the Gaussian kernel at the i-th scale;

[0033] Multimodal Gaussian kernel function:

[0034]

[0035] F i (x,y) is the i-th feature, σ i is the Gaussian kernel standard deviation of the i-th feature;

[0036] Dynamically adjust the threshold:

[0037]

[0038] T 0 (x,y) is the initial threshold, α is the adjustment coefficient, F i (x,y ′ ) is the eigenvector in the neighborhood;

[0039] The final dynamic adaptive threshold function can be expressed as:

[0040]

[0041] in:

[0042] F(x,y) is the multimodal feature vector, P is the prior knowledge, is a conditional random field model, A is an attention module; C is a context-aware module, and F fuse (x,y) is the fused feature vector, and N is the deep neural network model.

[0043] In a feasible implementation, the prior knowledge includes the shape standard, size standard and color standard of the small-scale components, and the context information includes the relative position standard and spatial layout standard of the small-scale components in the image. Then, the preliminary image segmentation result is corrected by using the pre-collected prior knowledge and context information of the small-scale components to obtain the corrected image segmentation result, including:

[0044] Determine a current feature set of the small-scale components in the preliminary image segmentation result, wherein the current feature set includes a current shape, a current size and a current color, a current relative position and a current spatial layout of the small-scale components;

[0045] The preliminary image segmentation result is corrected based on the current feature set, prior knowledge and context information to obtain a corrected image segmentation result.

[0046] To achieve the above object, the second aspect of the present invention provides an image segmentation device for small-scale components, the device comprising:

[0047] Image acquisition module: used to acquire the image to be segmented, wherein the image to be segmented is an inspection image of the power transmission line, and the inspection image contains small-scale components to be segmented;

[0048] Region positioning module: used to identify and locate small-scale components in the image to be segmented by using multi-scale analysis technology to obtain the target image region of the small-scale components;

[0049] Boundary optimization module: used to perform boundary optimization processing on the target image area by using a preset adaptive threshold adjustment mechanism to obtain a preliminary image segmentation result;

[0050] Result correction module: used to correct the preliminary image segmentation result by using the prior knowledge and context information of small-scale components collected in advance, and obtain the corrected image segmentation result;

[0051] Result optimization module: used for performing morphological operation on the corrected image segmentation result to obtain an optimized image segmentation result;

[0052] Reprojection module: used to reproject the optimized image segmentation results into the original image coordinate system to generate the final image segmentation results of small-scale components.

[0053] To achieve the above-mentioned purpose, the third aspect of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps shown in the first aspect and any feasible implementation method.

[0054] To achieve the above-mentioned purpose, the fourth aspect of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps shown in the first aspect and any feasible implementation method.

[0055] The embodiments of the present invention have the following beneficial effects:

[0056] The present invention provides an image segmentation method for small-scale components, the method comprising: obtaining an image to be segmented, the image to be segmented is an inspection image of a power transmission line, and the inspection image contains small-scale components to be segmented; using a multi-scale analysis technology to identify and locate the small-scale components in the image to be segmented, and obtaining a target image region of the small-scale components; using a preset adaptive threshold adjustment mechanism to perform boundary optimization processing on the target image region, and obtaining a preliminary image segmentation result; using pre-collected prior knowledge and context information of the small-scale components, correcting the preliminary image segmentation result, and obtaining a corrected image segmentation result; performing morphological operation processing on the corrected image segmentation result, and obtaining an optimized image segmentation result; and re-projecting the optimized image segmentation result into an original image coordinate system, and generating a final image segmentation result of the small-scale components. Through the above method, image segmentation processing using multi-scale analysis technology, adaptive threshold adjustment mechanism, prior knowledge of small-scale components and contextual information is realized, which is more conducive to the image segmentation of small-scale components, improves the accuracy and reliability of image segmentation of small-scale components, and improves the segmentation effect. Finally, the optimized image segmentation result is re-projected into the original image coordinate system to generate the final image segmentation result of the small-scale component, which provides an important basis for subsequent fault diagnosis and health status evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] in:

[0059] Figure 1 This is a flow chart of a method for image segmentation of small-scale components in an embodiment of the present invention;

[0060] Figure 2 Another flow chart of a method for image segmentation of a small-scale component in an embodiment of the present invention;

[0061] Figure 3 It is a structural block diagram of an image segmentation device for a small-scale component in an embodiment of the present invention;

[0062] Figure 4 4 is a structural block diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] It should be noted that the present invention aims to solve the problems of low recognition accuracy, poor robustness and susceptibility to environmental factors in existing image segmentation methods in power transmission line inspection. Through high-resolution imaging, local contrast enhancement, adaptive threshold adjustment, prior knowledge correction and morphological post-processing and other technologies, the recognition accuracy and anti-interference ability of small-scale components are improved, accurate segmentation results are generated, and a reliable basis is provided for fault diagnosis and health assessment. For details, please refer to the following content.

[0065] See also Figure 1 , Figure 1 Flow chart of a method for image segmentation of small-scale components in an embodiment of the present invention. The method can be applied to both a terminal and a server. The terminal can be a desktop terminal or a mobile terminal. The mobile terminal can be at least one of a mobile phone, a tablet computer, a laptop computer, etc. The server can be implemented by an independent server or a server cluster composed of multiple servers. This embodiment is described by applying to a terminal as an example. Figure 1 The method shown comprises the following steps:

[0066] 101. Acquire an image to be segmented, where the image to be segmented is an inspection image of a power transmission line, and there are small-scale components to be segmented in the inspection image;

[0067] It should be noted that the image segmentation method of small-scale components shown in the present application is used to segment the image of small-scale components from the image to be segmented to achieve target segmentation, wherein the small-scale components are various electrical components provided on the transmission line, such as insulators or connectors. Generally speaking, these electrical components are smaller than other components, so they are considered to be small-scale components, wherein the image to be segmented is acquired during the inspection of the transmission line, and there are small-scale components to be segmented in the inspection image. Specifically, a high-resolution image acquisition device can be used to inspect the transmission line to obtain a high-definition inspection image, and the image to be segmented includes the high-definition inspection image. The image acquisition device can be a camera or a drone, etc.

[0068] 102. Using a multi-scale analysis technique to identify and locate small-scale components in the image to be segmented, and obtaining a target image region of the small-scale components;

[0069] It should be noted that the present invention uses multi-scale analysis technology to identify and accurately locate small-scale components in the image based on the characteristics of transmission line inspection imaging. This technology can effectively cope with the challenges brought about by changes in lighting conditions and background complexity. Therefore, firstly, multi-scale analysis technology is used to identify and accurately locate small-scale components in the image, effectively cope with changes in lighting and background complexity, and obtain the target image area.

[0070] 103. Utilize a preset adaptive threshold adjustment mechanism to perform boundary optimization processing on the target image region to obtain a preliminary image segmentation result;

[0071] Furthermore, an adaptive threshold adjustment mechanism is applied to optimize the definition of small-scale component boundaries and improve the recognition accuracy of component edges. This mechanism dynamically adjusts the threshold according to the grayscale change rate of the component edge to ensure the accuracy of the edge. That is, the adaptive threshold adjustment mechanism is then used to optimize the definition of component boundaries and improve edge recognition accuracy. The threshold is dynamically adjusted according to the edge grayscale change rate to ensure accuracy, and a preliminary image segmentation result is obtained.

[0072] 104. Using the prior knowledge and context information of the small-scale components collected in advance, the preliminary image segmentation result is corrected to obtain a corrected image segmentation result;

[0073] It should be noted that the preliminary segmentation results are corrected using the prior knowledge and context information of the components to reduce the impact of noise and erroneous segmentation. The prior knowledge and context information of the components are collected in advance as standard reference information. The preliminary segmentation results are corrected using the prior knowledge and context information of the components, which can reduce noise and erroneous segmentation and obtain a corrected image segmentation result.

[0074] 105. Perform morphological operation processing on the corrected image segmentation result to obtain an optimized image segmentation result;

[0075] After obtaining the corrected image segmentation result, morphological operations are continued to be performed as a post-processing means to improve the quality of the segmentation result, wherein the morphological operations include but are not limited to dilation, erosion, opening and closing operations, etc., aiming to make the contours of the segmented components smoother and continuous; that is, by performing morphological operations such as dilation, erosion, opening and closing operations as post-processing means, the quality of the segmentation result is improved and the contours of the components are smoother and continuous.

[0076] 106. Reproject the optimized image segmentation result into the original image coordinate system to generate the final image segmentation result of the small-scale component.

[0077] Finally, the optimized image segmentation result is reprojected into the original image coordinate system to generate the final small-scale component image segmentation result. This result not only clearly marks the specific position of each component, but also provides detailed attribute information, which provides an important basis for subsequent fault diagnosis and health status evaluation.

[0078] In summary, the small-scale component image segmentation method based on the characteristics of transmission line inspection imaging proposed in the present invention effectively identifies and accurately locates small-scale components through multi-scale analysis technology, improves edge recognition accuracy through adaptive threshold adjustment mechanism, reduces noise and mis-segmentation by using prior knowledge and context information, and improves the smoothness and continuity of the segmentation results through morphological operations. The final generated segmentation results not only mark the specific location of the components, but also provide detailed attribute information, which significantly improves the accuracy and efficiency of fault diagnosis and health assessment.

[0079] The present invention provides an image segmentation method for small-scale components, the method comprising: obtaining an image to be segmented, the image to be segmented is an inspection image of a power transmission line, and the inspection image contains small-scale components to be segmented; using a multi-scale analysis technology to identify and locate the small-scale components in the image to be segmented, and obtaining a target image region of the small-scale components; using a preset adaptive threshold adjustment mechanism to perform boundary optimization processing on the target image region, and obtaining a preliminary image segmentation result; using pre-collected prior knowledge and context information of the small-scale components, correcting the preliminary image segmentation result, and obtaining a corrected image segmentation result; performing morphological operation processing on the corrected image segmentation result, and obtaining an optimized image segmentation result; and re-projecting the optimized image segmentation result into an original image coordinate system, and generating a final image segmentation result of the small-scale components. Through the above method, image segmentation processing using multi-scale analysis technology, adaptive threshold adjustment mechanism, prior knowledge of small-scale components and contextual information is realized, which is more conducive to the image segmentation of small-scale components, improves the accuracy and reliability of image segmentation of small-scale components, and improves the segmentation effect. Finally, the optimized image segmentation result is re-projected into the original image coordinate system to generate the final image segmentation result of the small-scale component, which provides an important basis for subsequent fault diagnosis and health status evaluation.

[0080] See also Figure 2 , Figure 2 FIG. 4 is another flow chart of a method for image segmentation of a small-scale component in an embodiment of the present invention. Figure 2 The steps shown include:

[0081] 201. Acquire an image to be segmented, where the image to be segmented is an inspection image of a power transmission line, and there are small-scale components to be segmented in the inspection image;

[0082] It should be noted that step 201 and Figure 1 The content of step 101 is similar and will not be described here to avoid repetition. For details, please refer to the content of the aforementioned step 101.

[0083] In one feasible implementation, high-resolution cameras or cameras mounted on drones can be used to collect high-definition images of transmission lines and their components. These images may be affected by many factors, such as different light intensities, weather conditions, and the complexity of the background environment. By using high-resolution cameras or cameras mounted on drones to capture high-quality images of transmission lines and their components, the core purpose of this step is to obtain the clearest and most detailed original image data possible, ensuring that even tiny components can be accurately captured in the image for subsequent processing and analysis.

[0084] 202. Deblurring the image to be segmented by using an image stabilization technology to obtain a processed image to be segmented;

[0085] 203. Enhancing the detail expression in the processed image to be segmented based on a dynamic range extension technology to obtain an enhanced image to be segmented;

[0086] Furthermore, image stabilization technology is used to reduce image blur caused by wind speed or equipment vibration, and dynamic range expansion technology is used to enhance the details in the image, especially for small-scale components, to ensure that the image quality is not affected by environmental changes; it involves the initial processing stage of the image, that is, image stabilization technology is used to reduce the image blur caused by wind or vibration of the equipment itself, and obtain the deblurred image to be segmented. In addition, in order to improve the image quality, dynamic range expansion technology will also be used to increase the expressiveness of dark and bright details in the image, especially for some small-sized key components, to ensure that they can maintain good visual effects even in a complex environment, so as to obtain an enhanced image to be segmented.

[0087] 204. Using a multi-scale analysis technique to identify and locate small-scale components in the image to be segmented, and obtaining a target image region of the small-scale components;

[0088] It should be noted that step 204 and Figure 1 The content of step 102 is similar and will not be described here to avoid repetition. For details, please refer to the content of the aforementioned step 102.

[0089] In a feasible implementation, step 204 may include steps A01-A03:

[0090] A01. Decomposing the image to be segmented at multiple levels by using a multi-scale analysis technique to obtain a multi-scale decomposition result;

[0091] A02, in a multi-scale space, using a local contrast enhancement technique to highlight the difference between the small-scale components and the background on the multi-scale decomposition result, thereby obtaining a contrast-enhanced multi-scale decomposition result;

[0092] A03 uses a pre-trained machine learning algorithm to perform small-scale component recognition processing on the multi-scale decomposition results to obtain a target image area of ​​the small-scale component. The machine learning model is trained using a number of image samples of small-scale components.

[0093] For example, through step A01: using multi-scale analysis technology, the input high-definition image is decomposed at multiple levels to capture the features in the image from coarse to fine, especially the subtle differences in small-scale components. This process may include wavelet transform or other multi-resolution analysis methods, which are not limited here, to obtain a multi-scale decomposition result, and use multi-scale analysis technology to perform in-depth processing on the previously obtained high-definition image. This technology allows the image to be decomposed into different levels, from the relatively rough overall outline to the more delicate partial features, and gradually extracts useful information from the image. Multi-scale analysis can be implemented in a variety of ways, such as wavelet transform, pyramid transform, etc., in order to better capture the subtle differences in small-scale components in the image.

[0094] Furthermore, in step A02, in the multi-scale space, the local contrast enhancement technology is applied to highlight the difference between the target component and the background, so as to improve the contrast between the small-scale component and the surrounding background, improve the efficiency and accuracy of subsequent processing, and make the small-scale component clearly visible under various lighting conditions. In order to further improve the contrast between the target component and the surrounding background in the image, the local contrast enhancement technology will be applied. This technology helps to clearly display the details of the component under different lighting conditions, making the image area of ​​the small-scale component more prominent and distinct, and obtaining a more prominent and distinct contrast-enhanced multi-scale decomposition result, which is convenient for subsequent recognition and analysis.

[0095] Exemplarily, features of different scales are extracted through multi-scale analysis techniques (such as wavelet transform and pyramid transform) to improve the recognition accuracy of small-scale components. Through Gaussian filtering and downsampling, a Gaussian pyramid of the image is constructed to extract low-frequency information of different scales. Gaussian pyramid is a multi-resolution representation method of an image. It constructs an image sequence by continuously applying Gaussian filtering and smoothed downsampling, and the resolution of each subsequent image is lower than the previous one. This method can effectively reduce the amount of image data while retaining the main structural information of the image. It is particularly suitable for application scenarios that need to process multi-scale features, such as target detection, image registration, and texture analysis. The first step in constructing a Gaussian pyramid is to start with the original image, which is usually considered to be the bottom layer of the pyramid, that is, the highest resolution layer. A Gaussian filter is applied to the image of the current layer. The function of the Gaussian filter is to smooth the image, remove high-frequency noise, and keep the main features of the image unchanged. The kernel size and standard deviation of the Gaussian filter determine the degree of smoothing. Larger kernels and standard deviations will result in stronger smoothing effects. After Gaussian filtering, the image is downsampled, usually by taking the value of every other pixel, which will reduce the width and height of the image by half. The result of the downsampling constitutes the next layer of the Gaussian pyramid. The above Gaussian filtering and downsampling process can be repeated many times, and a new low-resolution image will be generated each time until the predetermined number of layers is reached or the image size becomes small enough.

[0096] It should be noted that the local contrast enhancement technology makes the boundary between small-scale components and the background more obvious by adjusting the brightness difference between each pixel in the image and its adjacent area, so that it is easier to identify and segment. This step helps to reduce the interference of background noise and improve the accuracy and efficiency of image processing. The present invention collects images of transmission lines and their components through high-resolution imaging equipment, ensuring the clarity and detail of small-scale components; then the local contrast enhancement technology is applied to effectively improve the contrast between small-scale components and the background, and improve the efficiency and accuracy of subsequent processing. These two steps work together to significantly improve the image quality and the accuracy of component recognition, providing a reliable basis for subsequent fault diagnosis and health assessment.

[0097] Finally, step A03 combines machine learning algorithms, such as convolutional neural network and other machine learning algorithm models, and pre-trains the model to identify and classify small-scale components in the image. The model can learn the multi-scale feature representation of the component, thereby improving the robustness and accuracy of recognition; specifically, a deep learning model such as a convolutional neural network (CNN) is used to train a system that can automatically identify and classify small-scale components in an image. This training process allows the model to learn how to understand the characteristics of the component from a multi-scale perspective, so that it can make accurate judgments and classifications when faced with a variety of images, greatly enhancing the robustness and recognition accuracy of the system. At this time, the image sample can be a multi-scale small-scale component image, such as a low-frequency sub-image and a high-frequency sub-image at each decomposition scale.

[0098] Exemplarily, by combining advanced machine learning algorithms, especially convolutional neural networks, the model is trained to identify and classify small-scale components in images. The model can learn multi-scale feature representations of components and significantly improve the robustness and accuracy of recognition through multi-level feature extraction and fusion.

[0099] The model uses a multi-scale feature pyramid network (FPN) and attention mechanism, which can automatically focus on key feature areas while suppressing the influence of background noise. By introducing a conditional random field model, the segmentation boundary is further optimized to ensure the accuracy and coherence of the component edges. These technical innovations work together to enable the model to maintain a high level of recognition performance under complex backgrounds and changing lighting conditions.

[0100] Convolution smoothing is to reduce the aliasing effect introduced during upsampling. Usually a 1x1 convolution layer is applied after the feature map fusion. Using methods such as bilinear interpolation or nearest neighbor interpolation, the low-resolution feature map is enlarged to the same size as the high-resolution feature map, and the upsampled feature map is element-wise added to the low-level feature map to obtain the fused feature map. A 1x1 convolution layer is applied to the fused feature map. This convolution layer usually contains a 3x3 or 5x5 convolution kernel, but in order to keep the resolution of the feature map unchanged, a 1x1 convolution kernel is usually selected.

[0101] is the feature map of low resolution, where H low and W low are the height and width, C low is the number of channels. is a high-resolution feature map, where H high and W high are the height and width, C high is the number of channels. The upsampling operation can be expressed as:

[0102] F up=Upsample(F low )

[0103] The upsampling operation usually uses methods such as bilinear interpolation or nearest neighbor interpolation to convert the low-resolution feature map F low Zoom in to the high-resolution feature map F high Same size H high ×W high .

[0104] Feature fusion is achieved by element-by-element addition:

[0105] F fuse =F up +F high

[0106] The fused feature map F fuse Continue to maintain high resolution high ×W high , but the number of channels is C high .

[0107] In order to reduce the aliasing effect introduced during upsampling, a 1×1 convolution layer is usually applied to the fused feature map for smoothing. The convolution operation of the 1×1 convolution layer can be expressed as:

[0108]

[0109] (F smooth ) i,j,k K is the value of the smoothed feature map at position (i, j) and channel k. 1x1 (1,1,c,k) is the weight of the 1x1 convolution kernel at position (1,1), input channel c, and output channel k. (F fuse ) i,j,c is the value of the fused feature map at position (i, j) and channel c. k is the bias term on output channel k.

[0110] Through the above formula, the 1×1 convolution layer not only recombines the channels in the feature map, but also smoothes the feature map, reduces the jagged edges and other distortions introduced during upsampling, and improves the quality of the feature map.

[0111] 205. Utilize a preset adaptive threshold adjustment mechanism to perform boundary optimization processing on the target image region to obtain a preliminary image segmentation result;

[0112] It should be noted that step 205 and Figure 1 The content of step 103 shown is similar and will not be described here to avoid repetition. For details, please refer to the content of the aforementioned step 103.

[0113] In one feasible implementation, the adaptive threshold adjustment mechanism includes an adaptive segmentation algorithm. Step 205 includes: using the adaptive segmentation algorithm and the target image area to perform boundary recognition to obtain a preliminary image segmentation result. Exemplarily, in order to design an adaptive segmentation algorithm with a highly complex dynamic adaptive threshold function, we can introduce Gaussian kernel functions at multiple levels to enhance the nonlinear relationship between features and the expressive power of the model.

[0114] Multiscale Gaussian kernel function:

[0115]

[0116] Among them, F context (x,y) i is the eigenvector of the i-th scale, σ i is the standard deviation of the Gaussian kernel at the i-th scale.

[0117] Multimodal Gaussian kernel function:

[0118]

[0119] Among them, F i (x,y) is the i-th feature, σ i is the Gaussian kernel standard deviation of the ith feature.

[0120] Dynamically adjust threshold:

[0121]

[0122] Among them, T 0 (x,y) is the initial threshold, α is the adjustment coefficient, F i (x,y ′ ) is the eigenvector in the neighborhood.

[0123] The final dynamic adaptive threshold function can be expressed as:

[0124]

[0125] Where: F(x,y) is the multimodal feature vector, P is the prior knowledge, is a conditional random field model, A is an attention module, C is a context-aware module, and F fuse (x,y) is the fused feature vector, and N is the deep neural network model.

[0126] Specifically, for the identified small-scale components, an adaptive segmentation algorithm is used to determine their precise boundaries. This algorithm takes into account the shape and size characteristics of small-scale components and can accurately extract the contours of small-scale components against a complex background; that is, for small-scale components that have been successfully identified, an adaptive segmentation algorithm will be used to accurately delineate their boundaries. This method takes into account the actual shape and size of the components and can effectively extract the contour information of the components even in very complex backgrounds. The design of the adaptive segmentation algorithm aims to address the limitations of traditional fixed parameter segmentation methods when faced with components of different sizes and shapes, and to provide a more flexible and accurate solution.

[0127] Among them, an adaptive threshold adjustment mechanism is applied to optimize the definition of small-scale component boundaries and improve the recognition accuracy of component edges. This mechanism dynamically adjusts the threshold according to the grayscale change rate of component edges to ensure the accuracy of edges; this mechanism dynamically adjusts the threshold according to the grayscale change rate of component edges to ensure the accuracy of edges. By analyzing the grayscale change rate of each pixel, the adaptive threshold adjustment mechanism can automatically determine the optimal threshold, so that the boundaries of small-scale components can still be accurately identified and extracted under different lighting conditions and background complexity. This step significantly improves the robustness and accuracy of edge recognition.

[0128] In summary, the present invention effectively improves the recognition accuracy and robustness of small-scale components of power transmission lines through high-resolution image acquisition, image stabilization and dynamic range expansion, multi-scale analysis, local contrast enhancement, machine learning recognition and adaptive segmentation, reduces the interference of environmental factors, ensures the accuracy and smoothness of component boundary extraction, and finally generates segmentation results that not only mark the specific location of the component, but also provide detailed attribute information, significantly improving the efficiency and reliability of fault diagnosis and health assessment. Specifically, the image of the power transmission line and its components is collected by high-resolution imaging equipment to ensure the clarity and detail of small-scale components; then the local contrast enhancement technology is applied to effectively improve the contrast between small-scale components and the background, and improve the efficiency and accuracy of subsequent processing; then the threshold is dynamically adjusted according to the grayscale change rate of the component edge through the adaptive threshold adjustment mechanism, the definition of the small-scale component boundary is optimized, and the accuracy and robustness of edge recognition are significantly improved. These steps work together to significantly improve the image quality and the accuracy of component recognition, providing a reliable basis for subsequent fault diagnosis and health assessment.

[0129] 206. Using the pre-collected prior knowledge and context information of the small-scale components, the preliminary image segmentation result is corrected to obtain a corrected image segmentation result;

[0130] It should be noted that step 206 and Figure 1The content of step 104 shown is similar and will not be described here to avoid repetition. For details, please refer to the content of the aforementioned step 104.

[0131] In a feasible implementation, the prior knowledge includes the shape standard, size standard and color standard of the small-scale components, and the context information includes the relative position standard and spatial layout standard of the small-scale components in the image. Then, the preliminary image segmentation result is corrected by using the pre-collected prior knowledge and context information of the small-scale components to obtain the corrected image segmentation result, including:

[0132] Determine a current feature set of the small-scale components in the preliminary image segmentation result, wherein the current feature set includes a current shape, a current size, a current color, a current relative position, and a current spatial layout of the small-scale components; and correct the preliminary image segmentation result based on the current feature set, prior knowledge, and context information to obtain a corrected image segmentation result.

[0133] It should be noted that the results of the preliminary segmentation are corrected by using the prior knowledge and context information of small-scale components to reduce the influence of noise and erroneous segmentation; specifically, the results of the preliminary segmentation are corrected by using the prior knowledge and context information of the components to reduce the influence of noise and erroneous segmentation. Prior knowledge includes features such as the shape, size, and color of the components, and context information includes the relative position and spatial layout of the components in the image. With this information, the noise and errors in the preliminary segmentation results can be corrected, such as removing edges that do not conform to the prior knowledge, filling in reasonable missing parts, and ensuring the accuracy and reliability of the segmentation results. This step further improves the quality of the segmentation results and provides a solid foundation for subsequent fault diagnosis and health assessment.

[0134] In summary, by further using the prior knowledge and context information of the components to correct the initial segmentation results, the noise impact and erroneous segmentation are reduced, and the accuracy and reliability of the segmentation results are further improved. These steps work together to significantly improve the image quality and component recognition accuracy, providing a solid foundation for subsequent fault diagnosis and health assessment.

[0135] 207. Perform morphological operation processing on the corrected image segmentation result to obtain an optimized image segmentation result;

[0136] It should be noted that step 207 and Figure 1 The content of step 105 is similar and will not be described here to avoid repetition. For details, please refer to the content of the aforementioned step 105.

[0137] In a feasible implementation, the morphological operation is performed on the corrected image segmentation result to obtain an optimized image segmentation result, including: performing dilation, erosion, opening and closing operations on the corrected image segmentation result to obtain an optimized image segmentation result, and the morphological operation at least includes the dilation, erosion, opening and closing operations.

[0138] It is understandable that morphological operations are performed as post-processing means to improve the quality of the segmentation results. These operations may involve expansion, erosion, opening and closing operations, etc., aiming to make the segmented component contours smoother and more continuous. The expansion operation can connect broken edges, the erosion operation can remove burrs and small protrusions on the edges, the opening operation can remove fine structures and noise, and the closing operation can fill in fine cracks and holes on the edges. These operations help to further optimize the segmentation results and make them smoother and more continuous.

[0139] In a feasible implementation, adaptive morphological operations are introduced to dynamically adjust the operation parameters according to the geometric characteristics of different components and the complexity of the background, so as to better preserve the details and boundary integrity of the components. Combined with multi-scale morphological analysis, the segmentation results at different scales are further optimized, ensuring that a high level of segmentation accuracy can be maintained under complex backgrounds and variable lighting conditions.

[0140] Combined with multi-scale morphological analysis, by constructing a multi-scale morphological pyramid, the morphological operation parameters at different scales are dynamically adjusted to adapt to the geometric characteristics and background complexity of different components. Specifically, the input image is first decomposed at multiple scales by using a Gaussian pyramid to generate feature maps of different resolutions. Then, adaptive dilation and erosion operations are applied to the feature maps of each scale, respectively, where the sizes of the dilation kernel and the erosion kernel are adaptively adjusted according to the complexity of the local features. Next, the segmentation results are further optimized by multi-scale opening and closing operations. The opening operation is used to remove fine noise and discontinuous edges, and the closing operation is used to fill fine cracks and holes, thereby ensuring the smoothness and continuity of the component contour. Finally, through multi-scale feature fusion, the optimization results at different scales are integrated to generate the final segmentation result. This multi-scale morphological analysis method not only significantly improves the quality and robustness of the segmentation results, but also demonstrates excellent performance under complex backgrounds and variable lighting conditions, providing a reliable basis for subsequent fault diagnosis and health assessment.

[0141] 208. Reproject the optimized image segmentation result into the original image coordinate system to generate the final image segmentation result of the small-scale component.

[0142] It should be noted that step 208 and Figure 1The content of step 106 shown is similar and will not be described here to avoid repetition. For details, please refer to the content of the aforementioned step 106.

[0143] Finally, the optimized segmentation results are reprojected into the original image coordinate system to generate the final small-scale component image segmentation results. This result not only clearly marks the specific location of each component, but also provides detailed attribute information, such as component type, size, shape, etc., which provides an important basis for subsequent fault diagnosis and health status evaluation. By reprojecting the segmentation results into the original image coordinate system, the segmentation results are aligned with the original image, which is convenient for further analysis and application.

[0144] The above steps constitute a complete image processing process, starting from high-resolution image acquisition, through local contrast enhancement, adaptive threshold adjustment, correction based on prior knowledge and context information, post-processing of morphological operations, until the final re-projection of the segmentation results, each step is aimed at improving the accuracy and reliability of small-scale component recognition. Through the execution of this set of processes, the recognition effect of small-scale components in transmission lines and their components can be effectively improved, and false detection and missed detection can be reduced, thereby providing more accurate data support for the maintenance and management of power systems. In addition, the optimized segmentation results not only clarify the specific location of the components, but also provide detailed attribute information, which is of great significance for in-depth understanding of the working status of the components, timely detection of potential faults, and reasonable arrangement of maintenance plans. Ultimately, the application of this series of technical means will help to ensure the safe and stable operation of the power system, reduce power outages caused by component failures, and improve the quality of power services.

[0145] The present invention provides an image segmentation method for small-scale components, which is a small-scale component image segmentation method based on the characteristics of transmission line inspection imaging. The method first collects images of transmission lines and their components through high-resolution imaging equipment to ensure the clarity and detail of the image. Then, the local contrast enhancement technology is applied to improve the contrast between the small-scale components and the background, thereby improving the efficiency and accuracy of subsequent processing. Then, an adaptive threshold adjustment mechanism is used to optimize the definition of the small-scale component boundary, and the threshold is dynamically adjusted according to the grayscale change rate of the component edge to ensure the accuracy of the edge. In addition, the result of the preliminary segmentation is corrected using the prior knowledge and context information of the component to reduce the influence of noise and the situation of erroneous segmentation. Finally, the quality of the segmentation result is improved by performing morphological operations, so that the contours of the segmented components are smoother and continuous, and the optimized segmentation results are re-projected into the original image coordinate system to generate the final small-scale component image segmentation results, which provides an important basis for subsequent fault diagnosis and health status evaluation.

[0146] See also Figure 3 , Figure 3FIG. 1 is a structural block diagram of an image segmentation device for a small-scale component in an embodiment of the present invention. Figure 3 The device shown comprises:

[0147] Image acquisition module 301: used to acquire an image to be segmented, wherein the image to be segmented is an inspection image of a power transmission line, and the inspection image contains small-scale components to be segmented;

[0148] The region positioning module 302 is used to identify and locate the small-scale components in the image to be segmented by using a multi-scale analysis technique, and obtain a target image region of the small-scale components;

[0149] Boundary optimization module 303: used to perform boundary optimization processing on the target image area by using a preset adaptive threshold adjustment mechanism to obtain a preliminary image segmentation result;

[0150] Result correction module 304: used to correct the preliminary image segmentation result by using the prior knowledge and context information of the small-scale components collected in advance, so as to obtain a corrected image segmentation result;

[0151] Result optimization module 305: used to perform morphological operation processing on the corrected image segmentation result to obtain an optimized image segmentation result;

[0152] Reprojection module 306: used to reproject the optimized image segmentation result into the original image coordinate system to generate the final image segmentation result of the small-scale component.

[0153] It should be noted that Figure 3 The functions of each module in the device shown are Figure 1 The contents of each step in the method shown are similar. To avoid repetition, they are not described here. For details, please refer to Figure 1 The content of each step in the method shown.

[0154] The present invention provides an image segmentation device for small-scale components, and the device comprises: an image acquisition module: used for acquiring an image to be segmented, wherein the image to be segmented is an inspection image of a power transmission line, and the inspection image contains small-scale components to be segmented; a region positioning module: used for identifying and positioning small-scale components in the image to be segmented by using a multi-scale analysis technology, and obtaining a target image region of the small-scale components; a boundary optimization module: used for performing boundary optimization processing on the target image region by using a preset adaptive threshold adjustment mechanism, and obtaining a preliminary image segmentation result; a result correction module: used for correcting the preliminary image segmentation result by using pre-collected prior knowledge and context information of the small-scale components, and obtaining a corrected image segmentation result; a result optimization module: used for performing morphological operation processing on the corrected image segmentation result, and obtaining an optimized image segmentation result; and a reprojection module: used for reprojecting the optimized image segmentation result into an original image coordinate system, and generating a final image segmentation result of the small-scale components. Through the above-mentioned device, image segmentation processing using multi-scale analysis technology, adaptive threshold adjustment mechanism, prior knowledge of small-scale components and contextual information is realized, which is more conducive to the image segmentation of small-scale components, improves the accuracy and reliability of image segmentation of small-scale components, and improves the segmentation effect. Finally, the optimized image segmentation result is re-projected into the original image coordinate system to generate the final image segmentation result of the small-scale component, which provides an important basis for subsequent fault diagnosis and health status evaluation.

[0155] Figure 4 FIG. 1 shows an internal structure diagram of a computer device in an embodiment. The computer device may be a terminal or a server. Figure 4 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. Those skilled in the art can understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0156] In one embodiment, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps: Figure 1 or Figure 2Steps shown.

[0157] In one embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the processor performs the following steps: Figure 1 or Figure 2 Steps shown.

[0158] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0159] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for image segmentation of small-scale components, characterized in that: The method comprises: Acquire an image to be segmented, where the image to be segmented is an inspection image of a power transmission line, and there are small-scale components to be segmented in the inspection image; Using multi-scale analysis technology to identify and locate small-scale components in the image to be segmented, and obtaining a target image area of ​​the small-scale components; Utilizing a preset adaptive threshold adjustment mechanism to perform boundary optimization processing on the target image region to obtain a preliminary image segmentation result; Using the prior knowledge and context information of the small-scale components collected in advance, the preliminary image segmentation result is corrected to obtain a corrected image segmentation result; Performing morphological operation processing on the corrected image segmentation result to obtain an optimized image segmentation result; The optimized image segmentation result is reprojected into the original image coordinate system to generate the final image segmentation result of the small-scale component.

2. The method according to claim 1, characterized in that: The method of using a multi-scale analysis technique to identify and locate small-scale components in the image to be segmented to obtain a target image region of the small-scale components also includes: Defuzzifying the image to be segmented by using image stabilization technology to obtain a processed image to be segmented; The detail expression in the processed image to be segmented is enhanced based on the dynamic range expansion technology to obtain an enhanced image to be segmented.

3. The method according to claim 1, characterized in that: The method of using a multi-scale analysis technique to identify and locate small-scale components in the image to be segmented to obtain a target image region of the small-scale components includes: Using multi-scale analysis technology, the image to be segmented is decomposed at multiple levels to obtain a multi-scale decomposition result; In the multi-scale space, a local contrast enhancement technique is used to highlight the difference between the small-scale components and the background on the multi-scale decomposition result, so as to obtain a contrast-enhanced multi-scale decomposition result; The multi-scale decomposition result is processed by using a pre-trained machine learning algorithm to identify small-scale components, so as to obtain a target image region of the small-scale components. The machine learning model is trained using a number of image samples of small-scale components.

4. The method according to claim 3, characterized in that: The machine learning algorithm includes a convolutional neural network, and the convolution operation of the convolutional neural network can be expressed as: In the formula, (F smooth ) i,j,k is the value of the smoothed feature map at position (i, j) and channel k; K 1x1 (1,1,c,k) is the weight of the 1x1 convolution kernel at position (1,1), input channel c, and output channel k; (F fuse ) i,j,c is the value of the fused feature map at position (i, j) and channel c; b k is the bias term on output channel k; F fuse =F up +F high In the formula, F up is the upsampled feature map, F high It is a high-resolution feature map; The upsampling operation can be expressed as: F up =Upsample(F low ) In the formula, F low is a low-resolution feature map, and the upsampling operation is used to convert the low-resolution feature map F low Zoom in to the high-resolution feature map F high Same size H high ×W high .

5. The method according to claim 1, characterized in that: The method of using a preset adaptive threshold adjustment mechanism to perform boundary optimization processing on the target image region to obtain a preliminary image segmentation result includes: An adaptive segmentation algorithm and the target image region are used to perform boundary recognition to obtain a preliminary image segmentation result, and the adaptive threshold adjustment mechanism includes an adaptive segmentation algorithm.

6. The method according to claim 5, characterized in that: The adaptive segmentation algorithm includes: Multiscale Gaussian kernel function: In the formula, F context (x,y) i is the eigenvector of the i-th scale, σ i is the standard deviation of the Gaussian kernel at the i-th scale; Multimodal Gaussian kernel function: F i (x,y) is the i-th feature, σ i is the Gaussian kernel standard deviation of the i-th feature; Dynamically adjust the threshold: T0(x,y) is the initial threshold, α is the adjustment coefficient, F i (x,y ′ ) is the eigenvector in the neighborhood; The final dynamic adaptive threshold function can be expressed as: in: F(x,y) is the multimodal feature vector, P is the prior knowledge, is a conditional random field model, A is an attention module; C is a context-aware module, and F fuse (x,y) is the fused feature vector, and N is the deep neural network model.

7. The method according to claim 1, characterized in that: The prior knowledge includes the shape standard, size standard and color standard of the small-scale components, and the context information includes the relative position standard and spatial layout standard of the small-scale components in the image. Then, the preliminary image segmentation result is corrected by using the pre-collected prior knowledge and context information of the small-scale components to obtain the corrected image segmentation result, including: Determine a current feature set of the small-scale components in the preliminary image segmentation result, wherein the current feature set includes a current shape, a current size and a current color, a current relative position and a current spatial layout of the small-scale components; The preliminary image segmentation result is corrected based on the current feature set, prior knowledge and context information to obtain a corrected image segmentation result.

8. An image segmentation device for small-scale components, characterized in that: The device comprises: Image acquisition module: used to acquire the image to be segmented, wherein the image to be segmented is an inspection image of the power transmission line, and the inspection image contains small-scale components to be segmented; Region positioning module: used to identify and locate small-scale components in the image to be segmented by using multi-scale analysis technology to obtain the target image region of the small-scale components; Boundary optimization module: used to perform boundary optimization processing on the target image area by using a preset adaptive threshold adjustment mechanism to obtain a preliminary image segmentation result; Result correction module: used to correct the preliminary image segmentation result by using the prior knowledge and context information of small-scale components collected in advance, and obtain the corrected image segmentation result; Result optimization module: used for performing morphological operation on the corrected image segmentation result to obtain an optimized image segmentation result; Reprojection module: used to reproject the optimized image segmentation results into the original image coordinate system to generate the final image segmentation results of small-scale components.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

10. A computer device comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

Images