Insulation string image detection method and system fusing SAM model and shallow knowledge constraint
By integrating the SAM model with shallow knowledge constraints, combined with unsupervised key point detection and local texture features, the existing insulator string detection methods are solved, and more accurate image segmentation and detection effects are achieved.
Patent Information
- Application Number
- CN202411290572.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2025-02-14
AI Technical Summary
The existing insulator string detection methods rely on manual construction features, the model generalization performance is insufficient, and a large amount of manual labeling of data is required, and the accuracy and efficiency are not high.
Using a method of fusing SAM model and shallow knowledge constraints, unsupervised key point detection and point prompting, combined with local texture features and structural features for image matching and positioning, and through RGB to HSV color model, adaptive Gaussian blur and multi-scale morphology Learn to reconstruct and perform fine segmentation.
The accuracy and efficiency of insulator string detection are improved, the robustness of matching results is enhanced, background interference is reduced, and the image boundary shape is optimized, making the segmentation result more accurate.
Smart Images
Figure CN119445062A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to image detection means, belonging to the technical field of image information processing, and in particular to an insulation string image detection method and system integrating a SAM model with shallow knowledge constraints. Background Art
[0002] Insulator strings are mainly used for electrical insulation and mechanical support. They are necessary components in the process of erecting transmission lines. Therefore, the detection and quantity statistics of insulators play an important role in the construction of power systems. At present, the detection operation of substation insulator strings mainly adopts manual on-site operations. This operation method is not only dangerous, labor-intensive, and inefficient, but also difficult to replace. Therefore, the development of a method for automatically detecting insulator strings has become an actual urgent need for the power system.
[0003] Traditional insulator string detection methods are developed from image processing methods in computer vision, including filtering, Gaussian blurring, and other methods. With the development of machine learning, methods based on support vector machines (SVM), random forests (RF), and decision trees (DT) have been used in insulator string detection and practical application needs, and have achieved good results. However, traditional machine learning relies heavily on manually constructed features, the model generalization performance is insufficient, and a large amount of manually labeled data is required to obtain a robust model, with low accuracy and efficiency. Summary of the invention
[0004] The purpose of the present invention is to overcome the above-mentioned defects and problems existing in the prior art and to provide an insulation string image detection method and system which integrates the SAM model and shallow knowledge constraints with high accuracy and efficiency.
[0005] To achieve the above objectives, the technical solution of the present invention is: an insulation string image detection method integrating SAM model and shallow knowledge constraints, comprising:
[0006] S1, obtaining the original image data of the insulator string and performing preprocessing;
[0007] S2. Based on the improved SAM large model, the image pixel points of the original image data are selected in an unsupervised key point guided point prompting method to obtain the rough extraction result of the insulator string image;
[0008] S3, based on the improved standard correlation matching algorithm, the rough extraction result and the original image data are matched and positioned by taking into account the local texture features and structural features, and the absolute coordinates of the rough extraction result in the original image data are obtained;
[0009] S4. Set relevant shallow knowledge and rules and execute them sequentially on the image data of absolute coordinates to obtain the final fine segmentation result of the insulator string image; the relevant shallow knowledge and rules include:
[0010] An RGB to HSV shallow color model is constructed, and the color features of the image data are abstracted through the shallow color model to obtain the actual range of the original image data. The Gaussian noise in the image data is smoothed based on adaptive Gaussian blur, and the noise points at the image boundary are removed. Based on the multi-scale morphological reconstruction method, the holes inside the image data are filled, and the bounding box is drawn and statistically analyzed to obtain the final finely segmented insulator string image detection result.
[0011] In step S2, the expression of the rough extraction result is as follows:
[0012]
[0013] Where: I c is the rough extraction result, IE is the image encoder, is a feature fusion operation, KP is unsupervised key point detection, PP is key point guided point prompting, and MC is refined mask segmentation and cropping.
[0014] In step S2, unsupervised key point detection specifically includes:
[0015] S21, using the image encoder IE to encode the input image data of the insulator string to generate a multi-scale feature map; the expression of the multi-scale feature map is as follows:
[0016] F = IE(I);
[0017] Where: F is a multi-scale feature map, I is the input original image data;
[0018] S22. In the multi-scale feature map F, the similarity between the pixels in the original image data is calculated by autocorrelation analysis, and an autocorrelation matrix is constructed; the expression of the autocorrelation matrix is as follows:
[0019] S = sim(Fi, Fj);
[0020] Where: S is the autocorrelation matrix, sim is the similarity measurement function, Fi and Fj represent the i-th and j-th points in the multi-scale feature map respectively;
[0021] S23, based on the density clustering algorithm, the regions with high autocorrelation in the original image data are identified in the autocorrelation matrix S to obtain potential key point positions; the potential key point position set is as follows:
[0022] KP = DBSCAN(S);
[0023] Among them: KP is the set of identified potential key point positions, and DBSCAN is a density clustering algorithm.
[0024] The step S3 specifically includes:
[0025] The local texture features of the rough extraction results are obtained based on the local binary pattern; the structural features of the rough extraction results are obtained based on the directional gradient histogram; the standard correlation matching algorithm is improved by combining the local texture features and structural features of the rough extraction results and the image grayscale value of the original image data, and the rough extraction results are matched with each sub-image in the original image data, and the correlation coefficient R(i, c) is calculated to obtain the rough extraction result I c In the original image data I i The absolute coordinates I p ;
[0026] The expression of the correlation coefficient is as follows:
[0027]
[0028] Where: I i (x, y) is subgraph I i The gray value of the pixel (x, y) in Subgraph I i The average gray value of LBP(I c (x, y)) is the rough extraction result I c The local texture feature at the pixel point (x, y), HOG (I c (x, y)) is the rough extraction result I c The structural features at the pixel point (x, y), The rough extraction result I c Average texture and structural features of
[0029] The correlation coefficient R(i, c) is limited to between -1 and 1, indicating the absolute coordinates of the rough extraction result in the original image data; wherein: 1 indicates that the grayscale value, local texture features and structural features between the images are exactly the same, -1 indicates that the grayscale value, local texture features and structural features between the images are exactly opposite, and 0 indicates that there is no obvious relationship between the grayscale value, local texture features and structural features between the images.
[0030] The expression of the local binary pattern is as follows:
[0031]
[0032] Where: LBP(x, y) is the LBP value of the center pixel coordinates, I(x f ,y f) is the gray value of the i-th pixel around the central pixel.
[0033] The expression of the directional gradient histogram is as follows:
[0034] G x =I(x+1,y)-I(x-1,y);
[0035] G y =I(x,y+1)-I(x,y-1);
[0036]
[0037] Where: G x , G y are the horizontal gradient and vertical gradient respectively, G is the gradient amplitude value, and θ is the direction value.
[0038] In step S4, the expression of converting RGB to HSV shallow color model is as follows:
[0039]
[0040] v = max;
[0041] Among them: (r, g, b) are the red, green, and blue coordinates of a color respectively, max is the maximum value among (r, g, b), min is the minimum value among (r, g, b), and (h, s, v) is the HSV feature.
[0042] In step S4, the expression of adaptive Gaussian blur is as follows:
[0043]
[0044] Where: G(x, y) is the Gaussian kernel, x, y are the coordinate values of the pixels in the image data, σ is the standard deviation, k is the size of the Gaussian kernel, I blur (x, y) is the coordinate value of the pixel after Gaussian blur.
[0045] In step S4, the expression of multi-scale morphological reconstruction is as follows:
[0046]
[0047] in: Indicates the use of structural feature B i The expansion operation performed, F c is the complement of the foreground pixels.
[0048] The preprocessing in step S1 includes: performing any one or any combination of illumination adjustment, color dithering, contrast adjustment, random erasure, and lossless compression on the original image data.
[0049] An insulation string image detection system integrating a SAM model and shallow knowledge constraints, the system is applied to the above method, and the system comprises:
[0050] The original image data acquisition module is used to acquire the original image data of the insulator string and perform preprocessing;
[0051] A coarse extraction module is used to select image pixels of the original image data in a point prompting manner guided by unsupervised key points based on an improved SAM large model to obtain a coarse extraction result of the insulator string image;
[0052] A matching and positioning module is used to match and position the rough extraction result with the original image data based on an improved standard correlation matching algorithm, taking into account local texture features and structural features, and obtain the absolute coordinates of the rough extraction result in the original image data;
[0053] The fine segmentation module is used to set relevant shallow knowledge and rules and execute them on the image data of absolute coordinates in sequence to obtain the final fine segmentation result of the insulator string image; the relevant shallow knowledge and rules include:
[0054] An RGB to HSV shallow color model is constructed, and the color features of the image data are abstracted through the shallow color model to obtain the actual range of the original image data. The Gaussian noise in the image data is smoothed based on adaptive Gaussian blur, and the noise points at the image boundary are removed. Based on the multi-scale morphological reconstruction method, the holes inside the image data are filled, and the bounding box is drawn and statistically analyzed to obtain the final finely segmented insulator string image detection result.
[0055] An insulation string image detection device integrating a SAM model and shallow knowledge constraints, the device comprising a processor and a memory;
[0056] The memory is used to store computer program code and transmit the computer program code to the processor;
[0057] The processor is used to execute the above-mentioned insulation string image detection method integrating the SAM model and shallow knowledge constraints according to the instructions in the computer program code.
[0058] Compared with the prior art, the present invention has the following beneficial effects:
[0059] The invention discloses an insulating string image detection method and system integrating a SAM model with shallow knowledge constraints. The method first obtains original image data and pre-processes it; then, based on an improved SAM model, point selection is performed in a point prompting manner guided by unsupervised key points to obtain a rough extraction result; then, based on an improved standard correlation matching algorithm, matching positioning is performed taking into account local texture features and structural features to obtain absolute coordinates; finally, steps such as color mode conversion, Gaussian blurring, and multi-scale morphological reconstruction are set to be executed sequentially on the image data of the absolute coordinates to obtain a finely segmented image result; in application, the design automatically discovers points with structural significance in the image by means of unsupervised key point detection, provides an important guiding role for subsequent point prompts, and combines local texture features and structural features to make it suitable for the task of accurately matching image details in complex scenes, thereby enhancing the robustness of the matching result; finally, through steps such as relevant shallow knowledge and rules, background interference is effectively reduced, the image boundary shape is more comprehensively optimized, and the segmentation result is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Figure 1 It is a flow chart of the method steps of the present invention.
[0061] Figure 2 It is a schematic diagram of the crude extraction result of the present invention.
[0062] Figure 3 It is a schematic diagram of the matching positioning result of the present invention.
[0063] Figure 4 It is a schematic diagram of the insulator string segmentation result of the present invention.
[0064] Figure 5 It is a schematic diagram of the final insulator string detection result of the present invention.
[0065] Figure 6 It is a schematic diagram of the system structure of the present invention.
[0066] Figure 7 It is a schematic diagram of the device structure of the present invention.
[0067] In the figure: original image data acquisition module 1, coarse extraction module 2, matching and positioning module 3, fine segmentation module 4, processor 5, memory 6, computer program code 61. DETAILED DESCRIPTION
[0068] The present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0069] Embodiment 1:
[0070] See also Figure 1 , an insulation string image detection method integrating SAM model and shallow knowledge constraints, including:
[0071] S1, obtaining the original image data of the insulator string and performing preprocessing;
[0072] Furthermore, the preprocessing includes: performing any one or any combination of illumination adjustment, color jitter, contrast adjustment, random erasure, and lossless compression on the original image data to ensure that the sample quality is richer, while reducing the image storage space and making the algorithm loading more efficient and convenient.
[0073] S2. Based on the improved SAM large model, the image pixel points of the original image data are selected in an unsupervised key point guided point prompting method to obtain the rough extraction result of the insulator string image;
[0074] Further, the expression of the rough extraction result is as follows:
[0075]
[0076] Where: I c is the rough extraction result; IE is the image encoder, which is responsible for encoding the input raw image data into feature representation. The parameters of this part of the encoding selection are frozen and not retrained; is a feature fusion operation; KP is an unsupervised key point detection, which automatically detects the key points of power towers and insulator strings through unsupervised learning and provides preliminary spatial prompts; PP is a point prompt guided by key points, which combines the detected key points as point prompts to guide the model to accurately locate the target object; MC is a refined mask segmentation and cropping, which performs fine segmentation and cropping of power towers and insulator strings based on more accurate prompt information to obtain more accurate extraction results, such as Figure 2 shown.
[0077] Further, unsupervised key point detection includes:
[0078] S21, using the image encoder IE to encode the input image data of the insulator string to generate a multi-scale feature map; the expression of the multi-scale feature map is as follows:
[0079] F = IE(I);
[0080] Where: F is a multi-scale feature map, I is the input original image data;
[0081] S22. In the multi-scale feature map F, the similarity between the pixels in the original image data is calculated by autocorrelation analysis, and an autocorrelation matrix is constructed; the expression of the autocorrelation matrix is as follows:
[0082] S = sim(Fi, Fj);
[0083] Where: S is the autocorrelation matrix, sim is the similarity metric function (preferably cosine similarity), Fi and Fj represent the i-th and j-th points in the multi-scale feature map respectively;
[0084] S23, based on the density clustering algorithm, the regions with high autocorrelation in the original image data are identified in the autocorrelation matrix S to obtain potential key point positions; the potential key point position set is as follows:
[0085] KP = DBSCAN(S);
[0086] Among them: KP is the set of identified potential key point positions, and DBSCAN is a density clustering algorithm.
[0087] The key to unsupervised key point detection is to automatically discover points with structural significance in the image, such as the top or connection points of power towers, through techniques such as autocorrelation analysis and density clustering. These points play an important role in the subsequent point prompting process, further improving the model's segmentation accuracy for power towers and insulator strings.
[0088] S3, based on the improved standard correlation matching algorithm, the rough extraction result and the original image data are matched and positioned by taking into account the local texture features and structural features, and the absolute coordinates of the rough extraction result in the original image data are obtained;
[0089] Since the crude extraction results c There is no coordinate information of the original image data in the image, that is, it cannot match the original input image. Therefore, the rough extraction result at this time is not enough to support the detection of the insulator string. In order to obtain the rough extraction result I c At the precise location of the original image data, this scheme proposes to use the similarity matching method and the improved standard correlation matching algorithm that takes into account texture and structural features to carry out matching and absolute positioning of the rough extraction results.
[0090] Specifically, the improved standard correlation matching algorithm is used to convert the rough extraction result I c Each sub-image I of the original image data I i (i represents the serial number of the sub-image of the original image data) for matching.
[0091] In calculating each subgraph I i with I c When the standardized correlation coefficient is calculated, the local texture features and structural features are combined to enhance the robustness of the matching. After successful matching, the rough extraction result I is finally obtained. c At the absolute coordinate I of the original image data I p ;
[0092] The expression of the standardized correlation coefficient is as follows:
[0093]
[0094] Where: I i (x,y) is subgraph I i The gray value of the pixel (x, y) in Subgraph I i The average gray value, T(I c (x, y)) is the rough extraction result I c Texture and structural features at the pixel point (x, y), The rough extraction result I c Average texture and structural features.
[0095] This formula not only considers the correlation of image grayscale values, but also introduces texture features T(I c ), making the matching results more accurate and robust. The calculated correlation coefficient R(i, c) is limited to between -1 and 1, indicating the absolute coordinates of the rough extraction result in the original image data; where: 1 means that the grayscale value, local texture features and structural features between the images are exactly the same, -1 means that the grayscale value, local texture features and structural features between the images are exactly the opposite, and 0 means that there is no obvious relationship between the grayscale value, local texture features and structural features between the images; this method is particularly suitable for tasks that require accurate matching of image details in complex scenes. By introducing texture and structural information, the improved standard correlation matching algorithm can more accurately locate I c The position in the original image data I provides more reliable support for subsequent insulator string detection.
[0096] Furthermore, in this scheme, local texture features can be extracted through the local binary pattern (LBP) of the image. The local binary pattern can describe the texture information of the pixel neighborhood in the image; for each pixel in the image, its gray value is compared with the surrounding pixels. If the gray value of the surrounding pixels is greater than the central pixel, it is marked as 1, otherwise it is marked as 0. These binary values are combined into a binary number, which is the LBP value of the pixel.
[0097] The expression of the local binary pattern is as follows:
[0098]
[0099] Where: LBP(x, y) is the LBP value of the center pixel coordinates; I(x f ,y f ) is the gray value of the i-th pixel around the central pixel; s(x) is the sign function, which is defined as: when x≥0, s(x)=1; when x<0, s(x)=0.
[0100] Furthermore, in this solution, shape features can be extracted through the image's gradient direction histogram (HOG). The HOG feature obtains the shape information of the image by calculating the gradient direction and amplitude of each pixel in the image, and divides the image into small blocks, and obtains the HOG feature by counting the gradient direction of each small block.
[0101] The expression of the directional gradient histogram is as follows:
[0102] G x =I(x+1,y)-I(x-1,y);
[0103] G y =I(x,y+1)-I(x,y-1);
[0104]
[0105] Where: G x , G y are the horizontal gradient and vertical gradient respectively, G is the gradient amplitude value, and θ is the direction value.
[0106] Furthermore, when calculating the correlation coefficient, the local texture feature T(I c ) can be represented by LBP value, and shape feature can be represented by HOG value; then the improved correlation coefficient formula in this scheme can be further expressed as:
[0107]
[0108] Where: I i (x, y) is subgraph I i The gray value of the pixel (x, y) in the Subgraph I i The average gray value of LBP(I c (x, y)) is the rough extraction result I c The local texture feature at the pixel point (x, y), namely the LBP value; HOG (I c (x,y)) is the rough extraction result I c The structural features at the pixel point (x, y), that is, the HOG value; The rough extraction result I c The improved correlation coefficient can enhance the robustness of image matching, especially when dealing with images with complex textures and shapes. Figure 3 shown.
[0109] S4. Set relevant shallow knowledge and rules and execute them sequentially on the image data of absolute coordinates to obtain the final fine segmentation result of the insulator string image; the relevant shallow knowledge and rules include:
[0110] (1) A shallow color model for converting RGB to HSV is constructed. The color features of the image data are abstracted through the shallow color model to obtain the actual range of the original image data. Through this conversion, the color features of the insulator string can be effectively distinguished from the color of the power tower, thereby reducing the impact of background interference on subsequent processing and improving the saliency of the insulator string.
[0111] Furthermore, the expression of R0B to HSV shallow color model is as follows:
[0112]
[0113] v = ma;
[0114] Among them: (r, g, b) are the red, green, and blue coordinates of a color, respectively, and their values are real numbers between 0 and 1; max is the maximum value of (r, g, b); min is the minimum value of (r, g, b); (h, s, v) are HSV features, namely hue, saturation, and brightness; the value of h is usually normalized to between 0 and 360°.
[0115] In order to obtain the absolute coordinates I p After obtaining the HSV features, the relevant value is set according to the HSV range of the insulator string image data to obtain the connected domain of the insulator string, and the threshold is set to remove the noise points that do not belong to the insulator string image data, so as to obtain the actual range of the original image of the insulator string.
[0116] (2) Based on adaptive Gaussian blur, the Gaussian noise in the image data is smoothed and the noise points at the image boundary are removed; the adaptive Gaussian blur adjusts the blur degree according to the local noise intensity, so as to more accurately smooth the Gaussian noise in the image, ensure a more coherent boundary, and improve the overall visual quality of the image.
[0117] The expression of the adaptive Gaussian blur is as follows:
[0118]
[0119] Where: G(x, y) is the Gaussian kernel, x, y are the coordinate values of the pixels in the image data, σ is the standard deviation, k is the size of the Gaussian kernel, I blur (x, y) is the coordinate value of the pixel after Gaussian blur.
[0120] (3) Based on the multi-scale morphological reconstruction method, the holes in the image data are filled, and the bounding box is drawn and statistically analyzed to obtain the final fine segmentation result of the insulator string image, such as Figure 4 shown.
[0121] By processing hole filling and boundary drawing at different scales, the shape of the insulator string can be more comprehensively optimized, the boundary box can be accurately drawn, the recognition and counting accuracy of the insulator string can be improved, and the final insulator string fine target detection results and counting statistics can be output by combining the matching and positioning results, such as Figure 5 shown.
[0122] The expression of the multi-scale morphological reconstruction is as follows:
[0123]
[0124] in: Indicates the use of structural feature B i The expansion operation performed, F c is the complement of the foreground pixels.
[0125] Embodiment 2:
[0126] See also Figure 6 , an insulation string image detection system integrating SAM model and shallow knowledge constraints, including:
[0127] The original image data acquisition module 1 is used to acquire the original image data of the insulator string and perform preprocessing;
[0128] Furthermore, the preprocessing method of the original image data acquisition module 1 includes: performing any one or any combination of illumination adjustment, color dithering, contrast adjustment, random erasure, and lossless compression on the original image data.
[0129] The rough extraction module 2 is used to select the image pixel points of the original image data in a point prompting manner guided by unsupervised key points based on the improved SAM large model to obtain the rough extraction result of the insulator string image;
[0130] Furthermore, the expression for performing rough extraction in the rough extraction module 2 is as follows:
[0131]
[0132] Where: I c is the rough extraction result, IE is the image encoder, is a feature fusion operation, KP is unsupervised key point detection, PP is key point guided point prompting, and MC is refined mask segmentation and cropping.
[0133] The coarse extraction module 2 performs unsupervised key point detection according to the following steps:
[0134] S21, using the image encoder IE to encode the input image data of the insulator string to generate a multi-scale feature map; the expression of the multi-scale feature map is as follows:
[0135] F = IE(I);
[0136] Where: F is a multi-scale feature map, I is the input original image data;
[0137] S22. In the multi-scale feature map F, the similarity between the pixels in the original image data is calculated by autocorrelation analysis, and an autocorrelation matrix is constructed; the expression of the autocorrelation matrix is as follows:
[0138] S = sim(Fi, Fj);
[0139] Where: S is the autocorrelation matrix, sim is the similarity measurement function, Fi and Fj represent the i-th and j-th points in the multi-scale feature map respectively;
[0140] S23, based on the density clustering algorithm, the regions with high autocorrelation in the original image data are identified in the autocorrelation matrix S to obtain potential key point positions; the potential key point position set is as follows:
[0141] KP = DBSCAN(S);
[0142] Among them: KP is the set of identified potential key point positions, and DBSCAN is a density clustering algorithm.
[0143] A matching and positioning module 3 is used to match and position the rough extraction result with the original image data based on an improved standard correlation matching algorithm, taking into account local texture features and structural features, and obtain the absolute coordinates of the rough extraction result in the original image data;
[0144] Furthermore, the matching and positioning module 3 obtains the absolute coordinates in the following manner:
[0145] The local texture features of the rough extraction results are obtained based on the local binary pattern; the structural features of the rough extraction results are obtained based on the directional gradient histogram; the standard correlation matching algorithm is improved by combining the local texture features and structural features of the rough extraction results and the image grayscale value of the original image data, and the rough extraction results are matched with each sub-image in the original image data, and the correlation coefficient R(i, c) is calculated to obtain the rough extraction result I c In the original image data I i The absolute coordinates I D ;
[0146] The expression of the correlation coefficient is as follows:
[0147]
[0148] Where: I i (x, y) is subgraph I iThe gray value of the pixel (x, y) in Subgraph I i The average gray value of LBP(I c (x, y)) is the rough extraction result I c The local texture feature at the pixel point (x, y), HOG (I c (x, y)) is the rough extraction result I c The structural features at the pixel point (x, y), The rough extraction result I c Average texture and structural features of
[0149] The correlation coefficient R(i, c) is limited to between -1 and 1, indicating the absolute coordinates of the rough extraction result in the original image data; wherein: 1 indicates that the grayscale value, local texture features and structural features between the images are exactly the same, -1 indicates that the grayscale value, local texture features and structural features between the images are exactly opposite, and 0 indicates that there is no obvious relationship between the grayscale value, local texture features and structural features between the images.
[0150] The expression of the local binary pattern is as follows:
[0151]
[0152] Where: LBP(x, y) is the LBP value of the center pixel coordinates, I(x i ,y i ) is the gray value of the i-th pixel around the central pixel.
[0153] The expression of the directional gradient histogram is as follows:
[0154] G x =I(x+1,y)-I(x-1,y);
[0155] G y =I(x,y+1)-I(x,y-1);
[0156]
[0157] Where: G x , G y are the horizontal gradient and vertical gradient respectively, G is the gradient amplitude value, and θ is the direction value.
[0158] The fine segmentation module 4 is used to set relevant shallow knowledge and rules and execute them sequentially on the image data of absolute coordinates to obtain the final fine segmentation result of the insulator string image; the relevant shallow knowledge and rules include:
[0159] An RGB to HSV shallow color model is constructed, and the color features of the image data are abstracted through the shallow color model to obtain the actual range of the original image data. The Gaussian noise in the image data is smoothed based on adaptive Gaussian blur, and the noise points at the image boundary are removed. Based on the multi-scale morphological reconstruction method, the holes inside the image data are filled, and the bounding box is drawn and statistically analyzed to obtain the final finely segmented insulator string image detection result.
[0160] Furthermore, the fine segmentation module 4 finely segments the insulator string image result according to the following method:
[0161] The expression of RGB to HSV shallow color model is as follows:
[0162]
[0163] v = max;
[0164] Among them: (r, g, b) are the red, green, and blue coordinates of a color respectively, max is the maximum value among (r, g, b), min is the minimum value among (r, g, b), and (h, s, v) is the HSV feature.
[0165] The expression of the adaptive Gaussian blur is as follows:
[0166]
[0167] Where: G(x, y) is the Gaussian kernel, x, y are the coordinate values of the pixels in the image data, σ is the standard deviation, k is the size of the Gaussian kernel, I blur (x, y) is the coordinate value of the pixel after Gaussian blur.
[0168] The expression of the multi-scale morphological reconstruction is as follows:
[0169]
[0170] in: Indicates the use of structural feature B i The expansion operation performed, F C is the complement of the foreground pixels.
[0171] Embodiment 3:
[0172] See also Figure 7 , an insulation string image detection device integrating a SAM model and shallow knowledge constraints, the device comprising a processor 5 and a memory 6;
[0173] The memory 6 is used to store the computer program code 61 and transmit the computer program code 61 to the processor 5;
[0174] The processor 5 is used to execute the insulation string image detection method integrating the SAM model and shallow knowledge constraints described in Example 1 according to the instructions in the computer program code 61.
[0175] This solution also includes an insulation string image detection storage medium that integrates the SAM model and shallow knowledge constraints, on which a computer program is stored. When the computer program is executed by a processor, it implements the insulation string image detection method that integrates the SAM model and shallow knowledge constraints described in Example 1.
[0176] Generally speaking, the computer instructions for implementing the method of the present invention may be carried in any combination of one or more computer-readable storage media. Non-transitory computer-readable storage media may include any computer-readable media, except for the signal itself that is temporarily propagating.
[0177] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EKROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.
[0178] Computer program code for performing the operation of the present invention can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, SMalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages, in particular, Python suitable for neural network computing and platform frameworks based on TensorFlow, PyTorch, etc. can be used. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer or to an external computer (for example, using an Internet service provider to connect via the Internet) through any type of network, including a local area network (LAN) or a wide area network (WAN).
[0179] The above-mentioned device and non-temporary computer-readable storage medium can refer to the specific description of an insulation string image detection method integrating SAM model and shallow knowledge constraints and its beneficial effects, which will not be repeated here.
[0180] Although the embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. An insulation string image detection method integrating SAM model and shallow knowledge constraints, characterized in that: include: S1, obtaining the original image data of the insulator string and performing preprocessing; S2. Based on the improved SAM large model, the image pixel points of the original image data are selected in an unsupervised key point guided point prompting method to obtain the rough extraction result of the insulator string image; S3, based on the improved standard correlation matching algorithm, the rough extraction result and the original image data are matched and positioned by taking into account the local texture features and structural features, and the absolute coordinates of the rough extraction result in the original image data are obtained; S4, setting relevant shallow knowledge and rules and executing them sequentially on the image data of absolute coordinates to obtain the final fine segmentation result of the insulator string image; The relevant shallow knowledge and rules include: Construct an RGB to HSV shallow color model, abstract the color features of the image data through the shallow color model, and obtain the actual range of the original image data; The Gaussian noise in the image data is smoothed based on adaptive Gaussian blur and the noise points at the image boundary are removed. The holes inside the image data are filled based on the multi-scale morphological reconstruction method, and the bounding box is drawn and statistically analyzed to obtain the final finely segmented insulator string image detection result.
2. The insulation string image detection method integrating the SAM model and shallow knowledge constraints according to claim 1 is characterized in that: In step S2, the expression of the rough extraction result is as follows: Where: I c is the rough extraction result, IE is the image encoder, is a feature fusion operation, KP is unsupervised key point detection, PP is key point guided point prompting, and MC is refined mask segmentation and cropping.
3. The insulation string image detection method integrating the SAM model and shallow knowledge constraints according to claim 2 is characterized in that: In step S2, unsupervised key point detection specifically includes: S21, using the image encoder IE to encode the input image data of the insulator string to generate a multi-scale feature map; the expression of the multi-scale feature map is as follows: F = IE(I); Where: F is a multi-scale feature map, I is the input original image data; S22. In the multi-scale feature map F, the similarity between the pixels in the original image data is calculated by autocorrelation analysis, and an autocorrelation matrix is constructed; the expression of the autocorrelation matrix is as follows: S = sim(Fi, Fj); Where: S is the autocorrelation matrix, sim is the similarity measurement function, Fi and Fj represent the i-th and j-th points in the multi-scale feature map respectively; S23, based on the density clustering algorithm, the regions with high autocorrelation in the original image data are identified in the autocorrelation matrix S to obtain potential key point positions; the potential key point position set is as follows: KP = DBSCAN(S); Among them: KP is the set of identified potential key point positions, and DBSCAN is a density clustering algorithm.
4. The insulation string image detection method integrating the SAM model and shallow knowledge constraints according to claim 1 is characterized in that: The step S3 specifically includes: The local texture features of the rough extraction results are obtained based on the local binary pattern; the structural features of the rough extraction results are obtained based on the directional gradient histogram; the standard correlation matching algorithm is improved by combining the local texture features and structural features of the rough extraction results and the image grayscale value of the original image data, and the rough extraction results are matched with each sub-image in the original image data, and the correlation coefficient R(i, c) is calculated to obtain the rough extraction result I c In the original image data I i The absolute coordinates I p ; The expression of the correlation coefficient is as follows: Where: I i (x, y) is subgraph I i The gray value of the pixel (x, y) in Subgraph I i The average gray value of LBP(I c (x, y)) is the rough extraction result I c The local texture feature at the pixel point (x, y), HOG (I c (x, y)) is the rough extraction result I c The structural features at the pixel point (x, y), The rough extraction result I c Average texture and structural features of The correlation coefficient R(i, c) is limited to between -1 and 1, indicating the absolute coordinates of the rough extraction result in the original image data; wherein: 1 indicates that the grayscale value, local texture features and structural features between the images are exactly the same, -1 indicates that the grayscale value, local texture features and structural features between the images are exactly opposite, and 0 indicates that there is no obvious relationship between the grayscale value, local texture features and structural features between the images.
5. The insulation string image detection method integrating the SAM model and shallow knowledge constraints according to claim 4 is characterized in that: The expression of the local binary pattern is as follows: Where: LBP(x, y) is the LBP value of the center pixel coordinates, I(x i ,y i ) is the gray value of the i-th pixel around the central pixel.
6. The insulation string image detection method integrating the SAM model and shallow knowledge constraints according to claim 4 is characterized in that: The expression of the directional gradient histogram is as follows: G x =I(x+1,y)-I(x-1,y); G y =I(x,y+1)-I(x,y-1); Where: G x , G y are the horizontal gradient and vertical gradient respectively, G is the gradient amplitude value, and θ is the direction value.
7. The insulation string image detection method integrating the SAM model and shallow knowledge constraints according to claim 1 is characterized by: In step S4, the expression of converting RGB to HSV shallow color model is as follows: v = max; Where: (r, g, b) are the red, green, and blue coordinates of a color, respectively; max is the maximum value among (r, g, b); min is the minimum value among (r, g, b); and (h, s, v) are HSV features; The expression of the adaptive Gaussian blur is as follows: Where: G(x, y) is the Gaussian kernel, x, y are the coordinate values of the pixels in the image data, σ is the standard deviation, k is the size of the Gaussian kernel, I blur (x, y) is the coordinate value of the pixel after Gaussian blur; The expression of the multi-scale morphological reconstruction is as follows: in: Indicates the use of structural feature B i The expansion operation performed, F c is the complement of the foreground pixels.
8. The insulation string image detection method integrating the SAM model and shallow knowledge constraints according to claim 1 is characterized in that: The preprocessing in step S1 includes: performing any one or any combination of illumination adjustment, color dithering, contrast adjustment, random erasure, and lossless compression on the original image data.
9. An insulation string image detection system integrating SAM model and shallow knowledge constraints, characterized by: The system is applied to the method described in any one of claims 1 to 8, and the system comprises: The original image data acquisition module (1) is used to acquire the original image data of the insulator string and perform preprocessing; A coarse extraction module (2) is used to select image pixel points of the original image data in a point prompting manner guided by unsupervised key points based on an improved SAM large model, so as to obtain a coarse extraction result of the insulator string image; A matching and positioning module (3) is used to match and position the rough extraction result with the original image data based on an improved standard correlation matching algorithm, taking into account local texture features and structural features, and obtain the absolute coordinates of the rough extraction result in the original image data; The fine segmentation module (4) is used to set relevant shallow knowledge and rules and execute them sequentially on the image data of the absolute coordinates to obtain the final fine segmentation result of the insulator string image; the relevant shallow knowledge and rules include: An RGB to HSV shallow color model is constructed, and the color features of the image data are abstracted through the shallow color model to obtain the actual range of the original image data. The Gaussian noise in the image data is smoothed based on adaptive Gaussian blur, and the noise points at the image boundary are removed. Based on the multi-scale morphological reconstruction method, the holes inside the image data are filled, and the bounding box is drawn and statistically analyzed to obtain the final finely segmented insulator string image detection result.
10. An insulation string image detection device integrating SAM model and shallow knowledge constraints, characterized by: The device comprises a processor (5) and a memory (6); The memory (6) is used to store computer program code (61) and transmit the computer program code (61) to the processor (5); The processor (5) is used to execute the insulation string image detection method integrating the SAM model and shallow knowledge constraints as described in any one of claims 1 to 8 according to the instructions in the computer program code (61).