Semantic segmentation-based spacer installation gap identification and positioning method and system

Through the gap recognition and positioning method of spacer installation based on semantic segmentation, the DeepLab3+ algorithm and feature extraction module are used to solve the problem of spacer gap identification in transmission lines, and high-precision spacer gap identification and positioning are achieved, which improves the safety of live operations. and efficiency.

CN119027663APending Publication Date: 2024-11-26STATE GRID HUNAN ELECTRIC POWER COMPANY LIMITED +2

Patent Information

Application Number
CN202411065417.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The prior art cannot accurately and accurately identify and locate the installation gap of transmission line spacing rods, resulting in safety hazards and labor intensity in high-risk environments, affecting the reliability of power supply.

Method used

The spacer installation gap recognition and positioning method based on semantic segmentation is adopted. The spacer installation gap recognition and positioning model is constructed through the DeepLab3+ algorithm. Combined with data enhancement, labeling and feature extraction modules, FocalLoss and DiceLoss are used for model training to achieve high accuracy of the spacer gap Identify.

Benefits of technology

It significantly improves the identification accuracy and efficiency of the gap between the spacer rod, reduces the labor intensity and safety threat to operators, and enhances the safety and identification efficiency of live operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027663A_ABST
    Figure CN119027663A_ABST
Patent Text Reader

Abstract

The invention discloses a spacer installation gap identification and positioning method based on semantic segmentation, and the method comprises the following steps: obtaining an image of a spacer installation process, and obtaining an image data set; performing data enhancement on the image data set, labeling the image, and finally performing data processing to obtain a training data set; based on a DeepLab3 + algorithm, constructing a spacer installation gap identification and positioning initial model; using the training data set to train the spacer installation gap identification and positioning initial model to obtain a spacer installation gap identification and positioning model; and carrying out spacer installation gap identification and positioning by using the spacer installation gap identification and positioning model. The invention also discloses a system for realizing the spacer installation gap identification and positioning method based on semantic segmentation. According to the method, the recognition efficiency is improved on the premise that the recognition precision is ensured, binarization processing is carried out by adopting an Otsu threshold method, and the images are effectively divided without being influenced by image brightness and contrast.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of spacer bar installation, and in particular to a spacer bar installation gap identification and positioning method and system based on semantic segmentation. Background Art

[0002] As a carrier of power transportation, the power transmission line of the power system plays a key role in power transportation. The spacer is an important component of the transmission line, which is mainly used to split the transmission line and fix the spacing between the split lines. In severe weather such as strong winds, ice and snow, the spacer can prevent the lines from whipping and colliding with each other, and suppress the vibration and sub-span oscillation caused by strong winds. Once the transmission line spacer fails or is abnormal, it will directly affect the reliability of power transmission. At present, the installation of spacers is mainly carried out by operators wearing insulating clothing for live operations. Live operations in high-risk environments such as high altitude, high voltage, and strong electric fields are extremely dangerous and labor-intensive, which can easily lead to casualties such as falls or high-voltage electric shocks. In addition, the serious lack of professional talents for live operations has greatly affected the normalization of live operations.

[0003] With the continuous improvement of the level of production automation, robots are gradually replacing humans to complete various live-line operations, especially in some situations with high risk factors and high labor intensity. The widespread use of live-line working robots can greatly improve the efficiency and quality of live-line working, and is also one of the practical ways to solve the contradictory problem of power supply reliability and the safety of manual live-line maintenance work. However, with regard to the installation of transmission line spacers, live-line working robots are still unable to accurately and precisely identify, locate, and install transmission line spacers. In order to improve the safety of live-line working, reduce the labor intensity of operators and the personal threat of strong electric fields to operators, there is an urgent need for a method that can accurately identify and locate the installation gap of spacers. Summary of the invention

[0004] One of the purposes of the present invention is to provide a method for identifying and locating a spacer bar installation gap based on semantic segmentation, which can identify and locate the spacer bar installation gap with high precision.

[0005] A second objective of the present invention is to provide a system for implementing the spacer bar installation gap identification and positioning method based on semantic segmentation.

[0006] The present invention provides a method for identifying and locating the installation gap of a spacer bar based on semantic segmentation, comprising the following steps:

[0007] S1. Obtain an image of the spacer installation process to obtain an image data set;

[0008] S2. Perform data enhancement on the image data set obtained in step S1, annotate the images, and finally process the data to obtain a training data set;

[0009] S3. Construct an initial model for identifying and positioning the installation gap of the spacer bar based on the DeepLab3+ algorithm;

[0010] S4. Using the training data set obtained in step S2, the initial model for identifying and positioning the spacer installation gap obtained in step S3 is trained to obtain a spacer installation gap identification and positioning model;

[0011] S5. Use the spacer bar installation gap identification and positioning model obtained in step S4 to perform actual spacer bar installation gap identification and positioning.

[0012] The data enhancement in step S2 includes rotating the spacer bar image left and right, adding random salt and pepper noise or Gaussian noise within a preset range, randomly reducing the spacer bar image within a preset ratio range, and filling the border with black pixels.

[0013] The marking in step S2 specifically includes: using a marking tool to mark the spacer grooves, installation gaps and wires in the image.

[0014] The data processing in step S2 is specifically as follows: using the annotation labels to generate corresponding target pixel and background pixel information position files, and then batch processing the labels to generate a grayscale image containing target and background pixel information.

[0015] The initial model for identifying and positioning the installation gap of the spacer rod in step S3 includes an encoder part and a decoder part;

[0016] The encoder part includes a feature extraction module, a first attention mechanism module, an ASPP module, and a first 1×1 convolutional layer connected in series in sequence; the output of the first 1×1 convolutional layer is used as the output of the encoder part;

[0017] The feature extraction module includes a convolution module, a bneck module, an average pooling module, a fully connected layer module and a fully connected layer connected in series; the feature extraction module completes feature extraction through deep separable convolution, reducing the number of parameters and the amount of calculation;

[0018] The convolution module includes a convolutional layer, a batch normalization layer, and an hswish activation function connected in series;

[0019] The bneck module consists of a 1×1 dimensionality-increasing convolutional layer, a 3×3 depth-wise separable convolutional layer, a 1×1 dimensionality-reducing convolutional layer, and a SE attention mechanism connected in series.

[0020] The fully connected layer module includes a fully connected layer, a batch normalization layer, and an hswish activation function connected in series;

[0021] The first attention mechanism module includes an average pooling layer, an adaptive one-dimensional convolution kernel, and a normalization layer connected in series in sequence; the first attention mechanism module first obtains the global average value of each channel through the global average pooling layer, and then calculates the kernel_size through the adaptive one-dimensional convolution kernel and applies it to the feature map to obtain the weight vector of each channel. Finally, the weight vector of each channel passes through the normalization layer and is weighted and multiplied by the input feature map to obtain the output of the first attention mechanism module;

[0022] The adaptive one-dimensional convolution kernel calculation process is calculated using the following formula:

[0023]

[0024]

[0025] Among them, k is the convolution kernel size; C is the different channel dimensions; b and γ are preset parameters; |· odd To take the nearest odd number;

[0026] The ASPP module includes a 1×1 convolution module, a first compound convolution module, a second compound convolution module, a third compound convolution module and an image pooling layer; the input data enters the 1×1 convolution module, the first compound convolution module, the second compound convolution module, the third compound convolution module and the image pooling layer for processing, and then the outputs of the 1×1 convolution module, the first compound convolution module, the second compound convolution module, the third compound convolution module and the image pooling layer are feature fused to obtain the output of the ASPP module;

[0027] The first to third composite convolution modules have the same structure, including 3×1 convolutions and 1×3 convolutions connected in series; the convolution layer parameters in the first composite convolution module are padding=6, dilation=6; the convolution layer parameters in the second composite convolution module are padding=12, dilation=12; the convolution layer parameters in the third composite convolution module are padding=18, dilation=18;

[0028] The decoder part includes the second 1×1 convolution layer, the concat module, the first upsampling module, the second upsampling module, the second attention mechanism module and the 3×3 convolution layer; the second 1×1 convolution layer uses the output of the feature extraction module of the encoder part as input; the first upsampling module uses the output of the first 1×1 convolution layer of the encoder part as input; the concat module uses the output of the second 1×1 convolution layer and the first upsampling module as input; the second attention mechanism module uses the output of the concat module as input; the second attention mechanism module, the 3×3 convolution layer and the second upsampling module are connected in series in sequence; the output of the second upsampling module is used as the final output of the initial model for identifying and positioning the installation gap of the spacer bar;

[0029] The structure of the second attention mechanism module is the same as that of the first attention mechanism module.

[0030] Step S4 is specifically as follows: construct a loss function based on FocalLoss and DiceLoss, train the initial model for identifying and locating the installation gap of the spacer bar, and the total loss function is expressed by the following formula:

[0031] L=L Dice +L Focal

[0032] Among them, L is the total loss function; L Focal is the focal loss function, L Dice is the similarity coefficient loss function, expressed using the following formula:

[0033]

[0034] L Focal =-α i (1-p i ) γ logp i

[0035] Among them, y i is the predicted value, taking the value (0,1); t i is the target value, which can be 0 or 1; γ is the smoothing coefficient; α i is the weight factor; (1-p i ) γ is the adjustment factor; p i is the predicted probability of the corresponding category.

[0036] Step S5 specifically includes the following steps:

[0037] S5.1 obtain an image of the spacer bar installation process;

[0038] S5.2 inputting the image obtained in step S5.1 into the spacer bar installation gap recognition and positioning model obtained in step S4 to obtain a recognition result image;

[0039] S5.3 converts the recognition result image obtained in step S5.2 from the RGB color space to the HSV color space, performs three-channel separation on the HSV color model, extracts the color component of the spacer installation gap, and finally restores the current HSV image to the RGB image to obtain a segmentation result that only retains the spacer installation gap;

[0040] S5.4 uses the Otsu threshold method to binarize the segmentation result of the spacer installation gap obtained in step S5.3, and then performs contour detection to calculate the pixel points within the contour to obtain the gap area;

[0041] S5.5 determines whether the spacer bar groove is in contact with the conductor based on the gap area obtained in step S5.4, thereby completing the identification and positioning of the spacer bar installation gap.

[0042] Step S5.4 specifically comprises: calculating the grayscale histogram of the segmentation result of the spacer installation gap obtained in step S5.3, and counting the number of pixels at each grayscale level;

[0043] Then calculate the optimal segmentation threshold of the image to divide the image into foreground and background. The formula for calculating the optimal segmentation threshold is as follows:

[0044]

[0045]

[0046]

[0047]

[0048]

[0049] σ 2 =ω1×(μ-μ1) 2 +ω2×(μ-μ2) 2

[0050] Among them, ω1 is the ratio of the number of target pixels to the image; ω2 is the ratio of the number of background pixels to the image; N is the number of pixels less than the initial threshold T0; M is the number of pixels greater than the initial threshold T0; A is the number of pixels in the horizontal direction of the image; B is the number of pixels in the vertical direction of the image; A×B is the total number of pixels in the image; μ is the grayscale average of the entire image; μ1 is the grayscale average of the foreground area; μ2 is the grayscale average of the background area; h i is the number of pixels with gray value i, i∈[0,255]; σ is the inter-class variance of foreground and background;

[0051] The threshold T corresponding to the maximum value of σ is used as the optimal segmentation threshold. The pixels with grayscale values ​​greater than T in the input image are set as foreground, and the pixels with grayscale values ​​less than T are set as background.

[0052] The present invention also provides a system for implementing the spacer installation gap identification and positioning method based on semantic segmentation, comprising an image acquisition module, an image enhancement module, a model construction module, a model training module, and an installation gap identification module;

[0053] The image acquisition module acquires the image of the spacer bar installation process and uploads the data to the image enhancement module;

[0054] The image enhancement module performs data enhancement on the image based on the received data, annotates the image, and uploads the data to the model training module;

[0055] The model building module builds the initial model for identifying and locating the installation gap of the spacer bar based on the DeepLabV3+ algorithm, and uploads the data to the model training module;

[0056] The model training module performs model training based on the received data to obtain a spacer bar installation gap recognition and positioning model, and uploads the data to the installation gap recognition module;

[0057] The installation gap identification module completes the installation gap identification and positioning according to the received data.

[0058] The present invention discloses a method and system for identifying and locating the installation gap of a spacer bar based on semantic segmentation. The model constructed based on the DeepLabV3+ algorithm can significantly improve the speed of model training, and improve the recognition efficiency under the premise of ensuring the recognition accuracy. The loss function proposed in the method of the present invention can effectively deal with the problem of uneven samples in the data set. When obtaining the installation gap area of ​​the spacer bar, the present invention adopts the adaptive Otsu threshold method for binarization processing, which is simple and fast, is not affected by the brightness and contrast of the image, and effectively divides the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a schematic diagram of the process of the present invention;

[0060] Figure 2 It is a structural schematic diagram of the system of the present invention;

[0061] Figure 3 This is the result obtained by using the method of the present invention using the test set in the embodiment of the present invention. DETAILED DESCRIPTION

[0062] The present invention provides a method for identifying and locating the installation gap of a spacer bar based on semantic segmentation, and its flow chart is as follows: Figure 1 As shown, the following steps are included:

[0063] S1. Obtain an image of the spacer installation process to obtain an image data set;

[0064] S2. Perform data enhancement on the image data set obtained in step S1, annotate the images, and finally process the data to obtain a training data set;

[0065] The data enhancement includes rotating the spacer bar image left and right, adding random salt and pepper noise or Gaussian noise within a preset range, randomly reducing the spacer bar image within a preset ratio range, and filling the border with black pixels.

[0066] The marking specifically includes: using a marking tool to mark the spacer bar grooves, installation gaps and wires in the image.

[0067] The data processing specifically includes: generating corresponding target pixel and background pixel information position files using annotation labels, and then batch processing the labels to generate grayscale images containing target and background pixel information.

[0068] S3. Construct an initial model for identifying and positioning the installation gap of the spacer bar based on the DeepLab3+ algorithm;

[0069] The spacer bar installation gap identification and positioning initial model includes an encoder part and a decoder part;

[0070] The encoder part includes a feature extraction module, a first attention mechanism module, an ASPP module, and a first 1×1 convolutional layer connected in series in sequence; the output of the first 1×1 convolutional layer is used as the output of the encoder part;

[0071] The feature extraction module includes a convolution module, a bneck module, an average pooling module, a fully connected layer module and a fully connected layer connected in series; the feature extraction module completes feature extraction through deep separable convolution, reducing the number of parameters and the amount of calculation;

[0072] The feature extraction module uses deep separable convolution to complete feature extraction, replacing the original backbone network Xception in the DeepLab3+ algorithm. The advantages of lightweight memory and computing power make the network less computationally intensive and faster, thus improving the real-time performance of the algorithm.

[0073] The convolution module includes a convolutional layer, a batch normalization layer, and an hswish activation function connected in series;

[0074] The bneck module consists of a 1×1 dimensionality-increasing convolutional layer, a 3×3 depth-wise separable convolutional layer, a 1×1 dimensionality-reducing convolutional layer, and a SE attention mechanism connected in series.

[0075] The fully connected layer module includes a fully connected layer, a batch normalization layer, and an hswish activation function connected in series;

[0076] The first attention mechanism module includes an average pooling layer, an adaptive one-dimensional convolution kernel, and a normalization layer connected in series in sequence; the first attention mechanism module first obtains the global average value of each channel through the global average pooling layer, and then calculates the kernel_size through the adaptive one-dimensional convolution kernel and applies it to the feature map to obtain the weight vector of each channel. Finally, the weight vector of each channel passes through the normalization layer and is weighted and multiplied by the input feature map to obtain the output of the first attention mechanism module;

[0077] The adaptive one-dimensional convolution kernel calculation process is calculated using the following formula:

[0078]

[0079]

[0080] Among them, k is the convolution kernel size; C is the different channel dimensions; b and γ are preset parameters; |· odd To take the nearest odd number;

[0081] The addition of the first attention mechanism module and the second attention mechanism module allows the model to adaptively pay attention to important and effective target information when processing images, improving model performance;

[0082] The ASPP module includes a 1×1 convolution module, a first compound convolution module, a second compound convolution module, a third compound convolution module and an image pooling layer; the input data enters the 1×1 convolution module, the first compound convolution module, the second compound convolution module, the third compound convolution module and the image pooling layer for processing, and then the outputs of the 1×1 convolution module, the first compound convolution module, the second compound convolution module, the third compound convolution module and the image pooling layer are feature fused to obtain the output of the ASPP module;

[0083] The first to third composite convolution modules have the same structure, including 3×1 convolutions and 1×3 convolutions connected in series; the convolution layer parameters in the first composite convolution module are padding=6, dilation=6; the convolution layer parameters in the second composite convolution module are padding=12, dilation=12; the convolution layer parameters in the third composite convolution module are padding=18, dilation=18;

[0084] Based on the original ASPP structure, the ASPP module decomposes the 3×3 hole convolution into 3×1 convolution and 1×3 convolution, while keeping the hole rate unchanged. The number of parameters is only 2 / 3 of the original ASPP structure, and it can extract important feature information more effectively.

[0085] The decoder part includes the second 1×1 convolution layer, the concat module, the first upsampling module, the second upsampling module, the second attention mechanism module and the 3×3 convolution layer; the second 1×1 convolution layer uses the output of the feature extraction module of the encoder part as input; the first upsampling module uses the output of the first 1×1 convolution layer of the encoder part as input; the concat module uses the output of the second 1×1 convolution layer and the first upsampling module as input; the second attention mechanism module uses the output of the concat module as input; the second attention mechanism module, the 3×3 convolution layer and the second upsampling module are connected in series in sequence; the output of the second upsampling module is used as the final output of the initial model for identifying and positioning the installation gap of the spacer bar;

[0086] The structure of the second attention mechanism module is the same as that of the first attention mechanism module.

[0087] S4. Using the training data set obtained in step S2, the initial model for identifying and locating the spacer installation gap obtained in step S3 is trained to obtain a spacer installation gap identification and positioning model, specifically: a loss function is constructed based on FocalLoss and DiceLoss, and the initial model for identifying and locating the spacer installation gap is trained. The total loss function is expressed by the following formula:

[0088] L=L Dice +L Focal

[0089] Among them, L is the total loss function; L Focal is the focal loss function, L Dice is the similarity coefficient loss function, expressed using the following formula:

[0090]

[0091] L Focal =-α i (1-p i ) γ logp i

[0092] Among them, y i is the predicted value, taking the value (0,1); t i is the target value, which can be 0 or 1; γ is the smoothing coefficient; α i is the weight factor; (1-p i ) γ is the adjustment factor; p i is the predicted probability of the corresponding category.

[0093] S5. Using the spacer bar installation gap identification and positioning model obtained in step S4, actual spacer bar installation gap identification and positioning is performed, specifically comprising the following steps:

[0094] S5.1 obtain an image of the spacer bar installation process;

[0095] S5.2 inputting the image obtained in step S5.1 into the spacer bar installation gap recognition and positioning model obtained in step S4 to obtain a recognition result image;

[0096] S5.3 converts the recognition result image obtained in step S5.2 from the RGB color space to the HSV color space, performs three-channel separation on the HSV color model, extracts the color component of the spacer installation gap, and finally restores the current HSV image to the RGB image to obtain a segmentation result that only retains the spacer installation gap;

[0097] S5.4 uses the Otsu threshold method to binarize the segmentation result of the spacer installation gap obtained in step S5.3, and then performs contour detection to calculate the pixel points within the contour to obtain the gap area;

[0098] S5.5 determines whether the spacer bar groove is in contact with the conductor based on the gap area obtained in step S5.4, thereby completing the identification and positioning of the spacer bar installation gap.

[0099] Step S5.4 specifically comprises: calculating the grayscale histogram of the segmentation result of the spacer installation gap obtained in step S5.3, and counting the number of pixels at each grayscale level;

[0100] Then calculate the optimal segmentation threshold of the image to divide the image into foreground and background. The formula for calculating the optimal segmentation threshold is as follows:

[0101]

[0102]

[0103]

[0104]

[0105]

[0106] σ 2 =ω1×(μ-μ1) 2 +ω2×(μ-μ2) 2

[0107] Among them, ω1 is the ratio of the number of target pixels to the image; ω2 is the ratio of the number of background pixels to the image; N is the number of pixels less than the initial threshold T0; M is the number of pixels greater than the initial threshold T0; A is the number of pixels in the horizontal direction of the image; B is the number of pixels in the vertical direction of the image; A×B is the total number of pixels in the image; μ is the grayscale average of the entire image; μ1 is the grayscale average of the foreground area; μ2 is the grayscale average of the background area; h i is the number of pixels with gray value i, i∈[0,255]; σ is the inter-class variance of foreground and background;

[0108] The threshold T corresponding to the maximum value of σ is used as the optimal segmentation threshold. The pixels with grayscale values ​​greater than T in the input image are set as foreground, and the pixels with grayscale values ​​less than T are set as background.

[0109] The present invention also provides a system for implementing the spacer installation gap identification and positioning method based on semantic segmentation, comprising an image acquisition module, an image enhancement module, a model construction module, a model training module, and an installation gap identification module;

[0110] The image acquisition module acquires the image of the spacer bar installation process and uploads the data to the image enhancement module;

[0111] The image enhancement module performs data enhancement on the image based on the received data, annotates the image, and uploads the data to the model training module;

[0112] The model building module builds the initial model for identifying and locating the installation gap of the spacer bar based on the DeepLabV3+ algorithm, and uploads the data to the model training module;

[0113] The model training module performs model training based on the received data to obtain a spacer bar installation gap recognition and positioning model, and uploads the data to the installation gap recognition module;

[0114] The installation gap identification module completes the installation gap identification and positioning according to the received data.

[0115] The method of the present invention is further described below in conjunction with an embodiment:

[0116] 2785 images of spacer installation were collected as the data set, and the labels including ao, gap, and line represent the spacer groove, spacer installation gap, and transmission line, respectively.

[0117] The data set is preprocessed to expand the breadth and depth of the data set, thereby improving the robustness of the model. This embodiment selects three image processing measures: (1) the collected spacer bar image is rotated left and right by ±5°, ±15°, and ±25° to improve the network's multi-angle recognition of the spacer bars; (2) a certain range of random salt and pepper noise or Gaussian noise is randomly added to improve the network's invariance to camera distortion; (3) the spacer bar image is reduced at a random ratio and the border is filled with black pixels.

[0118] There are 6834 images in total in the preprocessed and expanded data set, of which 90% of the images are used as training sets and 10% of the images are used as test sets, which are processed by the method of the present invention. The frozen training method is adopted to improve the training efficiency and accelerate the convergence. The total number of iterations is set to 400. The mosaic data enhancement method is adopted for training in the first 90% of the iterations. The Adam optimizer is used throughout the process, and the transfer learning strategy is adopted. The freezing stage is iterated 100 times, the batch sample training number (Batch_size) is set to 16, and the learning rate is 0.001; the thawing stage is iterated 300 times, the Batch_size is set to 8, and the learning rate is 0.0001. After the training is completed, the system automatically saves the parameters of the neural network.

[0119] The model performance was tested using 500 images in the test set. The test results are as follows: Figure 3 According to the test results, the method of the present invention can accurately identify the installation gaps at various positions of the spacer bars and accurately output the area size of the installation gaps.

Claims

1. A spacer bar installation gap identification and positioning method based on semantic segmentation, It is characterized in that The following steps are involved: S1. Obtain an image of the spacer installation process to obtain an image data set; S2. Perform data enhancement on the image data set obtained in step S1, annotate the images, and finally process the data to obtain a training data set; S3. Construct an initial model for identifying and positioning the installation gap of the spacer bar based on the DeepLab3+ algorithm; S4. Using the training data set obtained in step S2, the initial model for identifying and positioning the spacer installation gap obtained in step S3 is trained to obtain a spacer installation gap identification and positioning model; S5. Use the spacer bar installation gap identification and positioning model obtained in step S4 to perform actual spacer bar installation gap identification and positioning.

2. According to the method for identifying and locating the installation gap of spacer bars based on semantic segmentation in claim 1, It is characterized in that The data enhancement in step S2 includes rotating the spacer bar image left and right, adding random salt and pepper noise or Gaussian noise within a preset range, randomly reducing the spacer bar image within a preset ratio range, and filling the border with black pixels.

3. According to the method for identifying and locating the installation gap of spacer bars based on semantic segmentation in claim 1, It is characterized in that The marking in step S2 specifically includes: using a marking tool to mark the spacer grooves, installation gaps and wires in the image.

4. The spacer bar installation gap identification and positioning method based on semantic segmentation according to claim 1, It is characterized in that The data processing in step S2 is specifically as follows: using the annotation labels to generate corresponding target pixel and background pixel information position files, and then batch processing the labels to generate a grayscale image containing target and background pixel information.

5. The spacer bar installation gap identification and positioning method based on semantic segmentation according to claim 1, It is characterized in that The initial model for identifying and positioning the installation gap of the spacer rod in step S3 includes an encoder part and a decoder part; The encoder part includes a feature extraction module, a first attention mechanism module, an ASPP module, and a first 1×1 convolutional layer connected in series in sequence; the output of the first 1×1 convolutional layer is used as the output of the encoder part; The feature extraction module includes a convolution module, a bneck module, an average pooling module, a fully connected layer module and a fully connected layer connected in series; the feature extraction module completes feature extraction through deep separable convolution, reducing the number of parameters and the amount of calculation; The convolution module includes a convolutional layer, a batch normalization layer, and an hswish activation function connected in series; The bneck module consists of a 1×1 dimensionality-increasing convolutional layer, a 3×3 depth-wise separable convolutional layer, a 1×1 dimensionality-reducing convolutional layer, and a SE attention mechanism connected in series. The fully connected layer module includes a fully connected layer, a batch normalization layer, and an hswish activation function connected in series; The first attention mechanism module includes an average pooling layer, an adaptive one-dimensional convolution kernel, and a normalization layer connected in series in sequence; the first attention mechanism module first obtains the global average value of each channel through the global average pooling layer, and then calculates the kernel_size through the adaptive one-dimensional convolution kernel and applies it to the feature map to obtain the weight vector of each channel. Finally, the weight vector of each channel passes through the normalization layer and is weighted and multiplied by the input feature map to obtain the output of the first attention mechanism module; The adaptive one-dimensional convolution kernel calculation process is calculated using the following formula: Among them, k is the convolution kernel size; C is the different channel dimensions; b and γ are preset parameters; |· odd To take the nearest odd number; The ASPP module includes a 1×1 convolution module, a first compound convolution module, a second compound convolution module, a third compound convolution module and an image pooling layer; the input data enters the 1×1 convolution module, the first compound convolution module, the second compound convolution module, the third compound convolution module and the image pooling layer for processing, and then the outputs of the 1×1 convolution module, the first compound convolution module, the second compound convolution module, the third compound convolution module and the image pooling layer are feature fused to obtain the output of the ASPP module; The first to third compound convolution modules have the same structure, including 3×1 convolutions and 1×3 convolutions connected in series; the convolution layer parameters in the first compound convolution module are padding=6, dilation=6; the convolution layer parameters in the second compound convolution module are padding=12, dilation=12; the convolution layer parameters in the third compound convolution module are padding=18, dilation=18.

6. The spacer bar installation gap identification and positioning method based on semantic segmentation according to claim 5, It is characterized in that The decoder part includes a second 1×1 convolutional layer, a concat module, a first upsampling module, a second upsampling module, a second attention mechanism module and a 3×3 convolutional layer; the second 1×1 convolutional layer uses the output of the feature extraction module of the encoder part as input; the first upsampling module uses the output of the first 1×1 convolutional layer of the encoder part as input; The concat module takes the output of the second 1×1 convolutional layer and the first upsampling module as input; The second attention mechanism module uses the output of the concat module as input; The second attention mechanism module, the 3×3 convolutional layer, and the second upsampling module are connected in series in sequence; the output of the second upsampling module is used as the final output of the initial model for identifying and positioning the installation gap of the spacer bar; The structure of the second attention mechanism module is the same as that of the first attention mechanism module.

7. The spacer bar installation gap identification and positioning method based on semantic segmentation according to claim 6, It is characterized in that Step S4 is specifically as follows: construct a loss function based on FocalLoss and DiceLoss, train the initial model for identifying and locating the installation gap of the spacer bar, and the total loss function is expressed by the following formula: L=L Dice +L Focal Among them, L is the total loss function; L Focal is the focal loss function, L Dice is the similarity coefficient loss function, expressed using the following formula: THE Focal =-α i (1-p i ) γ log i Among them, y i is the predicted value, taking the value (0,1); t i is the target value, which can be 0 or 1; γ is the smoothing coefficient; α i is the weight factor; (1-p i ) γ is the adjustment factor; p i is the predicted probability of the corresponding category.

8. The spacer bar installation gap identification and positioning method based on semantic segmentation according to claim 7, It is characterized in that Step S5 specifically includes the following steps: S5.1 obtain an image of the spacer bar installation process; S5.2 inputting the image obtained in step S5.1 into the spacer bar installation gap recognition and positioning model obtained in step S4 to obtain a recognition result image; S5.3 converts the recognition result image obtained in step S5.2 from the RGB color space to the HSV color space, performs three-channel separation on the HSV color model, extracts the color component of the spacer installation gap, and finally restores the current HSV image to the RGB image to obtain a segmentation result that only retains the spacer installation gap; S5.4 uses the Otsu threshold method to binarize the segmentation result of the spacer installation gap obtained in step S5.3, and then performs contour detection to calculate the pixel points within the contour to obtain the gap area; S5.5 determines whether the spacer bar groove is in contact with the conductor based on the gap area obtained in step S5.4, thereby completing the identification and positioning of the spacer bar installation gap.

9. The spacer bar installation gap identification and positioning method based on semantic segmentation according to claim 8, It is characterized in that Step S5.4 is specifically as follows: calculating the grayscale histogram of the segmentation result of the spacer installation gap obtained in step S5.3, and counting the number of pixels at each grayscale level; then calculating the optimal segmentation threshold of the image to segment the image into foreground and background. The formula for calculating the optimal segmentation threshold is as follows: s 2 =ω 1 ×(μ-μ 1 ) 2 +oh 2 ×(μ-μ 2 ) 2 Among them, ω 1 is the ratio of the number of target pixels to the image; ω 2 is the ratio of background pixels to the image; N is the number of pixels less than the initial threshold T0; M is the number of pixels greater than the initial threshold T0; A is the number of pixels in the horizontal direction of the image; B is the number of pixels in the vertical direction of the image; A×B is the total number of pixels in the image; μ is the grayscale average of the entire image; μ 1 is the average grayscale value of the foreground area; μ 2 is the average grayscale value of the background area; h i is the number of pixels with gray value i, i∈[0,255]; σ is the inter-class variance of foreground and background; The threshold T corresponding to the maximum value of σ is used as the optimal segmentation threshold. The pixels with grayscale values ​​greater than T in the input image are set as foreground, and the pixels with grayscale values ​​less than T are set as background.

10. A system for implementing the spacer bar installation gap identification and positioning method based on semantic segmentation as claimed in any one of claims 1 to 9, It is characterized in that It includes an image acquisition module, an image enhancement module, a model building module, a model training module, and an installation gap recognition module; The image acquisition module acquires the image of the spacer bar installation process and uploads the data to the image enhancement module; The image enhancement module performs data enhancement on the image based on the received data, annotates the image, and uploads the data to the model training module; The model building module builds the initial model for identifying and locating the installation gap of the spacer bar based on the DeepLabV3+ algorithm, and uploads the data to the model training module; The model training module performs model training based on the received data to obtain a spacer bar installation gap recognition and positioning model, and uploads the data to the installation gap recognition module; The installation gap identification module completes the installation gap identification and positioning according to the received data.

Citation Information

Patent Citations

  • Wire winding gap identification method and device and storage medium

    CN114266963A

  • Squeeze-enhanced axial transformer, its layer and methods thereof

    US11776240B1

  • System and method for boundary aware semantic segmentation

    US20210089807A1

Cited By

  • Spacer mounting robot with wire adjusting mechanism and control method thereof

    CN119482176A

  • A spacer bar mounting robot having a wire adjusting mechanism and a control method thereof

    CN119482176B