A method for positioning and automatic docking of a grounding point based on machine vision

Abstract

The application discloses a grounding point positioning and automatic docking method based on machine vision, and relates to the technical field of grounding detection.The application obtains an RGB image by S1: collecting a grounding site image and performing image preprocessing;S2: inputting the RGB image into a multi-task deep learning network, performing multi-scale feature extraction on the multi-task deep learning network taking a ConvNeXt-Tiny network as a backbone network, fusing the extracted multi-scale features through a bidirectional feature pyramid module BiFPN, performing feature enhancement on the fused features through an atrous convolution module ASPP, and performing a wire segmentation task and a grounding ring detection task to output spatial positions of the wires and pose information of the grounding ring; and S3: controlling a grounding operation terminal executive mechanism to move according to the spatial positions of the wires and the pose information of the grounding ring, realizing automatic docking with the grounding ring, confirming connection reliability in real time, and guaranteeing grounding safety.

Description

Technical Field

[0001] This invention relates to the field of grounding detection technology, and in particular to a grounding point location and automatic connection method based on machine vision. Background Technology

[0002] Grounding wires are the "lifeline" for ensuring the safety of personnel in the operation and maintenance of power systems. At present, the installation and removal of grounding wires are mainly carried out manually, which has problems such as low efficiency, high safety risks and poor operation standardization. Moreover, the grounding points are mostly located at high altitudes, and traditional operations require personnel to climb to structures or towers several meters or even tens of meters high to carry out the operation, which is a huge test of the physical and mental strength of the operators.

[0003] On the other hand, power grid operation and maintenance emphasizes rapid response to minimize power outage time. However, traditional manual grounding operations are cumbersome and time-consuming, heavily reliant on manpower from tool preparation and voltage testing to installation and removal, especially in complex wiring or multi-point grounding situations. Furthermore, grounding work has strict operating procedures, and in high-voltage environments, relying entirely on personnel's responsibility and experience inevitably leads to errors such as "going to the wrong bay," "missing or incorrectly installing grounding wires," and "operating without voltage testing." Therefore, there is an urgent need for an intelligent method that can automatically locate and connect grounding points to improve operational efficiency and safety. Summary of the Invention

[0004] In view of the above-mentioned prior art, the present invention provides a grounding point positioning and automatic docking method based on machine vision, which mainly solves the technical problems existing in the background art.

[0005] To achieve the above objectives, the technical solution of this invention is implemented as follows: This invention provides a machine vision-based method for grounding point location and automatic docking, the method comprising the following steps: S1: Acquire grounding site images and perform image preprocessing to obtain RGB images; S2: Input the RGB image into the multi-task deep learning network. The multi-task deep learning network uses the ConvNeXt-Tiny network as the backbone network to extract multi-scale features. The extracted multi-scale features are fused through the Bidirectional Feature Pyramid (BiFPN) module. The fused features are then enhanced through the dilated convolution module (ASPP) and the wire segmentation and ground loop detection tasks are performed. The spatial position of the wire and the pose information of the ground loop are output. S3: Based on the spatial position of the conductor and the orientation information of the grounding ring, control the movement of the grounding operation terminal actuator to achieve automatic docking with the grounding ring.

[0006] As a preferred embodiment of the present invention, in step S2, the structure of the ConvNeXt-Tiny network includes a sampling layer, a first-level convolutional next-generation network block, a second-level convolutional next-generation network block, a third-level convolutional next-generation network block, and a fourth-level convolutional next-generation network block. The first-level convolutional next-generation network block, the second-level convolutional next-generation network block, the third-level convolutional next-generation network block, and the fourth-level convolutional next-generation network block all include depthwise convolutional layers and pointwise convolutional layers. The sampling layer sequentially connects the four convolutional next-generation network blocks of different levels.

[0007] As a preferred embodiment of the present invention, in step S2, multi-scale feature extraction using the ConvNeXt-Tiny network as the backbone network is divided into four stages. During the transfer between the four stages, the spatial resolution gradually decreases and the number of channels gradually increases. Specifically, the first stage uses three cascaded first-level convolutional next-generation network blocks, the second stage uses three cascaded second-level convolutional next-generation network blocks, the third stage uses nine cascaded third-level convolutional next-generation network blocks, and the fourth stage uses three cascaded fourth-level convolutional next-generation network blocks. The four stages output a first feature map, a second feature map, a third feature map, and a fourth feature map, respectively.

[0008] As a preferred embodiment of the present invention, the specific process of outputting the first feature map, the second feature map, the third feature map, and the fourth feature map in the four stages respectively includes: The RGB image is downsampled using a sampling layer to obtain the original feature map; The original feature map is input into the first stage, where spatial features are extracted through a deep convolutional layer. After layer normalization, it is then passed through two pointwise convolutional layers for channel expansion and feature mapping. The first pointwise convolutional layer receives the feature map processed by the deep convolutional layer, expands the number of channels of the feature map to four times the number of input channels, and then performs a non-linear transformation through a Gaussian error linear unit activation function before being input to the second pointwise convolutional layer. The second pointwise convolutional layer compresses the number of channels from the expanded dimension back to the original input dimension. The original input feature map is added to the feature map after the second pointwise convolutional layer through residual connection to obtain the output of the current first-level convolutional next-generation network block. After being processed by three first-level convolutional next-generation network blocks in sequence, the first stage outputs the first feature map. The first feature map is input into the second stage, and the second feature map is obtained by performing the same processing as the first stage. The second feature map is input into the third stage, and the same processing as in the first stage is performed to obtain the third feature map; The third feature map is input into the fourth stage, and the fourth feature map is obtained by performing the same processing as in the first stage.

[0009] In a preferred embodiment of the present invention, in step S2, the extracted multi-scale features are fused using a feature pyramid module BiFPN, and then the fused features are enhanced using a dilated convolution module ASPP. The feature pyramid module BiFPN consists of two modules, and the specific fusion process is as follows: The first feature map, the second feature map, the third feature map, and the fourth feature map are respectively input into the first bidirectional feature pyramid module BiFPN for a round of bidirectional feature fusion from top to bottom and from bottom to top; In the top-down path, semantic information is passed from the highest level feature to the lowest level feature, resulting in four intermediate feature maps. Based on the top-down path, a bottom-up path is executed on the four intermediate feature maps, passing detailed information from the lowest level feature to the highest level feature, resulting in four output feature maps. Using the four output feature maps as the starting point of the top-down path in the second bidirectional feature pyramid module BiFPN, the same principle as the first bidirectional feature pyramid module BiFPN is used for processing, and finally four hierarchical feature maps are obtained, namely the first-level feature map, the second-level feature map, the third-level feature map, and the fourth-level feature map.

[0010] As a preferred embodiment of the present invention, in step S2, the fused features are enhanced using a dilated convolution module (ASPP), specifically including: The fourth-level feature map is input into the dilated convolution module ASPP for feature enhancement. The dilated convolution module ASPP includes five parallel branches. The fourth-level feature map is input into the first branch, whose effective receptive field corresponds to a 3×3 region to capture its local contextual information. The second branch extends the effective receptive field to a 13×13 region, capturing contextual information within that region. The third branch extends the effective receptive field to a 25×25 area, capturing contextual information within that area. The fourth branch extends the effective receptive field to a 37×37 area, capturing contextual information within that area. The fifth branch performs a global average of the spatial dimensions of the fourth-level feature map to obtain a global context feature vector. This feature vector is then upsampled to the same spatial resolution as the input feature map using bilinear interpolation to obtain a global feature map with restored resolution. This global feature map is then transformed to obtain global context information. The information obtained from the five parallel branches is spliced ​​and compressed along the same channel dimension. The compressed fourth-level feature map is then added element-wise to the original fourth-level feature map to obtain the high-level feature.

[0011] In a preferred embodiment of the present invention, step S2, performing the wire segmentation task, specifically includes: The multi-channel features in the first-level feature map are compressed into a single channel to generate the original score map of each pixel belonging to the wire region. Upsampling restores the resolution of the original score map to that of the original features. Figure 1 Then, the score value is mapped to the probability value through the activation function, and finally the wire segmentation mask image with the same size as the original feature map is output. An RGB image is input into a monocular depth estimation network to generate a scene depth map corresponding to the image; Based on the wire segmentation mask, the depth values ​​of all pixels belonging to the wire region are extracted from the scene depth map, and the average depth value of the wire is calculated by averaging all the depth values. The spatial position of the traverse in the camera coordinate system is calculated based on the average depth of the traverse.

[0012] In a preferred embodiment of the present invention, step S2, which involves performing a grounding loop detection task, specifically includes: The high-level features are input into the ground loop detection task head for processing, and feature extraction is performed through three convolutional layers to obtain a deep feature map; Based on the deep feature map, bounding box regression and four predefined ground loop key points are performed to obtain the coordinate regression parameter map of the four key points corresponding to each spatial location. Based on the regression parameter map of the four key points, the two-dimensional coordinates of the regression parameter map of the four key points are obtained; Based on the known standard dimensions, the two-dimensional coordinates of the four key points are mapped to the three-dimensional coordinates in the object coordinate system, and the rotation matrix and translation vector of the grounding loop relative to the camera are solved by the PnP algorithm. The pose information of the grounding ring is obtained based on the rotation matrix and translation vector.

[0013] As a preferred embodiment of the present invention, in step S3, the automatic docking is achieved by adjusting the grounding operation terminal actuator according to the spatial position of the conductor and the six-degree-of-freedom pose of the grounding ring, specifically including: Based on the grounding ring pose information, the position of the grounding ring in the coordinate system of the grounding operation terminal actuator is taken as the target position, and its pose is taken as the target pose. The approximation path of the grounding operation terminal actuator is then planned. The target position and target pose are input to the inverse kinematics module of the grounding operation terminal actuator. The motion angle and displacement of each joint are solved analytically to obtain the joint target value and send it as a control command to the servo driver of each joint to drive the grounding operation terminal actuator to move towards the target pose according to the planned path. During the movement of the grounding operation terminal actuator, the current pose of the grounding ring is compared with the target pose in real time, the pose error is calculated, a correction command is generated based on the pose error, and it is superimposed on the basic motion command to form a closed-loop visual servo control. During the process of the grounding operation terminal actuator approaching and inserting the grounding ring, the pressure feedback signal is used to determine whether the docking is completed.

[0014] The beneficial effects of this invention are as follows: This invention achieves automatic docking of the grounding ring by controlling the grounding operation terminal actuator through machine vision. Specifically, it simultaneously realizes conductor segmentation and grounding ring detection through a multi-task deep learning network, which can accurately output the spatial position of the conductor and the six-degree-of-freedom pose of the grounding ring, providing precise visual guidance for automatic docking. Through the combined design of ConvNeXt-Tiny backbone network with BiFPN and ASPP, it takes into account both the depth of feature extraction and the fusion accuracy of multi-scale information. Furthermore, combined with closed-loop visual servo control, it realizes automatic and precise docking of the grounding operation terminal and the grounding ring, effectively replacing traditional manual operation and significantly improving the efficiency and safety of grounding operations. Attached Figure Description

[0015] Figure 1 This is a schematic diagram illustrating the steps of a machine vision-based grounding point location and automatic docking method. Detailed Implementation

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. In the following description, the expression "some embodiments" refers to a subset of all possible embodiments; however, it should be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with each other without conflict.

[0017] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0018] It should be understood that the present invention can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. Rather, providing these embodiments will make the disclosure thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Furthermore, the terminology used herein is intended only to describe particular embodiments and is not intended to limit the invention. When used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “compose” and / or “comprising,” when used in this specification, identify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups. When used herein, the term “and / or” includes any and all combinations of the associated listed items.

[0019] It should also be noted that when an element is referred to as being "fixed to" another element, it can be directly attached to the other element or there may be an intervening element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "inner," "outer," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementation.

[0020] To fully understand this invention, a detailed structure will be presented in the following description to illustrate the technical solution proposed by this invention. Optional embodiments of the invention are described in detail below; however, in addition to these detailed descriptions, the invention may have other embodiments.

[0021] Please refer to the attached document. Figure 1 This invention provides a machine vision-based method for grounding point location and automatic docking, the method comprising the following steps: S1: Acquire grounding site images and perform image preprocessing to obtain RGB images.

[0022] In this embodiment, a large number of grounding site images are acquired using a monocular camera. Image preprocessing includes dehazing and contrast enhancement to improve image quality.

[0023] Specifically, in the image acquisition stage, a monocular industrial camera is used. This camera is fixedly mounted on the actuator of the grounding operation terminal, with its optical axis parallel to the movement direction of the actuator. The camera's intrinsic parameters, including focal length, principal point coordinates, and distortion coefficients, are calibrated beforehand using the Zhang Zhengyou calibration method. At the grounding operation site, the camera is controlled to acquire a large number of raw images containing the grounding wire and grounding ring under various lighting conditions, such as sunny, cloudy, backlighting, and front lighting, at different shooting angles and distances, thus constructing a diverse image dataset. During acquisition, it is necessary to ensure that the grounding ring and the main body of the wire are located in the central area of ​​the image, and to record the approximate distance range between the camera and the target object, such as 0.5 meters to 2 meters, so that the spatial position of the target can be estimated later based on the image size.

[0024] Specifically, image preprocessing begins with dehazing. To address image degradation caused by potential fog, dust, or moisture in the original image, a dark channel prior dehazing algorithm is employed. This algorithm works as follows: For the input color image, the minimum values ​​of the red, green, and blue color channels within a local window of each pixel are calculated to obtain the dark channel image. The size of the local window is typically 15 x 15 pixels. Then, the top 0.1% of the brightest pixels in the dark channel image are selected, and the average brightness of the corresponding pixels in the original image is used as the atmospheric light value. This value is then used to estimate the red, green, and blue channels. Next, based on the physical model of foggy imaging, the transmittance of each pixel is initially estimated using the dark channel image and the atmospheric light value. A dehazing degree adjustment parameter is set, typically around 0.9. To eliminate the blocky effect generated in the initial estimation and preserve image edge details, a guided filter is used to refine the transmittance. Finally, based on the refined transmittance and atmospheric light value, the haze-free image is calculated, and a lower limit for transmittance is set to prevent calculation errors, thus obtaining the dehazed color image.

[0025] After dehazing, contrast enhancement is performed to improve the clarity of the edges of conductors and grounding rings. Specifically, a contrast-limited adaptive histogram equalization algorithm is used. This algorithm works as follows: First, the dehazed image is converted from the red-green-blue color space to a lightness, chroma, and hue color space, processing only the lightness channel to preserve color information. The lightness channel is divided into multiple equal-sized, non-overlapping sub-blocks, each typically 64 x 64 pixels. For each sub-block, its grayscale histogram is calculated, and a cropping threshold, typically set to three, is applied to suppress excessive noise amplification. The cropped histogram is then equalized, and bilinear interpolation is used to smooth the sub-block boundaries, resulting in the enhanced lightness channel. Finally, the enhanced lightness channel is recombine with the original chroma and hue channels and converted back to the red-green-blue color space to obtain the contrast-enhanced color image.

[0026] After image preprocessing, the image size needs to be normalized to meet the input requirements of subsequent deep learning networks. The enhanced color image is uniformly scaled to a preset resolution, such as 640 x 640 pixels or 800 x 800 pixels, using bicubic interpolation. Simultaneously, the image is kept in a three-channel color format, arranged in the order of red, green, and blue channels. Pixel values ​​are normalized to the range of zero to one, or standardized using the mean and standard deviation commonly used in image classification tasks, so that they can be directly input into subsequent multi-task deep learning networks. The final output is a high-quality RGB image after acquisition, dehazing, contrast enhancement, and size normalization, which is used as input data for subsequent wire segmentation and ground loop detection tasks.

[0027] S2: The RGB image is input into a multi-task deep learning network. This multi-task deep learning network uses the ConvNeXt-Tiny network as the backbone network to extract multi-scale features. The extracted multi-scale features are fused through the Bidirectional Feature Pyramid (BiFPN) module. The fused features are then enhanced through the dilated convolution module (ASPP) and the wire segmentation and ground loop detection tasks are performed. The spatial position of the wire and the pose information of the ground loop are output.

[0028] As a preferred embodiment of the present invention, in step S2, the structure of the ConvNeXt-Tiny network includes a sampling layer, a first-level convolutional next-generation network block, a second-level convolutional next-generation network block, a third-level convolutional next-generation network block, and a fourth-level convolutional next-generation network block. The first-level convolutional next-generation network block, the second-level convolutional next-generation network block, the third-level convolutional next-generation network block, and the fourth-level convolutional next-generation network block all include depthwise convolutional layers and pointwise convolutional layers. The sampling layer sequentially connects the four convolutional next-generation network blocks of different levels.

[0029] Specifically, the sampling layer uses a convolutional layer with a kernel size of 4×4 and a stride of 4 to process the input image. This sampling layer contains 96 convolutional kernels, reducing the resolution of the output feature map to 1 / 4 of the input, i.e., H / 4×W / 4, while expanding the number of channels to 96. This sampling aims to quickly reduce the spatial resolution and increase the channel dimension, preparing for subsequent deep feature extraction.

[0030] Specifically, each convolutional next-generation network block adopts the following structural design: First, the input feature map undergoes layer normalization. Then, a 7×7 depthwise convolutional layer (with the number of groups equal to the number of channels) is used for spatial feature extraction, which captures local spatial information while maintaining channel independence. Next, it undergoes layer normalization again. Then, it passes through two 1×1 pointwise convolutional layers (i.e., fully connected layers) in sequence. The first 1×1 convolutional layer expands the number of channels to 4 times the original number (i.e., the MLP ratio is set to 4), and the second 1×1 convolutional layer restores the number of channels to the original dimension. A Gaussian error linear unit is used for non-linear activation between the two pointwise convolutional layers. Finally, the input feature map is added to the feature map processed above through residual connections to obtain the output of the current network block.

[0031] As a preferred embodiment of the present invention, in step S2, multi-scale feature extraction using the ConvNeXt-Tiny network as the backbone network is divided into four stages. During the transfer between the four stages, the spatial resolution gradually decreases and the number of channels gradually increases. Specifically, the first stage uses three cascaded first-level convolutional next-generation network blocks, the second stage uses three cascaded second-level convolutional next-generation network blocks, the third stage uses nine cascaded third-level convolutional next-generation network blocks, and the fourth stage uses three cascaded fourth-level convolutional next-generation network blocks. The four stages output a first feature map, a second feature map, a third feature map, and a fourth feature map, respectively.

[0032] As a preferred embodiment of the present invention, the specific process of outputting the first feature map, the second feature map, the third feature map, and the fourth feature map in the four stages respectively includes: The RGB image is downsampled using a sampling layer to obtain the original feature map; The original feature map is input into the first stage, where spatial features are extracted through a deep convolutional layer. After layer normalization, it is then passed through two pointwise convolutional layers for channel expansion and feature mapping. The first pointwise convolutional layer receives the feature map processed by the deep convolutional layer, expands the number of channels of the feature map to four times the number of input channels, and then performs a non-linear transformation through a Gaussian error linear unit activation function before being input to the second pointwise convolutional layer. The second pointwise convolutional layer compresses the number of channels from the expanded dimension back to the original input dimension. Finally, the original input feature map is added to the feature map processed by the second pointwise convolutional layer through residual connections to obtain the output of the current first-level convolutional next-generation network block. After being processed by three first-level convolutional next-generation network blocks in sequence, the first stage outputs the first feature map. The first feature map is input into the second stage, and the second feature map is obtained by performing the same processing as the first stage. The second feature map is input into the third stage, and the same processing as in the first stage is performed to obtain the third feature map; The third feature map is input into the fourth stage, and the fourth feature map is obtained by performing the same processing as in the first stage.

[0033] Specifically, after the first stage of processing, the first feature map C2 is output, with a resolution of H / 4×W / 4 and 96 channels. Before entering the second stage, the resolution is reduced to H / 8×W / 8 and the number of channels is increased to 192 by a downsampling layer (usually a 2×2 convolutional layer with a stride of 2). Then, the next-generation network blocks of the second stage, which are cascaded in three second-level convolutional layers, perform feature processing and output the second feature map C2, which has a resolution of H / 8×W / 8 and a number of channels of 192, and contains rich details such as edges and textures.

[0034] Before entering the third stage, the sample is downsampled again to H / 16×W / 16, and the number of channels increases to 384. After being processed by the next-generation network blocks of the fourth-level convolutional layer in the third stage, the third feature map C3 is output, with a resolution of H / 16×W / 16 and a number of channels of 384, which contains mid-level semantic information.

[0035] Before entering the fourth stage, the sampled value is downsampled to H / 32×W / 32, and the number of channels is increased to 768. After being processed by the next-generation network blocks of the fourth-level convolutional layer in the fourth stage, the fourth feature map C4 is output, which has a resolution of H / 32×W / 32 and a number of channels of 768, and contains high-level semantic information.

[0036] In a preferred embodiment of the present invention, in step S2, the extracted multi-scale features are fused using a feature pyramid module BiFPN, and then the fused features are enhanced using a dilated convolution module ASPP. The feature pyramid module BiFPN consists of two modules, and the specific fusion process is as follows: The first feature map, the second feature map, the third feature map, and the fourth feature map are respectively input into the first bidirectional feature pyramid module BiFPN for a round of bidirectional feature fusion from top to bottom and from bottom to top; In the top-down path, semantic information is passed from the highest level feature to the lowest level feature, resulting in four intermediate feature maps. Based on the top-down path, a bottom-up path is executed on the four intermediate feature maps, passing detailed information from the lowest level feature to the highest level feature, resulting in four output feature maps. Using the four output feature maps as the starting point of the top-down path in the second bidirectional feature pyramid module BiFPN, the same principle as the first bidirectional feature pyramid module BiFPN is used for processing, and finally four hierarchical feature maps are obtained, namely the first-level feature map, the second-level feature map, the third-level feature map, and the fourth-level feature map.

[0037] As a preferred embodiment of the present invention, in step S2, the fused features are enhanced using a dilated convolution module (ASPP), specifically including: The fourth-level feature map is input into the dilated convolution module ASPP for feature enhancement. The dilated convolution module ASPP includes five parallel branches. The fourth-level feature map is input into the first branch, whose effective receptive field corresponds to a 3×3 region to capture its local contextual information. Specifically, the first branch is a convolutional layer with a kernel size of 1×1 and a stride of 1, containing 256 kernels. While maintaining the receptive field, this first branch performs a linear transformation and feature reorganization on the input feature map along the channel dimension, maintaining the output feature map resolution at H / 64×W / 64 and the number of channels at 256.

[0038] The second branch extends the effective receptive field to a 13×13 region, capturing contextual information within that region. Specifically, the second branch is a dilated convolutional layer with a kernel size of 3×3, a dilation rate of 6, and a stride of 1, containing 256 kernels. According to the principle of dilated convolution, when the dilation rate is 6, the effective receptive field of the kernel expands to 13×13, meaning that each output location can perceive information within a 13×13 range of the input feature map. The output feature map of this branch maintains a resolution of H / 64×W / 64 and has 256 channels, used to capture a moderate range of contextual information.

[0039] The third branch extends the effective receptive field to a 25×25 area, capturing contextual information within that area. Specifically, the third branch is a dilated convolutional layer with a kernel size of 3×3, a dilation rate of 12, and a stride of 1. When the dilation rate is 12, the effective receptive field of the convolutional kernel expands to 25×25. The output feature map of this branch maintains a resolution of H / 64×W / 64 and has 256 channels, which is used to capture a large range of contextual information.

[0040] The fourth branch extends the effective receptive field to a 37×37 area, capturing contextual information within that area. Specifically, the fourth branch is a dilated convolutional layer with a kernel size of 3×3, a dilation rate of 18, and a stride of 1. When the dilation rate is 18, the effective receptive field of the convolutional kernel expands to 37×37. The output feature map of this branch maintains a resolution of H / 64×W / 64 and has 256 channels to capture a wider range of contextual information.

[0041] The fifth branch performs a global average of the spatial dimensions of the fourth-level feature map to obtain a global context feature vector. This feature vector is then upsampled to the same spatial resolution as the input feature map using bilinear interpolation to obtain a global feature map with restored resolution. This global feature map is then transformed to obtain global context information. Specifically, the fifth branch is a global average pooling layer that performs a global average across the entire feature map's spatial dimensions, resulting in a 1×1×256 global context feature vector. Then, this feature vector is upsampled to the same spatial resolution as the input feature map (H / 64×W / 64) using bilinear interpolation, yielding a resolution-restored global feature map. Finally, this global feature map is input into a 1×1 convolutional layer with 256 kernels for channel-dimensional feature transformation, outputting a feature map with a resolution of H / 64×W / 64 and 256 channels. This is used to capture image-level global context information.

[0042] The information obtained from the five parallel branches is spliced ​​and compressed along the same channel dimension. The compressed fourth-level feature map is then added element-wise to the original fourth-level feature map to obtain the high-level feature.

[0043] In a preferred embodiment of the present invention, step S2, performing the wire segmentation task, specifically includes: The first-level feature map is input into the wire segmentation task header for processing. Specifically, the multi-channel features in the first-level feature map are compressed into a single channel to generate an original score map for each pixel belonging to the wire region. An upsampling operation is then used to restore the resolution of the original score map to that of the original feature map. Figure 1 Then, the score value is mapped to the probability value through the activation function, and finally the wire segmentation mask image with the same size as the original feature map is output. An RGB image is input into a monocular depth estimation network to generate a scene depth map corresponding to the image; Specifically, the monocular depth estimation network adopts an encoder-decoder architecture, such as the MiDaS network, which is pre-trained on a large-scale depth dataset to learn the mapping rules from a single RGB image to a scene depth map.

[0044] After the RGB image is input, it first undergoes multi-level convolution and downsampling operations in the encoder to progressively extract multi-scale depth features. Then, it undergoes upsampling and feature fusion operations in the decoder to progressively restore the spatial resolution. Finally, the output layer generates a scene depth map with the same resolution as the input image. Each pixel value in this depth map represents the depth distance of the corresponding scene point relative to the camera's optical center, in millimeters or meters. For the initial depth values ​​output by the monocular depth estimation network, necessary post-processing is performed based on the network's pre-training output characteristics. For example, disparity values ​​are converted into depth values, or the depth range is cropped and normalized to ensure the physical validity of the depth values.

[0045] Based on the wire segmentation mask, the depth values ​​of all pixels belonging to the wire region are extracted from the scene depth map, and the average depth value of the wire is calculated by averaging all the depth values. Specifically, by traversing each pixel in the wire segmentation mask, for pixels with a confidence level higher than a preset threshold (e.g., 0.5), it is determined that they belong to the wire region, and the depth value of the corresponding position is extracted from the scene depth map.

[0046] All depth values ​​belonging to the traverse region are grouped into a single depth value set. Considering the potential for outliers in the depth map (such as depth estimation errors or edge noise), median filtering or mean filtering is used to aggregate the depth value set. Median filtering is preferred; that is, the median value is taken after sorting the depth value set as the average depth value of the traverse. This average depth value represents the average distance of the traverse along the camera's optical axis.

[0047] The spatial position of the traverse in the camera coordinate system is calculated based on the average depth of the traverse.

[0048] Specifically, based on the pinhole camera imaging model, the pixel coordinates of the center point of the conductor region and the average depth value of the conductor are calculated by using the centroid of the conductor segmentation mask. Then, combined with the camera intrinsic parameter matrix obtained from the calibration, the three-dimensional spatial position of the conductor in the camera coordinate system can be calculated. The three-dimensional spatial position information of the conductor represents the spatial orientation of the conductor relative to the camera. After being transformed into the coordinate system of the grounding operation terminal actuator by the hand-eye transformation matrix, it is used to control the grounding operation terminal actuator to move in the direction of the conductor.

[0049] In a preferred embodiment of the present invention, step S2, which involves performing a grounding loop detection task, specifically includes: The high-level features are input into the ground loop detection task head for processing. After three convolutional layers, feature extraction is performed to obtain a deep feature map. Based on the deep feature map, bounding box regression and four predefined key points of the grounding loop are performed. The four key points include the left and right endpoints and the top and bottom vertices of the grounding loop, and the coordinate regression parameter map of the four key points corresponding to each spatial location is obtained. Based on the regression parameter map of the four key points, the two-dimensional coordinates of the regression parameter map of the four key points are obtained; Specifically, the bounding box regression employs a 1×1 convolutional layer with four kernels to perform pointwise convolution processing on the deep feature map. The output bounding box regression parameter map has the same spatial resolution as the deep feature map and four channels. The four channel values ​​at each spatial location represent the horizontal and vertical coordinate offsets of the center point of the corresponding preset anchor box, the width scaling factor, and the height scaling factor, respectively. These parameters, after decoding, yield the bounding box coordinates of the grounding loop in the image.

[0050] Specifically, the keypoint regression employs a 1×1 convolutional layer with 8 kernels to perform point-by-point convolution processing on the deep feature map. The output keypoint regression parameter map has the same spatial resolution as the deep feature map and 8 channels. The 8 channel values ​​at each spatial location represent the coordinate regression parameters of four predefined keypoints, where channels 1-2 correspond to the x and y coordinates of the left endpoint, channels 3-4 to the right endpoint, channels 5-6 to the top vertex, and channels 7-8 to the bottom vertex. The output parameters can be an offset relative to the anchor box center or directly regressed absolute pixel coordinates.

[0051] Based on the known standard dimensions, the two-dimensional coordinates of the four key points are mapped to the three-dimensional coordinates in the object coordinate system, and the rotation matrix and translation vector of the grounding loop relative to the camera are solved by the PnP algorithm (Perspective-n-Point). The pose information of the grounding loop is obtained based on the rotation matrix and translation vector.

[0052] Specifically, the solution process using the PnP algorithm is as follows: First, convert the 2D keypoint coordinates to normalized planar coordinates:

[0053]

[0054] in, These are the pixel coordinates of the key points. Principal point coordinates All are normalized focal lengths.

[0055] Then, a system of 3D-2D correspondence equations is constructed. For each key point, the projection relationship is as follows:

[0056] in, This is the depth scaling factor. These are the coordinates of a three-dimensional point in the object's coordinate system.

[0057] The 3D point is represented as a weighted sum of four control points. Then, the coordinates of the control points in the camera coordinate system are obtained by solving a system of linear equations. Finally, the rotation matrix is ​​recovered through singular value decomposition. Translation vector The rotation matrix The translation vector represents the attitude of the grounding loop relative to the camera. It characterizes the three-dimensional position of the grounding ring center relative to the camera optical center.

[0058] Based on the rotation matrix obtained from the solution Translation vector This allows for the acquisition of complete six-degree-of-freedom pose information of the grounding loop. Among these, the translation vector... The three components Represents the three-dimensional coordinates of the grounding ring center in the camera coordinate system; rotation matrix It can be further converted into Euler angles or expressed as a quaternion.

[0059] Specifically, the six-degree-of-freedom pose information characterizes the complete spatial state of the grounding ring relative to the camera, including position (3 degrees of freedom) and attitude (3 degrees of freedom). After being transformed into the coordinate system of the grounding operation terminal actuator by the calibrated hand-eye transformation matrix, it is used to control the movement of the grounding operation terminal actuator in the forward, backward, left, right, up, down, and rotation directions to achieve automatic docking with the grounding ring.

[0060] S3: Based on the spatial position of the conductor and the orientation information of the grounding ring, control the movement of the grounding operation terminal actuator to achieve automatic docking with the grounding ring.

[0061] In this embodiment, the grounding operation terminal actuator integrates a two-section four-degree-of-freedom telescopic structure and an end effector, and is installed on top of the lifting mechanism. The lifting mechanism is responsible for raising and lowering the grounding operation terminal actuator to the height of the grounding ring. The two-section four-degree-of-freedom telescopic structure is responsible for the precise alignment of the four degrees of freedom in the horizontal direction (front, back, left, and right). The end effector is responsible for inserting into the grounding ring and confirming the connection status through a pressure sensor.

[0062] As a preferred embodiment of the present invention, in step S3, the automatic docking is achieved by adjusting the grounding operation terminal actuator according to the spatial position of the conductor and the six-degree-of-freedom pose of the grounding ring, specifically including: Based on the grounding ring pose information, the position of the grounding ring in the coordinate system of the grounding operation terminal actuator is taken as the target position, and its pose is taken as the target pose. The approximation path of the grounding operation terminal actuator is then planned. The target position and target pose are input to the inverse kinematics module of the grounding operation terminal actuator. The motion angle and displacement of each joint are solved analytically to obtain the joint target value, which is then used as a control command and sent to the servo driver of each joint to drive the grounding operation terminal actuator to move towards the target pose according to the planned path. During the movement of the grounding operation terminal actuator, the current pose of the grounding ring is compared with the target pose in real time to calculate the pose error. A correction command is generated based on the pose error and superimposed on the basic motion command to form a closed-loop visual servo control. Specifically, after obtaining the target pose of the grounding loop, this target pose is input as the desired pose of the grounding operation end effector to the inverse kinematics module. The inverse kinematics module pre-stores the kinematic model of the grounding operation end effector, which describes the geometric mapping relationship between joint space variables and end effector pose.

[0063] The inverse kinematics module calculates the target joint variable values ​​that each driven joint should reach based on the real-time angle or displacement values ​​of each joint and the target pose using analytical or numerical iteration methods, including the target angle of each rotational joint and / or the target displacement of each translational joint.

[0064] After the solution is obtained, the target values ​​of each joint variable are converted into corresponding drive commands and sent to the servo drivers of each joint. The servo drivers drive the corresponding actuator motors to rotate according to the received commands, so that each joint moves in coordination according to the target values, thereby driving the end effector of the grounding operation terminal to approach the target pose from the current pose.

[0065] During the movement, the grounding ring continuously receives visual feedback on its current pose information, compares it with the target pose in real time, calculates the pose deviation, and generates a speed or position correction based on the deviation. This correction is then superimposed on the basic motion command of the inverse kinematics module to form a closed-loop control, which compensates for motion errors and environmental disturbances, until the grounding operation terminal actuator accurately reaches the target pose and completes the docking operation.

[0066] During the process of the grounding operation terminal actuator approaching and inserting the grounding ring, the pressure signal feedback from the pressure sensor is used to determine whether the docking is completed.

[0067] Specifically, during the process of the grounding operation end approaching and inserting the grounding ring, the pressure signal fed back by the pressure sensor installed on the end of the grounding operation terminal actuator is continuously monitored.

[0068] When the end of the grounding operation terminal actuator has not yet contacted the grounding ring, if the pressure feedback value measured by the pressure sensor is zero or below the preset contact threshold, it is determined that the current state is in the free movement stage, and the visual servo control mode continues to drive the grounding operation terminal to approach the target pose.

[0069] When the end of the grounding operation terminal actuator is inserted into the grounding ring and comes into contact with the inner wall of the grounding ring, the pressure sensor detects the contact pressure in real time. As the insertion depth increases, the contact area gradually increases, and the feedback value of the pressure sensor rises accordingly. The real-time collected pressure feedback value is compared with a preset pressure judgment threshold.

[0070] If the pressure feedback value continues to rise and reaches the preset contact judgment threshold, and the pressure feedback value remains stable for multiple consecutive control cycles (i.e., the fluctuation range is less than the preset stability judgment threshold), then it is determined that the grounding operation terminal actuator has been effectively inserted into the grounding ring, and the docking action is completed.

[0071] To further confirm the reliability of the connection, after the docking is completed, the grounding operation terminal actuator is controlled to perform a slight retraction or holding action, and the pressure feedback value is monitored to see if it changes accordingly. If the pressure feedback value decreases during retraction and remains stable during holding, the mechanical connection is finally confirmed to be established and reliably locked.

[0072] If the pressure feedback value exceeds the preset safety protection threshold during insertion, it is determined that the insertion resistance is too high or jamming has occurred. The drive will be stopped immediately and a fault alarm will be issued to avoid damage to the equipment or grounding ring.

[0073] After confirming that the docking is complete, a hold command is sent to the grounding operation terminal actuator to maintain the current position and to send a 'docking complete' signal back to the central control system, and then the grounding state holding phase begins.

[0074] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A machine vision-based method for grounding point positioning and automatic docking, characterized in that, The method includes the following steps: S1: Acquire grounding site images and perform image preprocessing to obtain RGB images; S2: Input the RGB image into the multi-task deep learning network. The multi-task deep learning network uses the ConvNeXt-Tiny network as the backbone network to extract multi-scale features. The extracted multi-scale features are fused through the Bidirectional Feature Pyramid (BiFPN) module. The fused features are then enhanced through the dilated convolution module (ASPP) and the wire segmentation and ground loop detection tasks are performed. The spatial position of the wire and the pose information of the ground loop are output. S3: Based on the spatial position of the conductor and the orientation information of the grounding ring, control the movement of the grounding operation terminal actuator to achieve automatic docking with the grounding ring.

2. The grounding point positioning and automatic docking method based on machine vision according to claim 1, characterized in that, In step S2, the structure of the ConvNeXt-Tiny network includes a sampling layer, a first-level convolutional next-generation network block, a second-level convolutional next-generation network block, a third-level convolutional next-generation network block, and a fourth-level convolutional next-generation network block. The first-level convolutional next-generation network block, the second-level convolutional next-generation network block, the third-level convolutional next-generation network block, and the fourth-level convolutional next-generation network block all include depthwise convolutional layers and pointwise convolutional layers. The sampling layer sequentially connects the four different levels of convolutional next-generation network blocks.

3. The grounding point positioning and automatic docking method based on machine vision according to claim 2, characterized in that, In step S2, multi-scale feature extraction using the ConvNeXt-Tiny network as the backbone network is divided into four stages. During the transition between the four stages, the spatial resolution gradually decreases while the number of channels gradually increases. Specifically, the first stage uses three cascaded first-level convolutional next-generation network blocks, the second stage uses three cascaded second-level convolutional next-generation network blocks, the third stage uses nine cascaded third-level convolutional next-generation network blocks, and the fourth stage uses three cascaded fourth-level convolutional next-generation network blocks. The four stages output a first feature map, a second feature map, a third feature map, and a fourth feature map, respectively.

4. The grounding point positioning and automatic docking method based on machine vision according to claim 3, characterized in that, The specific process of outputting the first feature map, second feature map, third feature map and fourth feature map in the four stages respectively includes: The RGB image is downsampled using a sampling layer to obtain the original feature map; The original feature map is input into the first stage, where spatial features are extracted through a deep convolutional layer. After normalization, it is then passed through two pointwise convolutional layers for channel expansion and feature mapping. The first pointwise convolutional layer receives the feature map processed by the deep convolutional layer, expands the number of channels of the feature map to four times the number of input channels, and then performs a non-linear transformation through a Gaussian error linear unit activation function before being input to the second pointwise convolutional layer. The second pointwise convolutional layer compresses the number of channels from the expanded dimension back to the original input dimension. The original input feature map is added to the feature map after the second pointwise convolutional layer through residual connection to obtain the output of the current first-level convolutional next-generation network block. After being processed by three first-level convolutional next-generation network blocks in sequence, the first stage outputs the first feature map. The first feature map is input into the second stage, and the second feature map is obtained by performing the same processing as the first stage. The second feature map is input into the third stage, and the same processing as in the first stage is performed to obtain the third feature map; The third feature map is input into the fourth stage, and the fourth feature map is obtained by performing the same processing as in the first stage.

5. The grounding point positioning and automatic docking method based on machine vision according to claim 4, characterized in that, In step S2, the extracted multi-scale features are fused using a feature pyramid module BiFPN, and then the fused features are enhanced using a dilated convolution module ASPP. There are two BiFPN modules, and the specific fusion process is as follows: The first feature map, the second feature map, the third feature map, and the fourth feature map are respectively input into the first bidirectional feature pyramid module BiFPN for a round of bidirectional feature fusion from top to bottom and from bottom to top; In the top-down path, semantic information is passed from the highest level feature to the lowest level feature, resulting in four intermediate feature maps. Based on the top-down path, a bottom-up path is executed on the four intermediate feature maps, passing detailed information from the lowest level feature to the highest level feature, resulting in four output feature maps. Using the four output feature maps as the starting point of the top-down path in the second bidirectional feature pyramid module BiFPN, the same principle as the first bidirectional feature pyramid module BiFPN is used for processing, and finally four hierarchical feature maps are obtained, namely the first-level feature map, the second-level feature map, the third-level feature map, and the fourth-level feature map.

6. The grounding point positioning and automatic docking method based on machine vision according to claim 1, characterized in that, In step S2, the fused features are enhanced using the dilated convolution module ASPP, specifically including: The fourth-level feature map is input into the dilated convolution module ASPP for feature enhancement. The dilated convolution module ASPP includes five parallel branches. The fourth-level feature map is input into the first branch, whose effective receptive field corresponds to a 3×3 region to capture its local contextual information. The second branch extends the effective receptive field to a 13×13 region, capturing contextual information within that region. The third branch extends the effective receptive field to a 25×25 area, capturing contextual information within that area. The fourth branch extends the effective receptive field to a 37×37 area, capturing contextual information within that area. The fifth branch performs a global average of the spatial dimensions of the fourth-level feature map to obtain a global context feature vector. This feature vector is then upsampled to the same spatial resolution as the input feature map using bilinear interpolation to obtain a global feature map with restored resolution. This global feature map is then transformed to obtain global context information. The information obtained from the five parallel branches is spliced ​​and compressed along the same channel dimension. The compressed fourth-level feature map is then added element-wise to the original fourth-level feature map to obtain the high-level feature.

7. The grounding point positioning and automatic docking method based on machine vision according to claim 5, characterized in that, In step S2, the wire segmentation task is performed, specifically including: The multi-channel features in the first-level feature map are compressed into a single channel to generate the original score map of each pixel belonging to the wire region. The resolution of the original score map is restored to the same level as the original feature map by upsampling, and then the score value is mapped to the probability value by activation function. Finally, the wire segmentation mask map with the same size as the original feature map is output. An RGB image is input into a monocular depth estimation network to generate a scene depth map corresponding to the image; Based on the wire segmentation mask, the depth values ​​of all pixels belonging to the wire region are extracted from the scene depth map, and the average depth value of the wire is calculated by averaging all the depth values. The spatial position of the traverse in the camera coordinate system is calculated based on the average depth of the traverse.

8. The grounding point positioning and automatic docking method based on machine vision according to claim 6, characterized in that, In step S2, the ground loop detection task is performed, specifically including: The high-level features are input into the ground loop detection task head for processing. After three convolutional layers, feature extraction is performed to obtain a deep feature map. Based on the deep feature map, bounding box regression and four predefined ground loop key points are performed to obtain the coordinate regression parameter map of the four key points corresponding to each spatial location. Based on the regression parameter map of the four key points, the two-dimensional coordinates of the regression parameter map of the four key points are obtained; Based on the known standard dimensions, the two-dimensional coordinates of the four key points are mapped to the three-dimensional coordinates in the object coordinate system, and the rotation matrix and translation vector of the grounding loop relative to the camera are solved by the PnP algorithm. The pose information of the grounding ring is obtained based on the rotation matrix and the translation vector.

9. The grounding point positioning and automatic docking method based on machine vision according to claim 1, characterized in that, In step S3, the specific process of controlling the movement of the grounding operation terminal actuator to achieve automatic docking with the grounding ring includes: Based on the grounding ring pose information, the position of the grounding ring in the coordinate system of the operating terminal actuator is taken as the target position, and its pose is taken as the target pose, and the approximation path of the grounding operating terminal actuator is planned. The target position and target pose are input to the inverse kinematics module of the grounding operation terminal actuator. The motion angle and displacement of each joint are solved analytically to obtain the joint target value and send it as a control command to the servo driver of each joint to drive the grounding operation terminal actuator to move towards the target pose according to the planned path. During the movement of the grounding operation terminal actuator, the current pose of the grounding ring is compared with the target pose in real time, the pose error is calculated, a correction command is generated based on the pose error, and it is superimposed on the basic motion command to form a closed-loop visual servo control. During the process of the grounding operation terminal actuator approaching and inserting the grounding ring, the pressure feedback signal is used to determine whether the docking is completed.

Images