Electric pole grabbing method and system based on visual identification

Abstract

The invention provides an electric pole grabbing method and system based on visual identification, and the method can be used for the construction of an overhead line of a distribution network, and comprises the steps: obtaining an image of a manual pole erecting process; carrying out noise reduction processing on the image by adopting a BM3D noise reduction algorithm; establishing a defect image data set according to the noise-reduced image, and training a convolutional neural network to obtain a training model; on the basis of the yolov5 and the training model, the clamping position of the electric pole is recognized and positioned; based on a binocular vision three-dimensional reconstruction method, determining a space coordinate of a gripping point of the electric pole; and on the basis of a visual SLAM algorithm, an obstacle avoidance path in the electric pole grabbing process is planned. By judging the posture of the electric pole, the electric pole is clamped in a self-adaptive mode, and grabbing of the electric pole can be accurately completed.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision and relates to a method and system for grasping electric poles based on visual recognition. Background Art

[0002] In the construction of overhead lines of distribution networks, poles have clear construction requirements and standards. For example, the standard center position of a 12-meter pole is 5.5 meters. The pole clamp accurately grasps the pole, adjusts the angle, and vertically inserts into the pole hole to ensure the verticality of the pole. In the grasping of the pole, a slewing mechanism is used to twist the pole. The traditional worm gear mechanism has a large center of gravity offset, which can easily cause the gripper reducer to overload and damage. At the same time, during the rotation process, the degree of clamping of the pole claw on the pole surface determines the stability and reliability of the pole during the grasping and vertical pole erection process. In the traditional on-site process, the pole is often not firmly grasped, resulting in slippage.

[0003] At present, the construction site of overhead lines mainly relies on human eyes to judge the grabbing position, which is limited by the experience of personnel. It is difficult to judge accurately, resulting in eccentricity and easy damage to the pole. Summary of the invention

[0004] In order to solve the above problems, it is necessary for the present invention to propose a method for positioning the pole gripping position based on visual recognition and adaptively controlling the robotic arm to perform obstacle avoidance path planning, thereby ensuring the stable use of the pole clamp and improving the service life of the main machine. At the same time, it can reduce pole damage and operational errors during construction and increase the pole erection accuracy rate to 99%.

[0005] Therefore, the present invention provides a pole grabbing method based on visual recognition, which is used in the construction of overhead lines of distribution networks, comprising:

[0006] Acquire images of the manual pole erection process;

[0007] Using the BM3D noise reduction algorithm to perform noise reduction on the image;

[0008] According to the image after the noise reduction process, a defect image data set is established and a convolutional neural network is trained according to the defect image data set to obtain a training model;

[0009] Based on yolov5 and the training model, the clamping position of the pole is identified and located;

[0010] Determining the spatial coordinates of the gripping point of the pole based on binocular vision 3D reconstruction method; and

[0011] Based on the visual SLAM algorithm, an obstacle avoidance path is planned during the pole grabbing process.

[0012] Further, including:

[0013] The image is divided into pixel blocks of size k*k, and a block with the smallest difference is searched in a domain of size n*n around a reference block and a three-dimensional matrix is ​​synthesized with the reference block. The formula of the search process is as follows:

[0014] G(P)={Q:d(P,Q)≤τ step};

[0015] Where d(P, Q) is the Euclidean distance between two different blocks; P is the reference block; Q is any surrounding block of the reference block; τ is the difference; step is the number of search steps;

[0016] Performing two-dimensional and one-dimensional transformations on the three-dimensional matrix and counting the number of non-zero components;

[0017] Putting the block with the smallest difference back to its original position after inverse transformation, using the number of non-zero components to count the weights, and dividing the weighted image by the weight of each point to obtain a basic estimated image; and

[0018] Repeat the above method to obtain all obstacle images.

[0019] Furthermore, the step of establishing a defect image dataset based on the image after the noise reduction process and training a convolutional neural network based on the defect image dataset to obtain a training model includes:

[0020] The defect images are manually annotated to establish a defect image dataset D containing m samples = {(x1, y1), (x2, y2), ..., (x m ,y m )}, and split D into training set and validation set in a ratio of 70%-90%.

[0021] Furthermore, the identifying and locating the clamping position of the pole based on yolov5 and the training model includes:

[0022] Divide the obstacle image into S*S grid cells, and each grid cell has B bounding boxes;

[0023] Processing and extracting image features through the training model;

[0024] Convert the parameters of the ground truth box to t x ,t y ,t w ,t h The corresponding g x , g y , g w , g h , and calculate the error; where,

[0025] g x =σ(t x )+c x ;

[0026] g y =σ(t y )+c y ;

[0027]

[0028] Where t x , t y , t w , t h They are the central horizontal coordinate, central vertical coordinate, width and height directly predicted by the bounding box; a w , a h The width and height of the anchor box are g x , g y , g w , g h are the predicted center horizontal coordinate, center vertical coordinate, width and height after conversion; c x , c y The coordinates of the upper left corner of the grid cell relative to the entire image;

[0029] Calculate confidence during anchor box selection:

[0030]

[0031] In the formula represents the confidence of the jth bounding box of the i-th grid cell; P r (·) is the probability function; Object represents the target box;

[0032] Non-maximum suppression is used to remove duplicate frames to obtain a true target frame; and based on the target frame, the clamping position of the pole is identified and located.

[0033] Further, including:

[0034] Normalize the samples in the training set and normalize the samples in the first layer z[ l] Perform normalization:

[0035]

[0036] Where u and σ are the mean values ​​of the samples, m is the number of samples, γ, β, ε are hyperparameters. β=u, then

[0037] Further, including:

[0038] Sort the candidate box list in descending order according to confidence;

[0039] Select the box A with the highest confidence and add it to the output list and delete it from the candidate box list;

[0040] Calculate the Intersection over Union (IOU) value between A and all boxes in the candidate box list:

[0041]

[0042] When IOU>0.5, the result is considered good, and candidate boxes larger than the threshold are deleted;

[0043] Repeat the above process until the candidate box list is empty and return the output list.

[0044] Further, including:

[0045] Calibrate cameras C1 and C2. The matrices of cameras C1 and C2 are M1 and M2 respectively. Then:

[0046]

[0047] Where (u1, v1, 1) and (u2, v2, 1) are the image homogeneous coordinates of points P1 and P2 in their respective images; Z c1 and Z c2 is the intermediate parameter; (X, Y, Z, 1) is the homogeneous coordinate of point P in the world coordinate system; M k The i-th row and j-th column element of , k represents the coordinate solution coefficient;

[0048] Eliminate Z c1 and Z c2 We get four linear equations about X, Y, and Z:

[0049]

[0050] The spatial coordinates (X, Y, Z) of the gripping point of the pole are obtained using the least square method.

[0051] Furthermore, the planning of the obstacle avoidance path in the pole grabbing process based on the visual SLAM algorithm includes:

[0052] Acquire data information in the environment through the camera;

[0053] The position of the camera is calculated by the data information in the front-end visual odometer and a closed-loop detection is performed to determine whether the pole grabbing robot has reached the place it has passed before;

[0054] Using back-end nonlinear optimization, the position and posture of the camera obtained by the front-end visual odometer are optimized to obtain a global optimal state; and

[0055] According to the position of the camera at each moment and the information of the target in space, a corresponding map is established to obtain surrounding information and judge the corresponding obstacles before the pole grasping robot moves, so as to plan the path of the pole grasping robot and find the optimal forward route.

[0056] In addition, the present application also provides a system for applying the above-mentioned electric pole grasping method based on visual recognition, comprising:

[0057] An acquisition module, used for acquiring images of a manual pole erection process;

[0058] A noise reduction processing module, used for performing noise reduction processing on the image using a BM3D noise reduction algorithm;

[0059] A data processing module, used to establish a defect image data set according to the image after the noise reduction process and train a convolutional neural network according to the defect image data set to obtain a training model;

[0060] An identification and positioning module, used for identifying and positioning the clamping position of the pole based on yolov5 and the training model;

[0061] A coordinate determination module, used to determine the spatial coordinates of the gripping point of the pole based on a binocular vision three-dimensional reconstruction method; and

[0062] The path planning module is used to plan the obstacle avoidance path in the process of grasping the electric pole based on the visual SLAM algorithm.

[0063] The beneficial effects of the present invention are as follows: The present invention provides a method for adaptively controlling pole clamping by guiding the grasping of poles based on visual recognition of pole parameters and judging the posture of poles, which can solve the problem that the current on-site construction of overhead lines mainly relies on the human eye to determine the grasping position, which is limited by personnel experience and is difficult to accurately determine, resulting in eccentricity and easy damage to poles. The present invention uses the BM3D noise reduction algorithm to perform noise reduction on the collected construction pictures, and then establishes a defect image data set and trains the convolutional neural network to obtain a training model, and then uses yolov5 and combines the training model to identify and locate the pole clamping position, and finally determines the spatial coordinates of the grasping point based on the binocular vision three-dimensional reconstruction method, thereby completing the determination of the pole clamping position. Then, the visual SLAM algorithm is used to plan the obstacle avoidance path in the process of the robot arm grasping the pole. Finally, the pole grasping is completed in the most adaptive and accurate way. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is a flow chart of the electric pole grabbing method of the present invention;

[0065] Figure 2 It is the Anchor frame generated by the obstacle image;

[0066] Figure 3 It is the basic model for 3D reconstruction of a spatial point P;

[0067] Figure 4 is a schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0068] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0069] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms and the like is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device, element, module, system, platform or device referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. The following description of the present invention is only understood as a description of individual embodiments of the technical solution of the present invention. Other embodiments are not reflected in the following description, but it does not mean that the present invention excludes these other embodiments, and the technical solution of the present invention is not limited to the specific implementation methods described below, and the protection scope of the present invention is not limited to only the specific implementation methods described below. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should belong to the scope of protection of the present invention.

[0070] It should be noted that if the terms "first", "second", etc. appear in the specification and claims of the present invention and the above-mentioned drawings, the description is only used to distinguish similar objects, and is not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0071] The present invention is further described in detail below with reference to the accompanying drawings and specific examples:

[0072] The invention provides a pole grabbing method based on visual recognition, and the pole grabbing method is used for the construction of overhead lines of distribution networks.

[0073] In some embodiments, Figure 1 As shown, the electric pole grabbing method based on visual recognition includes:

[0074] S1: Acquire images of the manual pole erection process;

[0075] Specifically, the present invention is based on the pole clamping scenario in the construction of overhead lines in a distribution network. It demonstrates the manual pole erection process by manually grabbing the poles, and simulates various obstacles to collect image data during the demonstration.

[0076] S2: Using BM3D noise reduction algorithm to perform noise reduction on the image;

[0077] S3: establishing a defect image dataset according to the image after the noise reduction process and training a convolutional neural network according to the defect image dataset to obtain a training model;

[0078] S4: Based on yolov5 and the training model, the clamping position of the pole is identified and located;

[0079] S5: Determine the spatial coordinates of the gripping point of the pole based on binocular vision 3D reconstruction method; and

[0080] S6: Based on the visual SLAM algorithm, plan an obstacle avoidance path during the pole grabbing process.

[0081] In some embodiments, step S2 includes:

[0082] The image is divided into pixel blocks of size k*k, and the block with the smallest difference is searched in the n*n domain around the reference block and integrated with the reference block into a three-dimensional matrix; the search process formula is as follows:

[0083] G(P)={Q:d(P,Q)≤τ step}

[0084] Where d(P, Q) is the Euclidean distance between two different blocks; P is the reference block; Q is any surrounding block of the reference block; τ is the difference; step is the number of search steps;

[0085] Specifically, all obstacle images are divided into small pixel blocks of 8*8 in size, and the block with the smallest difference is searched in a 21*21 domain around the reference block and integrated with the reference block into a three-dimensional matrix.

[0086] Performing two-dimensional and one-dimensional transformations on the three-dimensional matrix and counting the number of non-zero components;

[0087] After inverse transformation, the block with the smallest difference is put back to its original position, weights are superimposed by using the number of non-zero components, and the weighted image is divided by the weight of each point to obtain a basic estimated image;

[0088] Use the Euclidean distance to measure the similarity and sort the distances from small to large and then take the first MAXN1;

[0089] Specifically, the Euclidean distance is used to measure the similarity and the first 16 are selected after sorting the distances from small to large.

[0090] Stack the image blocks and the image blocks of the noisy original image into two three-dimensional arrays respectively;

[0091] Performing two-dimensional and one-dimensional transformations on the two three-dimensional matrices, and scaling the coefficients of the three-dimensional matrix formed by the noise image using Wiener filtering; and

[0092] Repeat the above method to obtain all obstacle images.

[0093] The process of coefficient scaling may be as follows:

[0094]

[0095] Where ω p is the coefficient of Wiener filter; T 3Dhard is a three-dimensional hard threshold transformation; T 3Dwein is the three-dimensional Wiener transform.

[0096] In some embodiments, step S3 includes:

[0097] The defect images are manually annotated to establish a defect image dataset D containing m samples = {(x1, y1), (x2, y2), ..., (x m ,y m )}, and split D into training set and validation set in a ratio of 70%-90%.

[0098] Specifically, the defect images can be manually annotated to establish a defect image dataset D containing 100,000 samples, and D is divided into a training set at a ratio of 80% (a total of 80,000) and a validation set at a ratio of 20% (a total of 20,000).

[0099] In some embodiments, step S4 includes:

[0100] Divide the obstacle image into S*S grid cells, and each grid cell has B bounding boxes;

[0101] Processing and extracting image features through the training model;

[0102] Convert the parameters of the ground truth box to t x ,t y ,t w ,t h The corresponding g x , g y , g w , g h , and calculate the error; where,

[0103] g x =σ(t x )+c x ;

[0104] g y =σ(t y )+c y ;

[0105]

[0106] Where t x , t y , t w , t h are the center horizontal coordinate, center vertical coordinate, width and height directly predicted by the bounding box; a w , a h The width and height of the anchor box are g x , g y , g w , g h are the predicted center horizontal coordinate, center vertical coordinate, width and height after conversion; c x , c y The coordinates of the upper left corner of the grid cell relative to the entire image;

[0107] Calculate confidence during anchor box selection:

[0108]

[0109] In the formula represents the confidence of the jth bounding box of the i-th grid cell; P r (·) is the probability function; Object represents the target box;

[0110] Non-maximum suppression is used to remove duplicate frames to obtain a true target frame; and based on the target frame, the clamping position of the pole is identified and located.

[0111] In some embodiments, the processing and extracting image features through the training model includes:

[0112] Normalize the samples in the training set and [l] Perform normalization:

[0113]

[0114] Where u and σ are the mean values ​​of the samples, m is the number of samples, γ, β, ε are hyperparameters. β=u, then

[0115] Furthermore, the use of non-maximum suppression to remove duplicate frames and obtain the true target frame includes:

[0116] Sort the candidate box list in descending order according to confidence;

[0117] Select the box A with the highest confidence and add it to the output list and delete it from the candidate box list;

[0118] Calculate the Intersection over Union (IOU) value between A and all boxes in the candidate box list:

[0119]

[0120] When IOU>0.5, the result can be considered good, and candidate boxes larger than the threshold are deleted;

[0121] Repeat the above process until the candidate box list is empty and return the output list.

[0122] Specifically, the above steps can divide the obstacle image into 80*80 grid units with 9 anchor boxes, such as Figure 2 As shown. Therefore, each grid unit has 9 bounding boxes. Using an 8-layer convolutional neural network, first normalize each layer of data as follows:

[0123]

[0124] Then, by replacing activation functions such as Relu, leakrelu, and elu, and using the ResNet residual structure, the entire network is "fine-tuned" during pre-training, and gradient clipping and weight regularization methods are used to process and extract obstacle image features, including t x ,t y ,t w ,t h Etc. Convert the parameters of the ground truth box to t x ,t y ,t w ,t h The corresponding g x , gy , g w , g h , and then calculate the error. As shown in the following formula:

[0125] g x =σ(t x )+c x

[0126] g y =σ(t y )+c y

[0127]

[0128] Calculate confidence during anchor box selection:

[0129]

[0130] Next, we first set the confidence threshold of the target box to 0.5, then sort the candidate box list in descending order according to the confidence, select the box with the highest confidence again and add it to the output list and delete it from the candidate box list, and calculate the IOU value between the current box and all the boxes in the candidate box list again. Delete the candidate box when IOU>0.5, and repeat the above process until the candidate box list is empty and return to the output list.

[0131] In some embodiments, step S5 includes:

[0132] Calibrate cameras C1 and C2. The matrices of cameras C1 and C2 are M1 and M2 respectively. Then:

[0133]

[0134] Where (u1, v1, 1) and (u2, v2, 1) are the image homogeneous coordinates of points P1 and P2 in their respective images; Z c1 and Z c2 is the intermediate parameter; (X, Y, Z, 1) is the homogeneous coordinate of point P in the world coordinate system; M k The i-th row and j-th column element of , k represents the coordinate solution coefficient;

[0135] Eliminate Z c1 and Z c2 We get four linear equations about X, Y, and Z:

[0136]

[0137]

[0138] The spatial coordinates (X, Y, Z) of the gripping point of the pole are obtained using the least square method.

[0139] In some embodiments, step S6 includes:

[0140] Acquire data information in the environment through the camera;

[0141] The position of the camera is calculated by the data information in the front-end visual odometer and a closed-loop detection is performed to determine whether the pole grabbing robot has reached the place it has passed before;

[0142] Using back-end nonlinear optimization, the position and posture of the camera obtained by the front-end visual odometer are optimized to obtain a global optimal state; and

[0143] According to the position of the camera at each moment and the information of the target in space, a corresponding map is established to obtain surrounding information and judge the corresponding obstacles before the pole grasping robot moves, so as to plan the path of the pole grasping robot and find the optimal forward route.

[0144] Specifically, first, the camera on the grasping robot arm obtains data information in the environment, that is, images frame by frame. The position of the camera is calculated through these image information in the front-end visual odometer, and the bag-of-words model is used for closed-loop detection to determine whether the grasping robot arm has reached the place it has passed before. Then, the back-end nonlinear optimization is used to optimize the camera posture obtained by the front end to obtain the global optimal state. Finally, according to the camera posture at each moment and the information of the target in space, the corresponding grid map is established according to the needs, so that before the grasping robot arm moves, the surrounding information has been obtained through the background, the corresponding obstacles are judged, warnings are given, and the optimal path for the grasping robot arm to move is found.

[0145] In some embodiments, Figure 4 As shown, the present invention also provides a system 1 for applying the above-mentioned electric pole grasping method based on visual recognition, comprising:

[0146] An acquisition module 11 is used to acquire images of a manual pole erection process;

[0147] A noise reduction processing module 12 is used to perform noise reduction processing on the image using a BM3D noise reduction algorithm;

[0148] A data processing module 13, used to establish a defect image data set according to the image after the noise reduction process and train a convolutional neural network according to the defect image data set to obtain a training model;

[0149] An identification and positioning module 14, used for identifying and positioning the clamping position of the pole based on yolov5 and the training model;

[0150] A coordinate determination module 15, for determining the spatial coordinates of the gripping point of the pole based on a binocular vision three-dimensional reconstruction method; and

[0151] The path planning module 16 is used to plan an obstacle avoidance path during the pole grabbing process based on a visual SLAM algorithm.

[0152] Optionally, specific examples in the present invention may refer to the examples described in the above embodiments and optional implementation modes.

[0153] The specific embodiments of the present invention described above are for description only and do not represent the advantages or disadvantages of the embodiments.

[0154] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0155] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the system embodiments described above are only exemplary, for example, multiple devices can be combined or integrated into another system, or some features can be ignored or not executed.

[0156] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A pole grabbing method based on visual recognition, the pole grabbing method is used for distribution network overhead line construction, characterized in that: include: Acquire images of the manual pole erection process; Using the BM3D noise reduction algorithm to perform noise reduction on the image; According to the image after the noise reduction process, a defect image data set is established and a convolutional neural network is trained according to the defect image data set to obtain a training model; Based on yolov5 and the training model, the clamping position of the pole is identified and located; Determine the spatial coordinates of the gripping point of the pole based on binocular vision 3D reconstruction method; as well as Based on the visual SLAM algorithm, an obstacle avoidance path is planned during the pole grabbing process.

2. The electric pole grabbing method based on visual recognition according to claim 1, characterized in that: The adopting the BM3D noise reduction algorithm to perform noise reduction processing on the image includes: The image is divided into pixel blocks of size k*k, and a block with the smallest difference is searched in a domain of size n*n around a reference block and a three-dimensional matrix is ​​synthesized with the reference block. The formula of the search process is as follows: G(P)={Q:d(p,Q)≤τ step }; Where d(P, Q) is the Euclidean distance between two different blocks; P is the reference block; Q is any surrounding block of the reference block; τ is the difference; step is the number of search steps; Performing two-dimensional and one-dimensional transformations on the three-dimensional matrix and counting the number of non-zero components; Putting the block with the smallest difference back to its original position after inverse transformation, using the number of non-zero components to count the weights, and dividing the weighted image by the weight of each point to obtain a basic estimated image; and Repeat the above method to obtain all obstacle images.

3. The electric pole grabbing method based on visual recognition according to claim 2, characterized in that: The step of establishing a defect image dataset based on the image after the noise reduction process and training a convolutional neural network based on the defect image dataset to obtain a training model includes: The defect images are manually annotated to establish a defect image dataset D containing m samples = {(x1, y1), (x2, y2), ..., (x m ,y m )}, and divide D into training set and validation set in the ratio of 70%-90%.

4. The electric pole grabbing method based on visual recognition according to claim 3, characterized in that: The identifying and locating the clamping position of the pole based on yolov5 and the training model includes: Divide the obstacle image into S*S grid cells, and each grid cell has B bounding boxes; Processing and extracting image features through the training model; Convert the parameters of the ground truth box to t x ,t y ,t w ,t h The corresponding g x , g y , g w , g h , and calculate the error; where, g x =σ(t x )+c x ; g y =σ(t y )+c y ; Where t x ,t y ,t w ,t h They are the central horizontal coordinate, central vertical coordinate, width and height directly predicted by the bounding box; a w ,a h are the width and height of the anchor box respectively; g x ,g y ,g w ,g h are the predicted center horizontal coordinate, center vertical coordinate, width and height after conversion; c x ,c y The coordinates of the upper left corner of the grid cell relative to the entire image; Calculate confidence during anchor box selection: In the formula represents the confidence of the jth bounding box of the i-th grid cell; P r (·) is the probability function; Object represents the target box; Non-maximum suppression is used to remove duplicate frames to obtain a true target frame; and based on the target frame, the clamping position of the pole is identified and located.

5. The electric pole grabbing method based on visual recognition according to claim 4, characterized in that: The step of processing and extracting image features through the training model includes: Normalize the samples in the training set and [l] Perform normalization: Where u and σ are the mean values ​​of the samples, m is the number of samples, γ, β, ε are hyperparameters. β=u, then 6. The electric pole grabbing method based on visual recognition according to claim 5, characterized in that: The method of using non-maximum suppression to remove duplicate frames and obtain the true target frame includes: Sort the candidate box list in descending order according to confidence; Select the box A with the highest confidence and add it to the output list and delete it from the candidate box list; Calculate the Intersection over Union (IOU) value between A and all boxes in the candidate box list: When IOU>0.5, the result is considered good, and candidate boxes larger than the threshold are deleted; Repeat the above process until the candidate box list is empty and return the output list.

7. The electric pole grabbing method based on visual recognition as claimed in claim 6, characterized in that: The method of determining the spatial coordinates of the gripping point of the pole based on binocular vision three-dimensional reconstruction includes: Calibrate cameras C1 and C2. The matrices of cameras C1 and C2 are M 1 With M 2 ,but: Where (u1, v1, 1) and (u2, v2, 1) are the image homogeneous coordinates of points P1 and P2 in their respective images; Z c1 and Z c2 is the intermediate parameter; (X, Y, Z, 1) is the homogeneous coordinate of point P in the world coordinate system; M k The i-th row and j-th column element of , k represents the coordinate solution coefficient; Eliminate Z c1 and Z c2 We get four linear equations about X, Y, and Z: The spatial coordinates (X, Y, Z) of the gripping point of the pole are obtained using the least square method.

8. The electric pole grabbing method based on visual recognition according to claim 7, characterized in that: The planning of the obstacle avoidance path in the pole grabbing process based on the visual SLAM algorithm includes: Acquire data information in the environment through the camera; The position of the camera is calculated by the data information in the front-end visual odometer and a closed-loop detection is performed to determine whether the pole grabbing robot has reached the place it has passed before; Using back-end nonlinear optimization, the position and posture of the camera obtained by the front-end visual odometer are optimized to obtain a global optimal state; and According to the position of the camera at each moment and the information of the target in space, a corresponding map is established to obtain surrounding information and judge the corresponding obstacles before the pole grasping robot moves, so as to plan the path of the pole grasping robot and find the optimal forward route.

9. A system using the electric pole grabbing method based on visual recognition as described in any one of claims 1 to 8, characterized in that: include: An acquisition module, used for acquiring images of a manual pole erection process; A noise reduction processing module, used for performing noise reduction processing on the image using a BM3D noise reduction algorithm; A data processing module, used to establish a defect image data set according to the image after the noise reduction process and train a convolutional neural network according to the defect image data set to obtain a training model; An identification and positioning module, used for identifying and positioning the clamping position of the pole based on yolov5 and the training model; A coordinate determination module, used to determine the spatial coordinates of the gripping point of the pole based on a binocular vision three-dimensional reconstruction method; as well as The path planning module is used to plan the obstacle avoidance path in the process of grasping the electric pole based on the visual SLAM algorithm.

Images