Wire strain clamp three-dimensional pose identification method and system based on structured light

Abstract

The invention discloses a lead strain clamp three-dimensional pose identification method and system based on structured light, and the method comprises the steps: obtaining a multi-view structured light image, and carrying out the preprocessing of the structured light image, and obtaining a target structured light image; carrying out point cloud reconstruction according to the target structured light image, then carrying out denoising and abnormal value removal on the reconstructed three-dimensional point cloud, carrying out point cloud segmentation according to geometric features and spatial distribution of the point cloud, and obtaining a point cloud cluster after segmentation; selecting one point cloud in the point cloud cluster as a reference point cloud, and performing point cloud registration on other point clouds and the reference point cloud to obtain a registered point cloud cluster; and for the registered point cloud clusters, fitting a pre-established strain clamp three-dimensional model with the point cloud data by adopting a model-based pose estimation method, and determining a three-dimensional pose estimation result of each strain clamp. According to the invention, the problem of three-dimensional pose measurement of the strain clamp in a complex environment is effectively solved, and high-precision and high-robustness measurement effects are realized.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional posture recognition, and in particular relates to a three-dimensional posture recognition method and system for a conductor tension clamp based on structured light. Background Art

[0002] In the transmission line inspection scenario, the three-dimensional pose recognition of multiple tension clamps based on structured light faces many challenges. The surface of the tension clamp usually presents irregular geometric shapes and rough metal materials, which makes the structured light stripes prone to deformation and breakage, thus affecting the accuracy of three-dimensional reconstruction. In addition, since there are usually small parts such as bolts and gaskets on the tension clamp, these parts will produce complex shadows and highlights under structured light projection, interfering with the structured light decoding and point cloud matching process.

[0003] Furthermore, in complex power scenarios, multiple tension clamps often block each other and partially overlap. In this case, the structured light information of some tension clamps will be blocked by other clamps, resulting in missing point cloud data and difficulty in completely reconstructing the 3D model of the blocked clamps. Even if some point clouds can be reconstructed, the incompleteness of the point cloud data will affect the accuracy and robustness of the pose estimation. The material and structural characteristics of the tension clamps themselves exacerbate this problem. Metal reflections can cause overexposure in some areas, resulting in loss of structured light information.

[0004] In the case of overlap, the structured light information obtained from a single perspective is difficult to distinguish which tension clamp the overlapping area belongs to. This makes point cloud segmentation extremely difficult, and it is difficult to separate the point cloud data belonging to different clamps, making it impossible to independently estimate the pose of each clamp. Even if segmentation can be performed through some algorithms, the accuracy of the segmentation is difficult to guarantee, which will eventually affect the subsequent pose recognition process. At the same time, the structured light information of the overlapping parts will be mixed and superimposed, which further increases the difficulty of point cloud processing and pose estimation, making the results of 3D reconstruction even more difficult to interpret. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method for three-dimensional posture recognition of a conductor tension clamp based on structured light, comprising:

[0006] Acquire multi-view structured light images, and preprocess the structured light images to obtain target structured light images;

[0007] Reconstructing a point cloud according to the target structured light image, then denoising and removing outliers on the reconstructed three-dimensional point cloud, and segmenting the point cloud according to the geometric features and spatial distribution of the point cloud, and obtaining point cloud clusters after segmentation;

[0008] Selecting a point cloud in the point cloud cluster as a reference point cloud, performing point cloud registration on other point clouds and the reference point cloud to obtain a registered point cloud cluster;

[0009] For the registered point cloud cluster, a model-based pose estimation method is used to fit the pre-established three-dimensional model of the tension clamp with the point cloud data to determine the three-dimensional pose estimation result of each tension clamp.

[0010] Preferably, the process of acquiring multi-view structured light images and preprocessing the structured light images includes:

[0011] The structured light camera is used to obtain structured light images covering all surfaces of the tension clamp from different angles. For each structured light image obtained, noise is removed, distortion is corrected, and fringe contrast is enhanced. If there is an overexposed area in the image, the brightness of the overexposed area is adjusted to restore the structured light information.

[0012] Preferably, the process of acquiring structured light images covering various surfaces of the tension clamp from different angles by using a structured light camera includes:

[0013] According to the multi-camera calibration parameters, the camera posture in the three-dimensional space is reconstructed to obtain the relative position relationship between the cameras;

[0014] The structured light images of the tension clamps taken by each structured light camera are obtained, the deformed fringe information is calculated, and the phase value of each pixel is obtained by the phase unwrapping algorithm;

[0015] The system calibration parameters are used in combination with the phase value to calculate the three-dimensional point cloud coordinates corresponding to each pixel to obtain single-camera point cloud data; if there are holes in the single-camera point cloud, a point cloud completion operation is performed to generate a complete single-camera point cloud;

[0016] According to the camera posture transformation matrix, all single-camera point cloud data are converted to a unified world coordinate system to obtain multi-view point cloud data;

[0017] Through the point cloud registration algorithm, the multi-view point cloud data are aligned and fused to generate the final three-dimensional point cloud model of the tension clamp; if the registration error exceeds the preset threshold, the camera posture transformation matrix is ​​recalculated and registered again. After obtaining the final three-dimensional point cloud model of the tension clamp, the point cloud normal vector is calculated to determine whether there is a hole area on the point cloud surface and obtain the point cloud hole information; according to the point cloud hole information, the structured light camera posture parameters are adjusted or a new camera perspective is added, and the structured light image is collected again.

[0018] Preferably, the process of removing noise, correcting distortion, and enhancing fringe contrast for each acquired structured light image includes:

[0019] A bilateral filtering algorithm is used to perform denoising on the acquired structured light image to obtain a first denoised image;

[0020] According to the pre-calibrated camera parameters, a polynomial fitting method is used to perform distortion correction processing on the first denoised image to obtain a first corrected image;

[0021] Performing contrast enhancement processing on the first corrected image by using an adaptive histogram equalization method to obtain a second corrected image;

[0022] Calculating the pixel brightness value of the second corrected image, and if the pixel brightness value is greater than a preset threshold, determining that the pixel belongs to an overexposed area, and obtaining an overexposed area mask;

[0023] According to the overexposed area mask, the pixel brightness of the overexposed area in the second corrected image is adjusted.

[0024] Preferably, the process of adjusting the pixel brightness of the overexposed area in the second corrected image includes:

[0025] Select adjacent pixels to the pixels in the overexposed area from the non-overexposed area of ​​the second corrected image, calculate the brightness mean of the adjacent pixels, assign the brightness mean to the pixels in the overexposed area, restore the structured light information, and obtain the final corrected image.

[0026] Preferably, the process of reconstructing a point cloud according to the target structured light image includes:

[0027] A multi-perspective geometric reconstruction algorithm is used to integrate the information of multi-perspective structured light images to reconstruct the three-dimensional point cloud model of the tension clamp. If the structured light stripes at a certain perspective are deformed or broken, the weight of the perspective point cloud data is reduced as needed, and point cloud data with complete quality is used first for model reconstruction.

[0028] Preferably, the process of denoising and removing outliers on the reconstructed three-dimensional point cloud and segmenting the point cloud according to the geometric features and spatial distribution of the point cloud includes:

[0029] The reconstructed three-dimensional point cloud is denoised and outliers are removed. Noise points and outliers caused by metal reflections, complex shadows and highlight areas are removed. Statistical filters are used to remove outliers and retain the real point cloud data. Then, the point cloud data belonging to different tension clamps are segmented according to the geometric characteristics and spatial distribution of the point cloud. If two tension clamps overlap, they are segmented according to the curvature change and normal vector difference in the overlapping area. If the curvature change and normal vector difference are less than the preset threshold, the overlapping area is divided into a point cloud cluster with a larger area.

[0030] Preferably, the process of denoising and removing outliers from the reconstructed three-dimensional point cloud includes:

[0031] Obtain the reconstructed 3D point cloud data; use the local surface fitting algorithm to fit a local plane based on the neighborhood points around each point, and calculate the normal vector of each point;

[0032] The curvature value of each point is calculated according to the normal vector of each point and its neighboring points. If the curvature value is greater than a preset threshold, the point is judged to be located in a high curvature area and is a potential noise point or an abnormal point.

[0033] Construct a KD tree with 3D point cloud data to accelerate the search for neighboring points. For each point, search for neighboring points within a specified radius in the KD tree and calculate the average distance from the neighboring points to the point. If the average distance is greater than the preset threshold, the point is judged as an outlier.

[0034] The outliers are removed from the original point cloud data to obtain the point cloud data after preliminary denoising; the point cloud data after preliminary denoising is subjected to secondary statistical filtering to remove the remaining noise points and abnormal points to obtain the final denoised real point cloud data.

[0035] Preferably, a process of selecting a point cloud in the point cloud cluster as a reference point cloud and performing point cloud registration on other point clouds with the reference point cloud comprises:

[0036] For each point cloud cluster after segmentation, one of the point clouds is selected as the reference point cloud, and the other point clouds are aligned with the reference point cloud. All point cloud data are unified into the same coordinate system, and the iterative closest point algorithm is used for point cloud alignment, and the point-to-surface distance is used as the metric for the alignment error.

[0037] The present invention also provides a three-dimensional position recognition system for a conductor tension clamp based on structured light, comprising:

[0038] A multi-view image acquisition module, used to acquire multi-view structured light images;

[0039] A structured light image preprocessing module, used for preprocessing the structured light image to obtain a target structured light image;

[0040] A point cloud reconstruction and denoising module, used to reconstruct the point cloud according to the target structured light image, then denoise and remove outliers on the reconstructed three-dimensional point cloud, and segment the point cloud according to the geometric features and spatial distribution of the point cloud, and obtain point cloud clusters after segmentation;

[0041] A point cloud registration module, used for selecting a point cloud in the point cloud cluster as a reference point cloud, performing point cloud registration on other point clouds and the reference point cloud to obtain a registered point cloud cluster;

[0042] The three-dimensional model fitting module is used to fit the pre-established three-dimensional model of the tension clamp with the point cloud data using a model-based pose estimation method for the registered point cloud cluster, so as to determine the three-dimensional pose estimation result of each tension clamp.

[0043] Compared with the prior art, the present invention has the following advantages and technical effects:

[0044] The present invention discloses a method and system for recognizing the three-dimensional posture of a conductor tension clamp based on structured light, aiming to solve the problem that multiple tension clamps in complex scenes block each other and the surface reflect strongly, which makes it difficult for traditional measurement methods to accurately obtain their three-dimensional posture. The present invention adopts a multi-view structured light camera to obtain the complete point cloud data of the tension clamp, and through pre-processing steps such as removing noise, correcting distortion and enhancing fringe contrast, and dynamically adjusting the point cloud weight according to the fringe quality, to reduce the influence of factors such as metal reflection and shadow on point cloud reconstruction. In view of the possible overlap problem of multiple tension clamps, the present invention uses curvature changes and normal vector differences to segment the point cloud, and divides the overlapping area into a larger point cloud cluster to ensure the point cloud integrity of each tension clamp. Finally, the present invention adopts an iterative closest point algorithm to align the point cloud and fits the three-dimensional model based on the pose estimation method of the model, so as to accurately obtain the three-dimensional posture information of each tension clamp. The present invention effectively solves the problem of three-dimensional posture measurement of tension clamps in complex environments, and achieves high-precision and high-robustness measurement effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 A schematic diagram of a method flow chart of an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a process of acquiring structured light images covering various surfaces of a tension clamp from different angles according to an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of a process of removing noise, correcting distortion, and enhancing fringe contrast for each acquired structured light image according to an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of a process of denoising and removing outliers on a reconstructed three-dimensional point cloud according to an embodiment of the present invention;

[0050] Figure 5 Schematic diagram of the system structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0051] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0052] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0053] like Figure 1-4 As shown, in this embodiment, a method for recognizing a three-dimensional posture of a conductor tension clamp based on structured light is provided, comprising the following steps:

[0054] Acquire multi-view structured light images, and preprocess the structured light images to obtain target structured light images;

[0055] Reconstructing a point cloud according to the target structured light image, then denoising and removing outliers on the reconstructed three-dimensional point cloud, and segmenting the point cloud according to the geometric features and spatial distribution of the point cloud, and obtaining point cloud clusters after segmentation;

[0056] Selecting a point cloud in the point cloud cluster as a reference point cloud, performing point cloud registration on other point clouds and the reference point cloud to obtain a registered point cloud cluster;

[0057] For the registered point cloud cluster, a model-based pose estimation method is used to fit the pre-established three-dimensional model of the tension clamp with the point cloud data to determine the three-dimensional pose estimation result of each tension clamp.

[0058] As an optional implementation, the process of acquiring multi-view structured light images and preprocessing the structured light images includes:

[0059] The structured light camera is used to obtain structured light images covering all surfaces of the tension clamp from different angles. For each structured light image obtained, noise is removed, distortion is corrected, and fringe contrast is enhanced. If there is an overexposed area in the image, the brightness of the overexposed area is adjusted to restore the structured light information.

[0060] like Figure 2 As shown, as an additional implementation method, multiple structured light cameras are used to obtain structured light images of the tension clamp from different angles to cover all surfaces of the tension clamp as much as possible, which can reduce the loss of point cloud data caused by mutual occlusion.

[0061] Specifically, according to the multi-camera calibration parameters, the camera posture in the three-dimensional space is reconstructed to obtain the relative position relationship between the cameras; the structured light image of the tension clamp taken by each structured light camera is obtained, the deformation fringe information is calculated, and the phase value of each pixel is obtained through the phase unwrapping algorithm;

[0062] The system calibration parameters are used in combination with the phase value to calculate the three-dimensional point cloud coordinates corresponding to each pixel to obtain single-camera point cloud data; if there are holes in the single-camera point cloud, a point cloud completion operation is performed to generate a complete single-camera point cloud;

[0063] According to the camera posture transformation matrix, all single-camera point cloud data are converted to a unified world coordinate system to obtain multi-view point cloud data;

[0064] Through the point cloud registration algorithm, for example, the iterative closest point algorithm (ICP) is used in this embodiment to align the multi-view point cloud data and fuse them to generate the final three-dimensional point cloud model of the tension clamp. If the registration error exceeds the preset threshold, the camera posture transformation matrix is ​​recalculated, and the registration is performed again. After obtaining the final three-dimensional point cloud model of the tension clamp, the point cloud normal vector is calculated to determine whether there is a hole area on the point cloud surface and obtain the point cloud hole information. According to the point cloud hole information, the structured light camera posture parameters are adjusted or a new camera perspective is added, and the structured light image is collected again.

[0065] Exemplarily, this embodiment uses multiple structured light cameras to collect data from different angles, and then lays the foundation for subsequent point cloud registration and fusion to ultimately generate a complete three-dimensional model.

[0066] First, multi-camera calibration is the basis of the whole process. This process is similar to measuring the relative position and orientation of each camera with other cameras. For example, a special calibration plate with many regularly distributed dots can be used. Each camera takes multiple images of the calibration plate at different positions, and then the image processing algorithm recognizes these dots and calculates the internal and external parameters of each camera and their relative positions to form a transformation matrix. In this way, the connection between the camera coordinate system and the world coordinate system is established, which is like determining the exact position and shooting angle of each photographer at the shooting scene. Then, each structured light camera projects a beam of specific grating pattern onto the tension clamp and captures the deformed stripe image. These deformed stripes are like contour lines on a topographic map, reflecting the height information of the object surface. Through the phase unwrapping algorithm, these stripes can be converted into the phase value corresponding to each pixel. For example, the phase value of a pixel is 2π, which means that the point is one structured light wavelength higher than the point with a phase value of 0. Assuming that the wavelength of the structured light is 1mm, the height of the point is 1mm higher than the reference point. Next, combined with the system calibration parameters and phase values, the coordinates of each pixel in three-dimensional space can be calculated to form single-camera point cloud data. This is like determining the position of each point on the ground based on the latitude, longitude and altitude on the map. If there are holes in the single-camera point cloud, it may be caused by shooting angle or occlusion, and then a point cloud completion operation is required. Interpolation algorithms can be used to fill these holes. For example, the height information of the hole points can be estimated using the average value of the surrounding points. This ensures the accuracy of subsequent point cloud registration. After obtaining all single-camera point clouds, they need to be converted to a unified world coordinate system based on the camera posture transformation matrix. This is like stitching multiple local maps into a complete map. In this process, each point cloud will be rotated and translated according to the previously calculated camera posture, and finally fused together to form multi-view point cloud data. There may be overlapping parts in the multi-view point cloud data, and these overlapping areas need to be aligned through a point cloud registration algorithm, such as the iterative closest point algorithm (ICP). The ICP algorithm will iterate continuously to find the best rotation and translation parameters so that the point clouds of different perspectives overlap as much as possible, just like stitching together. Figure 1In this way, the position of the puzzle pieces is constantly adjusted until they fit perfectly. If the registration error exceeds the preset threshold, such as the average distance is greater than 0.5mm, it is necessary to recalculate the camera posture transformation matrix and perform registration again to ensure the accuracy of the final model. After the registration is completed, a complete 3D point cloud model of the tension clamp is obtained. In order to detect whether there are defects or holes on the surface of the model, the point cloud normal vector can be calculated. The point cloud normal vector describes the direction of each point on the point cloud surface and can be used to determine the degree of concavity of the point cloud. If the normal vectors in some areas change dramatically or there are points where the normal vectors cannot be calculated, it may be a void area. Assuming that in a certain area of ​​the model, the direction of the normal vector suddenly changes, and the point density in this area is significantly lower than that in other areas, then this part is likely to be a void. Finally, based on the point cloud void information, the structured light camera posture parameters can be adjusted or a new camera angle can be added. For example, if a void is found on the top of the model, the camera can be moved upwards, or a camera can be added to shoot from above. Then the structured light image is recaptured to generate a more complete 3D point cloud model of the tension clamp again. This forms a closed loop, and through continuous iteration, the model is gradually improved until the accuracy requirements are met.

[0067] like Figure 3 As shown, as an additional implementation method, after acquiring the structured light image, each acquired structured light image is subjected to removal processing, distortion correction and contrast enhancement. If there is an overexposed area in the image, the brightness of the overexposed area is adjusted to restore part of the structured light information.

[0068] Specifically, denoising processing: a bilateral filtering algorithm is used to denoise the acquired structured light image to obtain a first denoised image, which smoothes the noise while retaining the image edge information.

[0069] Distortion correction: According to the pre-calibrated camera parameters, a polynomial fitting method is used to perform distortion correction processing on the first denoised image to obtain a first corrected image to eliminate the influence of lens distortion.

[0070] Specifically, the process of performing distortion correction processing on the first denoised image using the polynomial fitting method includes:

[0071] First, the relationship between the pixel coordinates of the standard image and the distorted image is expressed by a bivariate polynomial, which contains two sets of unknown parameters. In order to find the unknown coefficients, a certain number of points need to be taken on the image, and their coordinate values ​​are substituted into the formula for calculation. The control points taken are the points with significant characteristics, and then the coefficient matrix is ​​obtained by polynomial fitting and least squares method, so as to obtain the change law between the coordinates. Finally, all the pixel points are corrected one by one according to the established model to achieve the purpose of correcting the image.

[0072] In the process of image correction, first collect the cylindrical distortion image and the standard image. The points in these two images are one-to-one corresponding. Let the coordinates of a point on the standard image be (x, y), and the coordinates of the corresponding pixel point on the cylindrical curved image be (u, v). The relationship between the pixel coordinates of the two images is established through a set of two-variable multi-time polynomials. In the polynomial, there are two sets of coefficient matrices that need to be solved. To solve the coefficient matrix, take one-to-one corresponding points in each of the two images and substitute them into the formula for calculation, and then calculate the least squares error. When the least squares error is 0, the coefficient matrix obtained is the required one. The polynomial obtained by substituting the coefficient matrix into the formula accurately expresses the relationship between the standard image and the distorted image. Then, according to the established model, the points on the distorted image are corrected one by one. Finally, the bilinear interpolation method is used to repair the grayscale values ​​of some pixels.

[0073] The specific steps of the method are as follows:

[0074] 1) Let the coordinates of a point on the standard image be (x, y), and the coordinates of the corresponding pixel point on the cylindrically curved image be (u, v), and express the corresponding relationship between the two coordinates;

[0075] Determine P (P ≥ 6) control points in the image. To facilitate finding corresponding points in the standard image, points with significant features should be selected as control points. Then use the least squares method to calculate the least squares error ε of the horizontal coordinate u and the vertical coordinate v. u and ε v , where n, i, and j are all integers, i and j are taken from 0 in sequence, and their corresponding relationship is n≥i, n≥j;

[0076] 3) When the least squares error ε u When both εv and εv reach their minimum values, the obtained a ij and b ij is a coefficient matrix that satisfies the conditions;

[0077] 4) Correct the distorted image point by point according to the established model;

[0078] 5) After the correction is completed, the image is interpolated using bilinear interpolation to repair the grayscale values ​​of some pixels, and finally a better image is obtained. The above method can be used to correct the cylindrical distortion image.

[0079] Contrast enhancement: The adaptive histogram equalization method is used to perform contrast enhancement on the first corrected image to obtain the second corrected image, which highlights the structured light fringe information and facilitates subsequent processing.

[0080] Overexposed area detection: By calculating the pixel brightness value of the second corrected image, if the pixel brightness value is greater than a preset threshold, it is determined that the pixel belongs to the overexposed area, and an overexposed area mask is obtained.

[0081] Brightness adjustment: According to the overexposed area mask, the brightness of the pixels in the overexposed area of ​​the second corrected image is adjusted. The adjustment method includes: selecting pixels adjacent to the pixels in the overexposed area from the non-overexposed area of ​​the second corrected image, calculating the brightness mean of these adjacent pixels, and assigning the mean to the pixels in the overexposed area to restore part of the structured light information and obtain the final corrected image.

[0082] As a preferred implementation, the structured light image processing in this embodiment further includes fringe center extraction, specifically using the Steger algorithm to extract the structured light fringe center line from the final corrected image for subsequent three-dimensional reconstruction.

[0083] Exemplarily, the denoising process is to eliminate noise interference in the image and improve the accuracy of subsequent processing.

[0084] The bilateral filtering denoising process of this embodiment is as follows:

[0085] Define the bilateral filtering weights and the normal of the data point, and then estimate the normal vector of the scattered data point; specifically, use the point set in the data point domain to perform least squares plane fitting, and use the normal vector of the fitted plane as the normal vector of the point; let P be a data point in the scattered point cloud, and its domain point set is recorded as kN(P), that is, it contains the k data points closest to P; let the normal vector of the fitted plane be N, and construct the objective function D to represent the distance from point Q in kN(P) to the fitted plane;

[0086] The minimum eigenvalue of the matrix M is obtained by numerical methods. The eigenvector corresponding to the minimum eigenvalue is the normal vector N of the fitting plane after normalization, that is, the unit normal vector of the scattered data point P; they represent the Gaussian filter constant coefficients in the tangent plane direction and the normal direction of the data point, respectively, reflecting the tangential and normal influence ranges when the bilateral filtering operation is performed on any data point p; the condition of N1·N2≈±1 for two adjacent data points can be set to overcome the problem of normal vector ambiguity, that is, the normal vector may not point to the same side of the surface;

[0087] Read point cloud data, find the m nearest neighboring points of each data point through KDTree, and calculate the normal vector of each data point according to the method of scattered point normal estimation; calculate the filter function parameters of each neighboring point, and find the parameters of the feature preservation weight function; find the filtering Gaussian function, calculate the bilateral filtering weight factor λ, bring the filtering Gaussian function into the bilateral filtering weight factor formula to solve, and get the new data point after filtering; when all data points have been updated, the bilateral filtering ends, and a high-quality data source is obtained as the basis for subsequent data applications.

[0088] The bilateral filtering algorithm of this embodiment is a nonlinear filtering method that can retain the edge information of the image while smoothing the noise. Compared with simple mean filtering or Gaussian filtering, bilateral filtering can better retain the clarity of the stripes. For example, in the captured structured light image, there may be some random noise points, which will affect the extraction accuracy of the center line of the stripes. Using the bilateral filtering algorithm, these noise points can be effectively removed while retaining the edge information of the stripes, such as the width and curvature of the stripes. Assuming that the brightness value of a pixel in the image is 100, and the brightness values ​​of the surrounding pixels are 90, 110, 95, and 105 respectively, if the mean filtering is used, the brightness value of the pixel will become 100, and if the bilateral filtering is used, the weighted average will be performed according to the brightness value difference and spatial distance between the surrounding pixels and the pixel, and the final brightness value may be closer to 100, but at the same time the sharpness of the edge is retained.

[0089] Distortion correction is to eliminate image distortion caused by the camera lens. Due to the limitations of the lens manufacturing process, the images taken by the camera often have certain distortions, such as barrel distortion or pincushion distortion. The polynomial fitting method can perform distortion correction on the image according to the pre-calibrated camera parameters. For example, assuming that the edge of the structured light image is bent outward due to lens distortion, the polynomial fitting method can correct these bent parts to the correct position and restore the true shape of the image. Assuming that the coefficients of the distortion polynomial are a, b, and c, the corrected pixel coordinates can be calculated using the original pixel coordinates and these coefficients. Contrast enhancement is to highlight the structured light fringe information for subsequent processing.

[0090] The adaptive histogram equalization method can enhance the contrast of the image according to the local features of the image. For example, in the structured light image of this embodiment, the stripes in some areas may be dim and difficult to identify. The adaptive histogram equalization method can enhance the contrast of these areas, make the stripes clearer, and improve the accuracy of stripe centerline extraction. Assuming that the pixel brightness values ​​in a certain area of ​​the image are distributed more concentratedly, the adaptive histogram equalization will redistribute these brightness values ​​to make them more uniform, thereby enhancing the contrast of the area.

[0091] Overexposed area detection is to identify overly bright areas in the image, where the structured light information is often lost. By calculating the pixel brightness value and comparing it with the preset threshold, it is possible to determine which pixels belong to the overexposed area. For example, in the structured light image of this embodiment, if the pixel brightness value of some areas exceeds 250 (assuming the threshold is 250), these areas will be marked as overexposed areas.

[0092] The brightness adjustment is to restore the structured light information of the overexposed area. By replacing the brightness value of the pixels in the overexposed area with the average brightness value of the pixels in the surrounding non-overexposed area, some structured light information can be restored. For example, assuming that the brightness value of a pixel in an overexposed area in the structured light image of this embodiment is 255, and the brightness values ​​of the pixels in the surrounding non-overexposed areas are 180, 190, and 200, respectively, the brightness value of the pixel in the overexposed area will be adjusted to 190, thereby restoring some structured light information.

[0093] Stripe center extraction is to extract the center line of the structured light stripes from the final corrected image, which is a key step in three-dimensional reconstruction. In this embodiment, the stripe center line extraction through the Steger algorithm can accurately locate the position of the stripe center line. For example, in the structured light image of this embodiment, the Steger algorithm can accurately locate the center line position of each deformed stripe, and the position information of these center lines will be used for subsequent three-dimensional point cloud computing. The extracted stripe center line is a collection of a series of pixel coordinates, such as (x1, y1), (x2, y2), (x3, y3)..., these coordinates are connected to form the center line of the stripe.

[0094] As an optional implementation, the process of reconstructing a point cloud according to the target structured light image includes:

[0095] A multi-perspective geometric reconstruction algorithm is used to integrate the information of multi-perspective structured light images to reconstruct the three-dimensional point cloud model of the tension clamp. If the structured light stripes at a certain perspective are deformed or broken, the weight of the perspective point cloud data is reduced as needed, and point cloud data with complete quality is used first for model reconstruction.

[0096] As an implementation method that can be added, this embodiment judges the quality of the structured light stripes by analyzing the continuity and smoothness of the center line of the structured light stripes extracted in each image. If the center line is deformed or broken, such as a sudden change, jump or interruption, the quality of the structured light stripes at this viewing angle is judged to be low, otherwise the quality is judged to be high.

[0097] According to the evaluation results of the structured light fringe quality, a weight is assigned to the point cloud data of each image. If the quality of the structured light fringe is high, a higher weight value is assigned; if the quality of the structured light fringe is low, a lower weight value is assigned.

[0098] Three-dimensional reconstruction is performed based on a multi-view geometric reconstruction algorithm. Using a multi-view geometric reconstruction algorithm, this embodiment uses SFM (Structure from Motion) or MVS (Multi-View Stereo) to fuse multi-view structured light image information to reconstruct a three-dimensional point cloud model of the tension clamp. During the reconstruction process, based on the weight of the point cloud data of each image, point cloud data with higher quality is used first, and point cloud data with lower quality is downgraded. The reconstructed three-dimensional point cloud model of the tension clamp is post-processed, including point cloud denoising, point cloud simplification, point cloud smoothing and other operations, to optimize the quality and accuracy of the point cloud model. A three-dimensional mesh model of the tension clamp is generated from the point cloud model to obtain a final three-dimensional model for subsequent analysis and application.

[0099] Exemplarily, the quality of structured light stripes is judged in order to screen out high-quality image data for subsequent three-dimensional reconstruction. If the center line of the stripes is broken or deformed, it means that the projected structured light stripes are affected by occlusion or changes in surface curvature, resulting in information loss or distortion. This is like measuring with a broken ruler, and the result is definitely inaccurate. For example, if the center line of the stripes suddenly changes in a certain area and jumps a few pixels, the quality of the stripes in this part is relatively low. The weight of the point cloud data is calculated to pay more attention to high-quality data and reduce the impact of low-quality data during the three-dimensional reconstruction process. For example, if the quality of the structured light stripes at a certain perspective is very high, continuous and smooth, then the corresponding point cloud data should be given a higher weight, such as 0.8; conversely, if the stripes are of low quality and broken or deformed, then its weight should be reduced, such as 0.2.

[0100] The purpose of 3D reconstruction based on multi-view geometric reconstruction algorithm is to fuse 2D image information from multiple viewpoints into a complete 3D model. This is like using multiple photos to piece together a panorama, which can more completely show the overall structure of the scene. The SFM algorithm can estimate the camera posture and the 3D structure of the scene from multiple images, while the MVS algorithm can calculate the 3D coordinates corresponding to each pixel based on the camera posture and the pixel information of the image, and finally form a dense point cloud model. Point cloud data post-processing is to optimize the quality and accuracy of the point cloud model. Denoising operation can remove outliers and noise in the point cloud, and simplifying operation can reduce the amount of point cloud data and improve processing efficiency, just like using simpler lines to outline the outline of an object. Smoothing operation can make the point cloud surface smoother. The purpose of obtaining a 3D model is to finally obtain a 3D mesh model of the tension clamp for subsequent analysis and application.

[0101] As an optional implementation, the process of denoising and removing outliers on the reconstructed three-dimensional point cloud and segmenting the point cloud according to the geometric features and spatial distribution of the point cloud includes:

[0102] The reconstructed three-dimensional point cloud is denoised and outliers are removed. Noise points and outliers caused by metal reflections, complex shadows and highlight areas are removed. Statistical filters are used to remove outliers and retain the real point cloud data. Then, the point cloud data belonging to different tension clamps are segmented according to the geometric characteristics and spatial distribution of the point cloud. If two tension clamps overlap, they are segmented according to the curvature change and normal vector difference in the overlapping area. If the curvature change and normal vector difference are less than the preset threshold, the overlapping area is divided into a point cloud cluster with a larger area.

[0103] like Figure 4 As shown, as an additional implementation method, denoising and outlier removal are performed on the reconstructed three-dimensional point cloud, including:

[0104] Obtain reconstructed three-dimensional point cloud data;

[0105] Calculate the normal vector of each point: Use the local surface fitting algorithm to fit a local plane based on the neighborhood points around each point to obtain the normal vector of the point.

[0106] Calculate the curvature of each point: Calculate the curvature value of each point based on the normal vector of each point and its neighboring points. If the curvature value is greater than the preset threshold, the point is judged to be in a high curvature area and is a potential noise point or an abnormal point.

[0107] Construct a KD tree of point cloud: Construct a KD tree with 3D point cloud data to accelerate the search for neighboring points.

[0108] Statistical filtering: For each point, search for neighboring points within the specified radius in the KD tree and calculate the average distance from the neighboring points to the point. If the average distance is greater than the preset threshold, the point is judged as an outlier.

[0109] Remove outliers: Remove the identified outliers from the original point cloud data to obtain the point cloud data after preliminary denoising.

[0110] Obtaining real point cloud data: Perform secondary statistical filtering on the point cloud data after preliminary denoising to remove the remaining noise points and abnormal points to obtain the final denoised real point cloud data.

[0111] Exemplarily, obtaining reconstructed three-dimensional point cloud data is the first step in point cloud denoising. Suppose you want to process the three-dimensional point cloud data of a tension clamp, which contains millions of points, each of which represents a spatial position on the surface of the clamp. The normal vector of each point is calculated to describe the local directional information of the point cloud surface. There are many ways to calculate the normal vector, and this embodiment uses a local surface fitting method. For example, 10 neighboring points around a point are selected, and a plane is fitted using the least squares method. The normal vector of this plane is the normal vector of the point. In some parts of the tension clamp, such as the connection, the surface normal vector changes greatly; while in other parts, such as the main body, the normal vector changes less. The curvature of each point is calculated to identify the degree of curvature of the point cloud surface. Where the curvature is large, the curvature of the surface is large, such as the edge of the tension clamp. Conversely, where the curvature is small, the surface is relatively flat, such as the plane part of the tension clamp.

[0112] This embodiment uses the difference in normal vectors of neighborhood points to calculate the curvature. Assuming that the curvature of a point is greater than the preset threshold of 0.1, it is considered that this point is located in a high curvature area and may be a noise point or an abnormal point. The KD tree of the point cloud is constructed to speed up the search for neighboring points. The KD tree is a data structure that can efficiently search for the point closest to a point in space. The KD tree can be imagined as a spatial index. After the KD tree is constructed, the neighborhood points of each point can be quickly found. Statistical filtering is a denoising method based on statistical principles. For each point of the tension clamp, the KD tree can be searched for neighborhood points with a radius of 2mm, and the average distance from these neighborhood points to the point is calculated. If the average distance is greater than the preset threshold of 1mm, the point is considered to be an outlier, which may be caused by noise or abnormal data. For example, due to measurement errors, some points deviate from the actual surface of the tension clamp, and these points will be identified in the statistical filtering. Removing outliers means deleting the outliers identified in the previous step from the point cloud data. After removing outliers, the quality of the point cloud data will be initially improved, closer to the real shape of the tension clamp. To obtain the real point cloud data, it is necessary to perform secondary statistical filtering on the point cloud data after preliminary denoising. This step is to remove the remaining noise points and abnormal points to further improve the accuracy of the point cloud data. After secondary statistical filtering, the obtained point cloud data is cleaner and can better reflect the real surface shape of the tension clamp, and can be used for subsequent applications such as 3D model reconstruction.

[0113] As an additional implementation method, performing point cloud segmentation according to the geometric features and spatial distribution of the point cloud includes:

[0114] Perform preliminary segmentation based on the spatial distribution of the point cloud. This embodiment uses a clustering algorithm based on Euclidean distance, such as DBSCAN or K-means algorithm, to segment the point cloud into multiple initial point cloud clusters. Each point cloud cluster may contain point cloud data of one or more tension clamps. For overlapping point cloud clusters, secondary segmentation is performed based on the curvature change and normal vector difference in the overlapping area. Calculate the average and standard deviation of the normal vector and curvature of the point cloud in the overlapping area. If the normal vector standard deviation or curvature standard deviation of the overlapping area is greater than the preset threshold, the regional growing algorithm is used to segment the overlapping area into smaller sub-clusters; if the normal vector standard deviation and curvature standard deviation of the overlapping area are both less than the preset threshold, the overlapping area is divided into a point cloud cluster with a larger area. Calculate the area of ​​each overlapping area and its adjacent point cloud clusters. Divide the points in the overlapping area into the point cloud cluster that overlaps with it and has the largest area. Based on the prior knowledge of the tension clamp, the segmented point cloud clusters are refined. For example, according to the shape characteristics of the tension clamp, some point cloud clusters that do not conform to the shape of the tension clamp are removed, or some fragmented point cloud clusters are merged to improve the accuracy and completeness of the segmentation. The final segmentation result is output. Each point cloud cluster is marked as a different category to represent a different tension clamp. The segmentation results are visualized so that users can check and verify the quality of the segmentation.

[0115] Exemplarily, preliminary segmentation is performed based on the spatial distribution of the point cloud, and the point cloud is segmented into multiple initial point cloud clusters using the DBSCAN algorithm. Each point cloud cluster may contain point cloud data of one or more tension clamps. The core idea of ​​the DBSCAN algorithm is that if a point has at least MinPts points within a radius of Eps, the point is considered to be a core point. All points that are densely connected to the core points are divided into the same cluster. Assume that Eps is set to 0.1 meters and MinPts is set to 5. If point A has 5 or more points within a radius of 0.1 meters, point A is a core point. All points that are densely connected to point A will be divided into the same cluster as point A. For overlapping point cloud clusters, secondary segmentation is required. Calculate the mean and standard deviation of the normal vector and curvature of the point cloud in the overlapping area. If the normal vector standard deviation or curvature standard deviation of the overlapping area is greater than the preset threshold, it means that the overlapping area may contain multiple tension clamps and further segmentation is required. For example, if the standard deviation of the normal vector of the overlapping area is greater than 10 degrees, the overlapping area can be segmented into smaller subclusters using the region growing algorithm. The core idea of ​​the region growing algorithm is to start from a seed point and gradually add points with similar features to the same area. If the standard deviation of the normal vector and the standard deviation of the curvature of the overlapping area are both less than the preset threshold, the overlapping area is divided into a point cloud cluster with a larger area. For example, if the overlapping area overlaps with both point cloud cluster A and point cloud cluster B, and the area of ​​point cloud cluster A is greater than the area of ​​point cloud cluster B, the overlapping area is divided into point cloud cluster A. Finally, according to the prior knowledge of the tension clamp, the segmented point cloud clusters are refined. For example, knowing that the shape of the tension clamp is usually long and strip-shaped, some point cloud clusters that do not conform to this shape can be removed. In addition, some fragmented point cloud clusters can be merged to improve the accuracy and completeness of the segmentation. For example, if the distance between two point cloud clusters is close and the shapes are similar, they can be merged into one point cloud cluster. Output the final segmentation result. Each point cloud cluster is marked as a different category, representing a different tension clamp. The segmentation results can be visualized so that users can check and verify the quality of the segmentation. For example, different colors can be used to represent different tension clamps, and the segmentation results can be displayed in a 3D point cloud viewer.

[0116] As an optional implementation, a process of selecting a point cloud in the point cloud cluster as a reference point cloud and performing point cloud registration on other point clouds with the reference point cloud includes:

[0117] For each point cloud cluster after segmentation, one of the point clouds is selected as the reference point cloud, and the other point clouds are aligned with the reference point cloud. All point cloud data are unified into the same coordinate system, and the iterative closest point algorithm is used for point cloud alignment, and the point-to-surface distance is used as the metric for the alignment error.

[0118] As an additional implementation method, the segmented point cloud cluster data is obtained, and a point cloud cluster is selected from the point cloud cluster data as a reference point cloud. For each point cloud cluster except the reference point cloud, the point cloud cluster is registered with the reference point cloud to obtain a transformation matrix. According to the obtained transformation matrix, the point cloud cluster is transformed into the coordinate system of the reference point cloud to obtain a transformed point cloud cluster. All the transformed point cloud clusters are fused together to obtain complete point cloud data in a unified coordinate system.

[0119] Exemplarily, the segmented point cloud cluster data is obtained. For example, the point cloud data of a power inspection scene has been segmented into three point cloud clusters through the previous steps, representing three tension clamps. Each point cloud cluster contains thousands of three-dimensional coordinate points, describing the shape and position of the corresponding tension clamp. Select a point cloud cluster as a reference point cloud. You can select one of the point cloud clusters, such as the point cloud cluster of the first tension clamp, as a reference point cloud. This means that the other two point cloud clusters will be converted to the coordinate system of this reference point cloud. The principle of selecting a reference point cloud can be determined according to the actual situation. For example, you can select the point cloud cluster with the highest data quality, or select the point cloud cluster with the most central position and the largest size. For each point cloud cluster except the reference point cloud, align the point cloud cluster with the reference point cloud to obtain a transformation matrix. For the second and third point cloud clusters, they need to be aligned with the reference point cloud respectively. The alignment process is to find an optimal transformation matrix to maximize the overlap of the two point cloud clusters. The commonly used alignment algorithm is the iterative closest point (ICP) algorithm. The ICP algorithm will iterate continuously, calculate the distance between the closest point pairs in the two point cloud clusters, and optimize the transformation matrix until the preset accuracy or number of iterations is reached. Assume that through the ICP algorithm, the transformation matrix T2 from the second point cloud cluster to the reference point cloud and the transformation matrix T3 from the third point cloud cluster to the reference point cloud are obtained. T2 and T3 contain rotation and translation information, for example, it may be a 4x4 matrix. According to the obtained transformation matrix, the point cloud cluster is transformed into the coordinate system of the reference point cloud to obtain the transformed point cloud cluster. Using the obtained transformation matrix T2, the three-dimensional coordinates of each point in the second point cloud cluster can be multiplied by T2 to obtain the new coordinates of the point in the reference point cloud coordinate system. Similarly, T3 can be used to transform the third point cloud cluster into the coordinate system of the reference point cloud. In this way, two transformed point cloud clusters are obtained. All the transformed point cloud clusters are merged together to obtain complete point cloud data in a unified coordinate system. Finally, the reference point cloud, the transformed second point cloud cluster, and the transformed third point cloud cluster are merged together. Now, all point cloud data are in the same coordinate system, forming a complete point cloud model, which fully describes the geometric information of the three tension clamps. In this way, the unified registration of the point cloud is completed, laying the foundation for subsequent 3D modeling, measurement, analysis and other work. The advantage of this is that the originally scattered point cloud data can be integrated together to form an overall model, which is convenient for subsequent analysis and processing. For example, it is more convenient to calculate the geometric relationship such as the distance and angle between the three tension clamps, or to perform overall defect detection. For example, assuming that the center point coordinates of the reference point cloud are (0, 0, 0), and the center point coordinates of the second point cloud cluster are (1, 1, 1), the transformation matrix T2 represents the translation of the second point cloud cluster by (-1, -1, -1).Then, after multiplying the coordinates of the center point of the second point cloud cluster by T2, the new coordinates are (0, 0, 0), which coincides with the center point of the reference point cloud. In this way, the second point cloud cluster is aligned to the coordinate system of the reference point cloud. Similarly, the third point cloud cluster can also be aligned to the coordinate system of the reference point cloud, and finally the three point cloud clusters are merged into a complete point cloud model. The advantage of doing this is that the entire scene can be easily analyzed and processed, such as measuring the distance and angle between different tension clamps.

[0120] As an additional implementation, the three-dimensional posture of each tension clamp is determined including the position and posture.

[0121] Specifically, the registered point cloud cluster data and the pre-established three-dimensional model of the tension clamp are obtained. For each point cloud cluster, the iterative closest point (ICP) algorithm is used for coarse registration to obtain the initial pose of the tension clamp. If the number of iterations of the ICP algorithm reaches the preset upper limit, the iteration is terminated and the current pose is obtained as the initial pose. According to the obtained initial pose, the Levenberg-Marquardt algorithm is used for fine pose estimation to minimize the distance deviation between the point cloud data and the three-dimensional model to obtain an accurate three-dimensional pose (position and attitude). If the number of algorithm iterations reaches the preset upper limit, the iteration is terminated and the current pose is obtained as the final pose. The average distance between the point cloud data and the fitted three-dimensional model is calculated. If the average distance is less than the preset error threshold, the pose estimation is judged to be successful, and the obtained three-dimensional pose is recorded in the database. If the average distance is greater than or equal to the preset threshold, the pose estimation is judged to fail, and the current point cloud cluster is marked as pending for subsequent manual intervention or other algorithm processing. For the tension clamps whose posture is estimated successfully, obtain their 3D posture information, including position coordinates (x, y, z) and rotation angle. According to the 3D posture information of the tension clamp, judge its installation status in the power line, for example: whether the installation angle meets the specifications, whether there are abnormal conditions such as offset or tilt. If the installation angle does not meet the specifications, an alarm message is generated to prompt relevant personnel to check and handle it. According to the 3D posture information of all tension clamps, reconstruct the 3D model of the power line for applications such as line inspection, fault diagnosis and status assessment.

[0122] For example, according to the design specifications, the pitch angle of the tension clamp should be between 8 and 12 degrees. If the measured pitch angle is 15 degrees, it means that the tension clamp is tilted, and an alarm message needs to be generated to prompt relevant personnel to check and handle it. This can help to promptly discover safety hazards in power lines. Based on the three-dimensional posture information of all tension clamps, the three-dimensional model of the power line is reconstructed for applications such as line inspection, fault diagnosis and status assessment. For example, the three-dimensional posture information of all tension clamps and the information of other line components can be integrated to construct a complete three-dimensional model of the power line. This model can be used to simulate the operating status of the line in different environments, perform line inspection, fault diagnosis and status assessment, and improve the operation and maintenance efficiency of the power line.

[0123] like Figure 2 As shown, this embodiment also provides a three-dimensional posture recognition system for a conductor tension clamp based on structured light, including:

[0124] A multi-view image acquisition module, used to acquire multi-view structured light images;

[0125] A structured light image preprocessing module, used for preprocessing the structured light image to obtain a target structured light image;

[0126] A point cloud reconstruction and denoising module, used to reconstruct the point cloud according to the target structured light image, then denoise and remove outliers on the reconstructed three-dimensional point cloud, and segment the point cloud according to the geometric features and spatial distribution of the point cloud, and obtain point cloud clusters after segmentation;

[0127] A point cloud registration module, used for selecting a point cloud in the point cloud cluster as a reference point cloud, performing point cloud registration on other point clouds and the reference point cloud to obtain a registered point cloud cluster;

[0128] The three-dimensional model fitting module is used to fit the pre-established three-dimensional model of the tension clamp with the point cloud data using a model-based pose estimation method for the registered point cloud cluster, so as to determine the three-dimensional pose estimation result of each tension clamp.

[0129] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for three-dimensional posture recognition of a conductor tension clamp based on structured light, characterized in that: include: Acquire multi-view structured light images, and preprocess the structured light images to obtain target structured light images; Reconstructing a point cloud according to the target structured light image, then denoising and removing outliers on the reconstructed three-dimensional point cloud, and segmenting the point cloud according to the geometric features and spatial distribution of the point cloud, and obtaining point cloud clusters after segmentation; Selecting a point cloud in the point cloud cluster as a reference point cloud, performing point cloud registration on other point clouds and the reference point cloud to obtain a registered point cloud cluster; For the registered point cloud cluster, a model-based pose estimation method is used to fit the pre-established three-dimensional model of the tension clamp with the point cloud data to determine the three-dimensional pose estimation result of each tension clamp.

2. The method according to claim 1, characterized in that The process of acquiring multi-view structured light images and preprocessing the structured light images includes: The structured light camera is used to obtain structured light images covering all surfaces of the tension clamp from different angles. For each structured light image obtained, noise is removed, distortion is corrected, and fringe contrast is enhanced. If there is an overexposed area in the image, the brightness of the overexposed area is adjusted to restore the structured light information.

3. The method according to claim 2, characterized in that The process of obtaining structured light images covering various surfaces of the tension clamp from different angles using a structured light camera includes: According to the multi-camera calibration parameters, the camera posture in the three-dimensional space is reconstructed to obtain the relative position relationship between the cameras; The structured light images of the tension clamps taken by each structured light camera are obtained, the deformed fringe information is calculated, and the phase value of each pixel is obtained by the phase unwrapping algorithm; The system calibration parameters are used in combination with the phase value to calculate the three-dimensional point cloud coordinates corresponding to each pixel to obtain single-camera point cloud data; if there are holes in the single-camera point cloud, a point cloud completion operation is performed to generate a complete single-camera point cloud; According to the camera posture transformation matrix, all single-camera point cloud data are converted to a unified world coordinate system to obtain multi-view point cloud data; Through the point cloud registration algorithm, the multi-view point cloud data are aligned and fused to generate the final three-dimensional point cloud model of the tension clamp; if the registration error exceeds the preset threshold, the camera posture transformation matrix is ​​recalculated and registered again. After obtaining the final three-dimensional point cloud model of the tension clamp, the point cloud normal vector is calculated to determine whether there is a hole area on the point cloud surface and obtain the point cloud hole information; according to the point cloud hole information, the structured light camera posture parameters are adjusted or a new camera perspective is added, and the structured light image is collected again.

4. The method according to claim 2, characterized in that: The process of removing noise, correcting distortion, and enhancing fringe contrast for each acquired structured light image includes: A bilateral filtering algorithm is used to perform denoising on the acquired structured light image to obtain a first denoised image; a polynomial fitting method is used to perform distortion correction on the first denoised image according to pre-calibrated camera parameters to obtain a first corrected image; Performing contrast enhancement processing on the first corrected image by using an adaptive histogram equalization method to obtain a second corrected image; Calculating the pixel brightness value of the second corrected image, and if the pixel brightness value is greater than a preset threshold, determining that the pixel belongs to an overexposed area, and obtaining an overexposed area mask; According to the overexposed area mask, the pixel brightness of the overexposed area in the second corrected image is adjusted.

5. The method according to claim 4, characterized in that The process of adjusting the pixel brightness of the overexposed area in the second corrected image includes: Select adjacent pixels to the pixels in the overexposed area from the non-overexposed area of ​​the second corrected image, calculate the brightness mean of the adjacent pixels, assign the brightness mean to the pixels in the overexposed area, restore the structured light information, and obtain the final corrected image.

6. The method according to claim 1, characterized in that The process of reconstructing a point cloud according to the target structured light image includes: A multi-perspective geometric reconstruction algorithm is used to integrate the information of multi-perspective structured light images to reconstruct the three-dimensional point cloud model of the tension clamp. If the structured light stripes at a certain perspective are deformed or broken, the weight of the perspective point cloud data is reduced as needed, and point cloud data with complete quality is used first for model reconstruction.

7. The method according to claim 1, characterized in that The process of denoising and removing outliers from the reconstructed 3D point cloud and segmenting the point cloud according to its geometric features and spatial distribution includes: The reconstructed three-dimensional point cloud is denoised and outliers are removed. Noise points and outliers caused by metal reflections, complex shadows and highlight areas are removed. Statistical filters are used to remove outliers and retain the real point cloud data. Then, the point cloud data belonging to different tension clamps are segmented according to the geometric characteristics and spatial distribution of the point cloud. If two tension clamps overlap, they are segmented according to the curvature change and normal vector difference in the overlapping area. If the curvature change and normal vector difference are less than the preset threshold, the overlapping area is divided into a point cloud cluster with a larger area.

8. The method according to claim 7, characterized in that The process of denoising and removing outliers from the reconstructed 3D point cloud includes: Obtain the reconstructed 3D point cloud data; use the local surface fitting algorithm to fit a local plane based on the neighborhood points around each point, and calculate the normal vector of each point; The curvature value of each point is calculated according to the normal vector of each point and its neighboring points. If the curvature value is greater than a preset threshold, the point is judged to be located in a high curvature area and is a potential noise point or an abnormal point. Construct a KD tree with 3D point cloud data to accelerate the search for neighboring points. For each point, search for neighboring points within a specified radius in the KD tree and calculate the average distance from the neighboring points to the point. If the average distance is greater than the preset threshold, the point is judged as an outlier. The outliers are removed from the original point cloud data to obtain the point cloud data after preliminary denoising; the point cloud data after preliminary denoising is subjected to secondary statistical filtering to remove the remaining noise points and abnormal points to obtain the final denoised real point cloud data.

9. The method according to claim 1, characterized in that: The process of selecting one point cloud in the point cloud cluster as a reference point cloud and performing point cloud registration on other point clouds with the reference point cloud includes: For each point cloud cluster after segmentation, one of the point clouds is selected as the reference point cloud, and the other point clouds are aligned with the reference point cloud. All point cloud data are unified into the same coordinate system, and the iterative closest point algorithm is used for point cloud alignment, and the point-to-surface distance is used as the metric for the alignment error.

10. A three-dimensional position recognition system for conductor tension clamps based on structured light, characterized in that: include: A multi-view image acquisition module, used to acquire multi-view structured light images; A structured light image preprocessing module, used for preprocessing the structured light image to obtain a target structured light image; A point cloud reconstruction and denoising module, used to reconstruct the point cloud according to the target structured light image, then denoise and remove outliers on the reconstructed three-dimensional point cloud, and segment the point cloud according to the geometric features and spatial distribution of the point cloud, and obtain point cloud clusters after segmentation; A point cloud registration module, used for selecting a point cloud in the point cloud cluster as a reference point cloud, performing point cloud registration on other point clouds and the reference point cloud to obtain a registered point cloud cluster; The three-dimensional model fitting module is used to fit the pre-established three-dimensional model of the tension clamp with the point cloud data using a model-based pose estimation method for the registered point cloud cluster, so as to determine the three-dimensional pose estimation result of each tension clamp.

Images