Photovoltaic support recognition and positioning method and device based on point cloud matching

Abstract

The application provides a photovoltaic support recognition and positioning method and device based on point cloud matching, comprising: scanning a to-be-detected photovoltaic support through a laser radar to obtain an initial point cloud set; performing a preprocessing operation to obtain a target point cloud set; calling a pre-stored photovoltaic support template library, wherein the template library stores a plurality of support point cloud sets corresponding to a plurality of types of photovoltaic supports; calculating a plurality of FPFH histogram feature similarities between the target point cloud set and the plurality of support point cloud sets, and screening a standard point cloud set with the largest similarity from the plurality of support point cloud sets; and aligning the target point cloud set and the standard point cloud set according to a normal distribution transformation algorithm to obtain a target pose of the to-be-detected photovoltaic support. The application can improve the accuracy and efficiency of photovoltaic support recognition and positioning in a complex environment.

Description

Technical Field

[0001] This application relates to the field of pose estimation technology, and in particular to a photovoltaic bracket identification and positioning method and device based on point cloud matching. Background Technology

[0002] With the development of the global new energy industry, photovoltaic power generation systems have been deployed on a large scale. In order to ensure the power generation efficiency and long-term stability of photovoltaic power plants, the installation accuracy and angle control of photovoltaic panels are particularly important. The installation position and angle of photovoltaic panels depend on the actual installation state of the photovoltaic support structure. Therefore, accurately identifying the photovoltaic support structure and calculating its posture on the construction site is a prerequisite for realizing automated and precise installation of photovoltaic panels.

[0003] Currently, traditional methods mainly rely on general point cloud technology to identify the pose of photovoltaic supports from the current environment. However, these methods are easily affected by ground point clouds in complex environments. Furthermore, the spatial locations of a large number of mixed point clouds are scattered, making it impossible to quickly and accurately identify photovoltaic supports from mixed point clouds. This makes it difficult to meet the needs for automatic, accurate, and efficient pose recognition of photovoltaic supports in actual photovoltaic installation scenarios. Summary of the Invention

[0004] This application provides a photovoltaic bracket identification and positioning method and device based on point cloud matching, which can realize automatic identification and high-precision spatial pose calculation of photovoltaic brackets, provide accurate geometric reference for automated photovoltaic panel installation, and improve installation efficiency and accuracy.

[0005] In a first aspect, embodiments of this application provide a photovoltaic support bracket identification and positioning method based on point cloud matching, including: The photovoltaic support under test is scanned using lidar to obtain an initial point cloud set; Perform preprocessing operations on the initial point cloud set to obtain the target point cloud set; Retrieve a pre-stored photovoltaic support template library, which stores multiple support point cloud sets corresponding to various types of photovoltaic supports; Calculate the similarity of multiple FPFH histogram features between the target point cloud set and the multiple support point cloud sets, and select the standard point cloud set with the highest similarity from the multiple support point cloud sets, wherein the photovoltaic support type corresponding to the standard point cloud set matches the photovoltaic support to be detected; The target point cloud set and the standard point cloud set are aligned using a normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected.

[0006] Secondly, embodiments of this application provide a photovoltaic bracket identification and positioning device based on point cloud matching, comprising: a scanning unit and a processing unit, wherein: The scanning unit is used to scan the photovoltaic support to be inspected using a lidar to obtain an initial point cloud set; The processing unit is configured to perform preprocessing operations on the initial point cloud set to obtain a target point cloud set; retrieve a pre-stored photovoltaic support template library, which stores multiple support point cloud sets corresponding to various types of photovoltaic supports; calculate the similarity of multiple FPFH histogram features between the target point cloud set and the multiple support point cloud sets; and select the standard point cloud set with the highest similarity from the multiple support point cloud sets, wherein the photovoltaic support type corresponding to the standard point cloud set matches the photovoltaic support to be detected; and align the target point cloud set and the standard point cloud set according to a normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected.

[0007] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing the steps in the first aspect of embodiments of this application.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the first aspect of embodiments of this application.

[0009] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of embodiments of this application. The computer program product may be a software installation package.

[0010] Implementing this application will have the following beneficial effects: As can be seen, the photovoltaic support identification and positioning method based on point cloud matching described in this application involves scanning the photovoltaic support to be detected using a lidar to obtain an initial point cloud set; performing preprocessing operations on the initial point cloud set to obtain a target point cloud set; retrieving a pre-stored photovoltaic support template library, which stores multiple support point cloud sets corresponding to various types of photovoltaic supports; calculating the similarity of multiple FPFH histogram features between the target point cloud set and the multiple support point cloud sets; selecting the standard point cloud set with the highest similarity from the multiple support point cloud sets, and matching the photovoltaic support type corresponding to the standard point cloud set with the photovoltaic support to be detected; aligning the target point cloud set and the standard point cloud set according to a normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected. Therefore, compared with existing photovoltaic support identification and positioning schemes based on general point cloud technology, this application can accurately adapt to the regular beam structure of photovoltaic supports, effectively avoid interference and point cloud dispersion problems on the ground, and achieve automatic identification and high-precision spatial pose calculation of photovoltaic supports. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0012] Figure 1 This is a schematic diagram of the structure of a photovoltaic module installation robot provided in an embodiment of this application; Figure 2 This is a flowchart of a photovoltaic bracket identification and positioning method based on point cloud matching provided in an embodiment of this application; Figure 3 This is a flowchart of a method for determining the similarity of FPFH histogram features provided in an embodiment of this application; Figure 4 This is a flowchart of a method for determining the target pose of a photovoltaic support according to an embodiment of this application; Figure 5 This is a schematic diagram of a photovoltaic support coordinate system provided in an embodiment of this application; Figure 6 This is a schematic diagram of a local coordinate system for FPFH feature vectors provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a photovoltaic bracket identification and positioning device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0014] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0015] It should be understood that the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, "multiple" refers to two or more.

[0016] In the embodiments of this application, "at least one item" or its similar expression refers to any combination of these items, including any combination of a single item or a plurality of items. "One or more" means one or more, while "multiple" means two or more. For example, "at least one item" of a, b, or c can represent the following seven cases: a, b, c; a and b; a and c; b and c; a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0017] In this application embodiment, "connection" refers to various connection methods such as direct connection or indirect connection to realize communication between devices. This application embodiment does not limit this in any way.

[0018] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0019] The electronic devices described in this application embodiment may include smartphones (such as Android phones, iOS phones, Windows Phones, etc.), tablet computers, PDAs, laptops, video matrices, monitoring platforms, mobile internet devices (MIDs), or wearable devices, etc. The above are merely examples and not exhaustive, and include but are not limited to the above devices.

[0020] Of course, the aforementioned electronic devices may also include photovoltaic module installation robots.

[0021] The following describes the relevant content, concepts, meanings, technical issues, technical solutions, and beneficial effects involved in the embodiments of this application.

[0022] First, let me explain some of the technical terms used in this application: LiDAR: A sensor that scans a target scene by emitting a laser beam and receives reflected signals to calculate information such as distance and orientation.

[0023] Point cloud set: It is a collection of massive three-dimensional coordinate points output by LiDAR scanning. Each point contains three-dimensional coordinate information of X, Y and Z in space, and the whole represents the three-dimensional geometric shape of the photovoltaic support.

[0024] Feature regions: These are key local areas with high recognizability in the point cloud of a photovoltaic support structure. For example, the area around the intersection of the horizontal and vertical beams of the support structure is a core area for point cloud matching, used to extract feature vectors for similarity comparison.

[0025] Rigid transformation matrix: It is a mathematical matrix that represents the "rotation" and "translation" transformation relationship between two point cloud sets. For example, a 3×4 augmented matrix contains a rotation matrix R (controlling angle transformation) and a translation vector t (controlling position transformation), which can realize the alignment of point clouds in space.

[0026] Voxel filtering is a downsampling algorithm in point cloud preprocessing. It divides the three-dimensional space into multiple tiny cubes (voxels), and retains only one representative point (such as the center point) in each voxel. While preserving the core geometric features of the point cloud, it reduces the number of points and the amount of computation.

[0027] Plane fitting algorithm: It is a target plane that fits the point cloud set through mathematical modeling, such as the least squares method, so as to accurately identify and remove ground point cloud interference.

[0028] DBSCAN density clustering algorithm: It is a noise reduction algorithm based on point cloud density. By setting the neighborhood radius and minimum number of points, it clusters densely distributed effective point clouds into one class, while removing scattered isolated noise points to improve the purity of point clouds.

[0029] K-Nearest Neighbors Algorithm: A basic nearest neighbor search algorithm that finds a predetermined number (K) of neighboring points for each point in a target point cloud. It is used in scenarios such as local plane fitting and feature extraction to ensure the accuracy of local geometric features.

[0030] Principal Component Analysis (PCA) algorithm: This is a data dimensionality reduction and feature extraction algorithm. By analyzing the covariance matrix of point cloud data, it finds the principal direction of the data distribution (the eigenvector corresponding to the largest eigenvalue), which is used to determine the extension direction of the support beam, the principal direction of the local plane, etc.

[0031] Three-dimensional straight line fitting algorithm: This is an algorithm that fits the spatial center line representing the direction of the beam structure (photovoltaic support) from the point cloud set of support beams through mathematical modeling (such as the least squares method). It is used to accurately locate the spatial position and extension trend of the horizontal and vertical beams in the photovoltaic support.

[0032] The Fast Point Feature Histogram (FPFH) algorithm is a local feature description algorithm for point clouds. It constructs a multi-dimensional histogram as feature vectors by calculating information such as the angle between the normal vectors of a point and its neighbors, and the distance between them. This effectively represents local geometric structures and can be used for similarity comparison between different point clouds. For example, it calculates the angle between the normal vectors of the core point and each neighboring point, the azimuth angle, and the distance difference, and divides these three types of geometric information into multiple intervals (e.g., 11 intervals for the normal vectors, 12 intervals for the azimuth angles, and 10 intervals for the distance differences). It then counts the number of neighboring points within each interval to form a multi-dimensional histogram. After normalizing the multi-dimensional histogram, it finally obtains the FPFH feature vector of the core point (feature region). The vectors of multiple feature regions are combined to form an FPFH feature set (e.g., the first target FPFH feature set, the second target FPFH feature set).

[0033] Euclidean distance similarity algorithm: This is an algorithm that measures the difference between two feature vectors, such as FPFH feature vectors. It calculates the Euclidean distance between the vectors, which is the straight-line distance in space. The smaller the distance, the higher the similarity. It is used to filter feature matching pairs.

[0034] Normal Distribution Transform (NDT) algorithm: This is a point cloud fine registration algorithm. It divides the point cloud into voxel grids, constructs a normal distribution model by calculating the mean and covariance of each voxel, and then corrects the rigid transformation matrix by optimizing the likelihood function, so as to achieve high-precision alignment between the standard point cloud and the target point cloud and improve the pose calculation accuracy.

[0035] Please see Figure 1 , Figure 1This is a structural schematic diagram of a photovoltaic module installation robot provided in an embodiment of this application. It can be seen that the photovoltaic module installation robot 100 may include: a control module 110, a lidar module 120, a movement module 130, and an installation module 140, etc., which are not limited here; wherein: The control module 110 is an industrial controller (such as a PLC or embedded motherboard), including an algorithm execution unit and a storage unit. As the "brain" of the photovoltaic module installation robot 100, the control module 110 is responsible for coordinating the operation of all modules. The control module 110 can receive the initial point cloud collected by the lidar module 120, perform preprocessing such as voxel filtering, ground removal, and DBSCAN noise reduction; it can also call the photovoltaic support template library to obtain point cloud sets of multiple template photovoltaic supports, run FPFH feature matching and NDT fine registration algorithms to calculate the target pose of the photovoltaic support; it can also generate installation parameters based on the target pose and output robotic arm control commands; and it can store template library data, scanned point clouds, calculation results, etc.

[0036] The lidar module 120 can be installed on the top or front of the photovoltaic module installation robot 100 to ensure that the scanning field of view covers the photovoltaic support to be inspected, and to perform three-dimensional scanning of the photovoltaic support to be inspected and the surrounding environment in real time, and output an initial point cloud set containing the geometry of the support.

[0037] The mobile module 130, such as the chassis, is used to enable the autonomous movement of the photovoltaic module installation robot 100, drive the lidar to traverse the area to be installed, and ensure that each set of brackets to be inspected can be scanned; and to precisely move the robot to the module installation position, providing a stable working reference for the operation of the robotic arm.

[0038] The installation module 140, such as a robotic arm, is used to perform grasping, positioning, and installation actions for photovoltaic modules (such as photovoltaic panels), and to receive installation parameters from the control module 110; and to move the end effector to the module storage location to grasp the photovoltaic module through the joint movement of the robotic arm; and to accurately place the photovoltaic module in the target installation position of the photovoltaic bracket according to the preset installation parameters, thereby completing the installation action.

[0039] It should be noted that, in practice, lidar can be a detachable device.

[0040] Please see Figure 2 , Figure 2 This is a flowchart of a photovoltaic support bracket identification and positioning method based on point cloud matching provided in an embodiment of this application; the method includes, but is not limited to, the following steps: S210. The photovoltaic support to be tested is scanned by lidar to obtain an initial point cloud set.

[0041] In this application embodiment, the lidar may include one of the following: mechanical lidar, solid-state lidar, triangulation lidar, etc., and is not limited thereto.

[0042] In a specific embodiment, the parameters of the lidar can be set before scanning the photovoltaic support to be inspected using lidar. The core parameters of lidar include scanning frequency, point cloud density, scanning angle range, and laser emission power. The settings of each parameter must be specifically considered in light of the structural characteristics of the photovoltaic support, the outdoor working environment, and the requirements of subsequent algorithms.

[0043] S220. Perform preprocessing operations on the initial point cloud set to obtain the target point cloud set.

[0044] The core purpose of the preprocessing operation is to correct the initial point cloud data: remove ground interference and discrete noise, reduce data redundancy, retain the core geometric features of the photovoltaic support, and ensure the accuracy and efficiency of identification and positioning.

[0045] In one possible embodiment, the preprocessing operation performed on the initial point cloud set to obtain the target point cloud set includes: performing a voxel filtering operation on the initial point cloud set to obtain a downsampled first point cloud set; performing plane fitting on the first point cloud set using a preset plane fitting algorithm to obtain a ground plane, and calculating the distance between each point in the first point cloud set and the ground plane to obtain multiple distances; determining points whose distances are less than a first preset distance as ground point clouds located within the ground plane; if the proportion of all ground point clouds in the first point cloud set is greater than a preset proportion, then removing all ground point clouds from the first point cloud set to obtain a second point cloud set; and performing noise reduction processing on the second point cloud set using the DBSCAN density clustering algorithm to remove discrete noise points to obtain the target point cloud set.

[0046] Among them, the voxel filtering operation divides the three-dimensional space into a uniform voxel grid, and retains only representative points (such as the center point and centroid point) in each voxel. While greatly reducing the number of point clouds, it fully preserves the core geometric features of the photovoltaic support structure, such as the beam orientation and intersections.

[0047] For example, the voxel filtering operation specifically includes: Step 1: For the initial point cloud set P = ( Let be the 3D coordinates of the i-th point in the initial point cloud set. , , Point clouds The three-dimensional spatial coordinates are voxelized, dividing the three-dimensional space into sections with side lengths of... A three-dimensional voxel grid, or voxel unit, where each grid corresponds to one voxel. ,in, This is the core control parameter for voxel filtering, and it needs to be set in conjunction with the structural characteristics of the photovoltaic support (such as the beam width). It is usually set to 1 / 3 to 1 / 2 of the beam width of the photovoltaic support; Step 2: Traverse each voxel unit And calculate the centroid of all points within the voxel. ,in, voxels The number of points contained within, the centroid = Let the coordinates of the representative point of this voxel be given; Step 3: the centroids of all voxels. The first point cloud set after downsampling is formed. { }, where m represents the total number of centroids.

[0048] Furthermore, to eliminate interference from ground point clouds, the first point cloud set after downsampling... A plane fitting method based on random sampling consistency is adopted. Three non-collinear points are randomly selected. Establish a (ground) planar model: Ax + By + Cz + D = 0, where A, B, and C are the normal vector components of the plane, and the plane normal vector n = ( )×( The result is calculated based on the cross product relationship, where D is a constant; then, the first point cloud set is calculated. Distance from each point to the ground plane Multiple distances are obtained; then, the first preset distance is set to... If the distance from a point to the ground plane is If the point is found to be in the ground plane, then it is determined to be a point in the ground plane; finally, the total number of points in the ground plane is counted. In the first cloud collection percentage (N is the first point cloud set) The total number of all points is greater than the set ratio. If this happens, these points in the ground plane are removed from the first point cloud set, resulting in a second point cloud set containing only photovoltaic support points. .

[0049] Furthermore, the second point cloud set was analyzed using DBSCAN density clustering. The core of noise reduction is to remove outliers (noise) by dynamically determining the neighborhood radius and the threshold of nearby points in the algorithm parameters.

[0050] For example, the DBSCAN density clustering noise reduction operation specifically includes: Step 1: Processing the second point cloud set First, construct a distance distribution matrix (recording the distance between any two points in the point cloud). Then, sort the row and column elements of the distance distribution matrix in ascending order to obtain a distance discrete curve. Second, fit the distance discrete curve to calculate the inflection point value (the inflection point corresponds to abrupt changes in point cloud density, distinguishing between dense support points and discrete noise points), thus obtaining candidate values ​​for the neighborhood radius. Step 3: Set the second point cloud any point in the middle Its neighborhood range is defined as: , For the second point cloud collection any point in, Point With point European distance, The neighborhood radius (i.e., the candidate value of the neighborhood radius obtained in step one) Step 4: Calculate the global point cloud density distribution and determine the nearest point threshold minpts. `minpts` represents the threshold for determining a core point. If the number of points in a point's neighborhood is greater than or equal to `minpts`, then that point is a core point. β is the scaling factor, and N is the set of the second point cloud. Total points This represents the sum of the number of neighboring points of all points. In this way, multiple densely distributed core points can be obtained. These core points and their neighboring points are then clustered, making the dense photovoltaic support points into effective clusters and the discrete noise points into outliers; Step 5: Candidate values ​​for the neighborhood radius... The nearest neighbor threshold (minpts) is applied to the DBSCAN density clustering algorithm to calculate the average density within clusters. and standard deviation , The coefficient of variation (CV) is calculated by taking the average number of neighboring points for all points. Thus, the clustering is most effective when the CV value is lowest. The optimal parameter set (the DBSCAN parameter set that minimizes the CV value, i.e., the neighborhood radius) is used. The target point cloud set is obtained by denoising using the nearest point threshold (minpts).

[0051] As can be seen, in this embodiment, for the initial point cloud set, the voxel filtering downsampling operation, random sampling consistency plane fitting operation, and DBSCAN dynamic parameter clustering noise reduction operation are used to significantly compress the amount of point cloud data while accurately removing ground interference and discrete noise, and fully preserving the core geometric features of the photovoltaic support. Finally, a high-quality target point cloud set is output, which not only improves the computational efficiency of subsequent support identification and positioning, but also ensures the accuracy of pose calculation.

[0052] S230. Retrieve the pre-stored photovoltaic support template library, which stores multiple support point cloud sets corresponding to various types of photovoltaic supports.

[0053] In this embodiment, photovoltaic (PV) supports come in different types, such as flat single-axis supports and fixed supports. The PV support template library contains pre-stored support point cloud sets that have undergone preprocessing (voxel filtering downsampling, random sampling consistency plane fitting, and DBSCAN dynamic parameter clustering noise reduction). Furthermore, the PV support template library may also include a point cloud feature set corresponding to each support point cloud set. The point cloud feature set may include feature information such as the FPFH histogram feature vector, normal vector, and curvature of the support point cloud set.

[0054] In a specific embodiment, the establishment of the photovoltaic support template library can be achieved by: acquiring 3D point cloud data of the photovoltaic support using a LiDAR scanning device; preprocessing the acquired point cloud such as denoising, filtering, and downsampling to ensure the quality of the point cloud and reduce unnecessary computation; or, using 3D models to establish a template library. Each point cloud has 3D coordinates and normal vector information. Effective feature information is extracted from the point cloud to establish a point cloud feature set, enabling matching within the target scene. Commonly used point cloud features include: FPFH histogram feature vectors, which effectively describe the local geometric features of points and are suitable for template matching; and normal vectors and curvature, which can further extract the structural features of the support based on normal vector and curvature information, helping to distinguish different beams or purlins.

[0055] S240. Calculate the similarity of multiple FPFH histogram features between the target point cloud set and the multiple support point cloud sets.

[0056] The FPFH histogram feature vector is a statistical representation of the "local geometry of the point cloud (such as beam orientation and intersection angles)," and the core differences in photovoltaic support systems lie in the geometry of the beams and intersections. Therefore, by comparing the similarity of the FPFH histograms of the two systems, different types of support systems can be effectively distinguished (the greater the structural difference, the lower the similarity).

[0057] Optional, please refer to Figure 3 , Figure 3 This is a flowchart of a method for determining FPFH histogram feature similarity according to an embodiment of this application. Step S240, calculating the multiple FPFH histogram feature similarities between the target point cloud set and the multiple support point cloud sets, may include... Figure 3 The steps shown are as follows: S31. Obtain the plane normal vector and principal direction vector of the ground plane, as well as the preset reference point located on the ground plane.

[0058] In this context, the ground plane refers to the ground plane determined during the plane fitting operation in step S220, which involves preprocessing the initial point cloud set. The plane normal vector of the ground plane is a vector perpendicular to the ground plane, and the principal direction vector of the ground plane is a vector parallel to the ground plane and consistent with the arrangement direction of the photovoltaic support beams. The principal direction vector can be obtained by performing PCA principal component analysis on the point cloud within the ground plane and solving for the eigenvector corresponding to the principal component with the largest variance. This vector is the "local direction vector," which essentially captures the main extension direction of the local area.

[0059] The reference point on the ground plane can be the centroid of multiple points within the ground plane.

[0060] S32. Establish a three-dimensional coordinate system based on the cross product relationship, with the preset reference point as the origin, the direction of the plane normal vector as the positive direction of the Z-axis, and the direction of the principal direction vector as the positive direction of the X-axis.

[0061] S33. Based on the preset K-nearest neighbor algorithm, perform plane fitting on the target point cloud set to obtain multiple local planes, and determine multiple local direction vectors corresponding to the multiple local planes through the PCA principal component analysis algorithm.

[0062] In a specific embodiment, each point in the target point cloud set is traversed, and the K nearest neighbor algorithm is used to find its K nearest neighbor points (K is a preset parameter that needs to be set in combination with the point cloud density and the size of the support components, such as 10 to 30 points); each point and its K nearest neighbors together constitute a "local point set". Since the horizontal and vertical beams of the photovoltaic support are continuous and regular components, their surface point clouds are dense and spatially continuous. The K nearest neighbor algorithm can ensure that each local point set contains only continuous points of the same component, avoiding point cloud overlap across components (such as horizontal and vertical beams).

[0063] Furthermore, after obtaining multiple "local point sets", the least squares method can be used to fit the planar model (consistent with the ground planar fitting operation principle in step S220). After fitting, multiple local planes are obtained, each of which precisely corresponds to the local surface of a certain component (horizontal beam / vertical beam) of the support. Then, the core extension direction of the component is extracted from the local planes according to the PCA principal component analysis algorithm, such as the horizontal extension direction of the horizontal beam and the vertical extension direction of the vertical beam.

[0064] For example, the PCA principal component analysis algorithm is used to calculate the local principal direction vector of the local plane and extract the horizontal / vertical beam structure of the photovoltaic support. Specifically, if the "X-axis component ratio ≥ 0.9" of a certain tangential direction (for example, the local principal direction vector is (0.98, 0.15, 0.02), the X-axis component ratio ≈ 0.98, which is close to the X-axis), then the beam extends along the X-axis direction and is determined to be a horizontal beam; if the "Y-axis component ratio ≥ 0.9" of a certain tangential direction (for example, the local principal direction vector is (0.12, 0.97, 0.03), the Y-axis component ratio ≈ 0.97, which is close to the Y-axis), then the beam extends along the Y-axis direction and is determined to be a vertical beam.

[0065] S34. Based on the multiple local direction vectors, the preset three-dimensional straight line fitting algorithm, and the three-dimensional coordinate system, multiple intersection points are determined, and the multiple intersection points represent the point cloud corresponding to the bracket connection position in the photovoltaic bracket to be detected.

[0066] Among them, the intersection point is the most distinctive structural feature of the support structure. For example, the intersection point of the horizontal beam and the vertical beam, its distribution and spacing directly determine the support structure type and are the core basis for FPFH feature matching.

[0067] In one possible embodiment, determining multiple intersection points based on the multiple local direction vectors, a preset three-dimensional line fitting algorithm, and the three-dimensional coordinate system includes: determining multiple axial extension directions of the multiple local direction vectors in the three-dimensional coordinate system, the axial extension directions including the X-axis extension direction and the Y-axis extension direction; merging multiple local planes with consistent and continuous axial extension directions into a support beam point cloud region, obtaining multiple support beam point cloud regions, each support beam point cloud region corresponding to a single support beam structure, the single support beam structure including a single support crossbeam and a single support vertical beam; determining points corresponding to the multiple support beam point cloud regions according to the target point cloud set, obtaining multiple fourth point cloud sets; extracting centerlines from the multiple fourth point cloud sets and performing three-dimensional line fitting based on the preset three-dimensional line fitting algorithm, obtaining multiple spatial centerlines; determining the multiple intersection points between the multiple spatial centerlines, the multiple intersection points including coplanar intersection points and skew intersection points, the skew intersection point being the midpoint of the common perpendicular of multiple spatial centerlines that are skew and have a spatial connection relationship.

[0068] Specifically, only local planes extending in the same axial direction (e.g., all in the X-axis direction) are merged to ensure that the merged area corresponds to a single beam frame with the same orientation, such as a horizontal beam extending along the X-axis direction or a vertical beam extending along the Y-axis direction. The center distance and the angle between the normal vectors of adjacent local planes are calculated. If the center distance is less than a preset distance threshold (e.g., 1.5 times the beam frame width) and the angle between the normal vectors is less than an angle threshold (e.g., 10°), they are determined to be continuous surfaces of the same beam frame and are merged.

[0069] For example, the three-dimensional straight line fitting operation specifically includes: for the fourth point cloud set corresponding to a single beam, the center trajectory (centerline) of the beam can be accurately located based on the point cloud density distribution through density clustering or centroid calculation; then, the extracted discrete centerline point set is fitted into a standard three-dimensional straight line equation using the following formula: , Let p be the set of centerline points corresponding to the j-th beam frame. For any three-dimensional coordinate point in the array, l is a candidate spatial line. This represents the square of the perpendicular distance from point p to the candidate line l. Let J be the fitted three-dimensional space centerline corresponding to the j-th beam. The operator for selecting the parameter that minimizes the objective function represents choosing the line l from all possible spatial lines that minimizes subsequent error terms. Therefore, the essence of this formula is least-squares fitting, finding a spatial line l such that the set of centerline points of the j-th beam is... The sum of the squares of the distances from all points to this straight line is minimized, so that the fitted straight line (the three-dimensional center line of the beam frame) can most accurately match the actual center trajectory of the beam frame.

[0070] Furthermore, it can be understood that while the horizontal and vertical beams of a photovoltaic support system are theoretically orthogonal and coplanar, slight errors (such as height differences or angle deviations) can occur during actual installation / scanning, causing their spatial center lines to become skew lines. In this case, the two lines do not intersect in the traditional geometric sense, but they are physically connected. For skew lines L1 and L2, the "midpoint of the common perpendicular" is used as the equivalent intersection point. The common perpendicular of skew lines is the shortest distance segment connecting the two lines, and the midpoint of the common perpendicular... , , These are the two endpoints of the shortest distance line segment.

[0071] S35. Based on the three-dimensional coordinate system, the area around the intersection point at a second preset distance is taken as the first feature area, and multiple first feature areas are obtained.

[0072] S36. Determine the points corresponding to the plurality of first feature regions based on the target point cloud set to obtain a third point cloud set.

[0073] In a specific embodiment, a neighborhood radius r is established at the intersection point to obtain multiple first feature regions. Each region corresponds to a sub-point cloud set. The multiple sub-point cloud sets corresponding to multiple first feature regions constitute a third point cloud set. Therefore, it is only necessary to match and compare the point clouds of the feature regions, without having to perform feature comparison on all points, thereby improving the efficiency and accuracy of photovoltaic bracket positioning.

[0074] S37. Calculate the similarity of multiple FPFH histogram features between the third point cloud set and the multiple support point cloud sets.

[0075] In one possible embodiment, calculating the similarity of multiple FPFH histogram features between the third point cloud set and the multiple support point cloud sets includes: determining a first target FPFH feature set corresponding to the third point cloud set, wherein the first target FPFH feature set includes multiple first target FPFH histogram feature vectors corresponding one-to-one with the multiple first feature regions; obtaining multiple second target FPFH feature sets pre-stored in the photovoltaic support template library, wherein the multiple second target FPFH feature sets correspond one-to-one with the multiple support point cloud sets, wherein a single support point cloud set includes multiple second feature regions, and a single second target FPFH feature set includes multiple second target FPFH histogram feature vectors in the single support point cloud set that correspond one-to-one with the multiple second feature regions; and calculating the similarity of the multiple FPFH histogram features between the first target FPFH feature set and the multiple second target FPFH feature sets respectively.

[0076] In this embodiment, the second feature region of the template library and the first feature region of the scaffold to be detected are extracted based on the same rule (neighborhood radius of the intersection point), ensuring that the feature vectors of the two have an aligned comparison benchmark and avoiding matching errors caused by different feature region definitions.

[0077] In a specific embodiment, after determining the third point cloud set corresponding to the photovoltaic support to be detected, it is necessary to calculate multiple first target FPFH histogram feature vectors corresponding to multiple sub-point cloud sets of multiple first feature regions in the third point cloud set, so as to form a first target FPFH feature set. A single FPFH histogram feature vector can characterize the local geometric features of the corresponding first feature region.

[0078] Specifically, the specific calculation methods for the FPFH histogram eigenvectors include: Step 1: For a point p within a certain first feature region in the third point cloud set, first extract its neighborhood N(p) (the set of neighboring points around point p). For any pair of points (p, q) within the neighborhood, then use the ternary angle feature (α, q) to... α) quantifies the positional relationship between this point and its neighboring points. Here, α is the angle between the line vector connecting point p to point q and the normal vector of point q. Let θ be the angle between the line vector connecting point p to point q and the normal vector of point p, and let θ be the angle formed by the projections of the normal vectors of point p and point q onto a plane perpendicular to the line vector connecting them.

[0079] Step 2: For all pairs of points (α, The frequencies of θ are statistically analyzed according to a preset interval (e.g., 11 intervals for each angle) to obtain SPFH(p). SPFH(p) is a histogram vector with a dimension of 3×11=33, which encodes the local geometric relationships of the neighborhood of point p.

[0080] Step 3: Through neighborhood weighted summarization, the features of points within the neighborhood are merged into the features of point p. The formula is: FPFH(p) = SPFH(p) + FPFH(p) is the final FPFH feature vector of point p, SPFH(p) is the simple point feature histogram vector of point p itself, N(p) is the number of points in the neighborhood of point p, N(q) is the number of points in the neighborhood of point q, and SPFH(q) is the simple point feature histogram vector corresponding to point q in the neighborhood of point p.

[0081] Step 4: Normalize FPFH(p) as follows to eliminate the influence of differences in the number of neighboring points: , Let be the normalized FPFH eigenvector of point p.

[0082] Step 5: [This involves] all points within the first feature region. Based on histogram statistics, the histograms are merged into a high-dimensional histogram vector, resulting in the first target FPFH histogram feature vector corresponding to the first feature region.

[0083] In one possible embodiment, the first target FPFH feature set includes m first feature regions and their corresponding m first target FPFH histogram feature vectors, and the second target FPFH feature set includes n second feature regions and their corresponding n second target FPFH histogram feature vectors, where n and m are both positive integers; the step of calculating the similarity of the multiple FPFH histogram features between the first target FPFH feature set and the multiple second target FPFH feature sets includes: calculating the m first target FPFH histogram feature vectors based on a preset Euclidean distance similarity algorithm. Multiple similarities are calculated between the n second target FPFH histogram feature vectors; the second target FPFH histogram feature vector with the highest similarity to each first target FPFH histogram feature vector is determined as a valid matching pair, resulting in multiple valid matching pairs; and a set of valid matching pairs is determined based on the multiple valid matches; multiple sets of valid matching pairs are obtained corresponding to the first target FPFH feature set and the multiple second target FPFH feature sets respectively; based on a preset set similarity algorithm, the similarity of the multiple FPFH histogram features corresponding to the multiple sets of valid matching pairs is calculated.

[0084] For example, the formula for calculating the similarity of FPFH histogram feature vectors based on a preset Euclidean distance similarity algorithm is: Sim( )=exp(- ),in, The i-th feature region of the second target FPFH feature set in the preset photovoltaic support template library (T is the identifier of the template point cloud set, which is an abbreviation of Template). The j-th feature region of the first target FPFH feature set of the photovoltaic support to be detected (Q is the identifier of the target point cloud set, which is an abbreviation for Query). The second target FPFH histogram feature vector corresponding to the i-th feature region mentioned above. The first target FPFH histogram feature vector corresponding to the j-th feature region mentioned above. denoted as the variance of the feature vector, and denoted as a preset parameter used to adjust the sensitivity of similarity to distance. Thus, if the first target FPFH feature set includes m first feature regions and the second target FPFH feature set includes n second feature regions, then m×n similarities corresponding to multiple region combinations can be obtained.

[0085] Furthermore, for each (m in total) first target FPFH histogram feature vector, there will be n corresponding similarities. The one with the highest similarity and its associated second target FPFH histogram feature vector are selected. This first target FPFH histogram feature vector with the highest similarity and the second target FPFH histogram feature vector are considered a valid matching pair. Repeating the above steps yields m valid matching pairs, corresponding to a set M of valid matching pairs, i.e. ( Thus, if there is a single valid matching pair between the target point cloud set and a single support point cloud set in the template library, then there are multiple valid matching pairs between the target point cloud set and multiple support point cloud sets respectively.

[0086] Furthermore, based on a preset set similarity algorithm (such as the proportion of matching pairs and the average similarity of matching pairs), the set-level similarity of each valid matching pair set is calculated as the FPFH histogram feature similarity between the target point cloud set and the support point cloud set support, thereby obtaining multiple FPFH histogram feature similarities.

[0087] As can be seen, in this embodiment, by constructing a unified coordinate system, analyzing the local structure of the support structure, locating the core connection points, focusing on the feature regions, and calculating the FPFH feature similarity, both the differences in support structure pose and environmental interference are eliminated, and the matching range is limited to the connection point region where the support structure is most distinguishable. This process significantly reduces the amount of computation and achieves high-precision feature matching between the support structure to be detected and the support structures in the template library. This ensures the accuracy of support structure type identification and improves matching efficiency, providing reliable support for the subsequent accurate pose calculation of photovoltaic support structures.

[0088] S250. Select the standard point cloud set with the highest similarity from the multiple support point cloud sets, and the photovoltaic support type corresponding to the standard point cloud set matches the photovoltaic support to be detected.

[0089] The standard point cloud set is the set of support point clouds with the highest similarity to the FPFH histogram features among the multiple support point cloud sets.

[0090] S260. Align the target point cloud set and the standard point cloud set according to the normal distribution transformation algorithm to obtain the target pose of the photovoltaic bracket to be detected.

[0091] Understandably, the Normal Distribution Transform (NDT) algorithm is a point cloud registration method. Its core is to divide the point cloud into multiple three-dimensional voxel units. The distribution of points in each unit is described by a Gaussian probability model (mean and covariance). Alignment is achieved by adjusting the rotation and translation parameters of the template point cloud so that after the template point cloud is transformed, its points have the "highest probability of appearing" in the voxel probability model of the target point cloud. Finally, a transformation matrix is ​​obtained that allows the two point cloud sets to accurately overlap, realizing the spatial alignment of the template and the target point cloud sets, thereby determining the pose of the scaffold to be detected.

[0092] In one possible embodiment, the lidar is mounted on a paving robot, which performs the photovoltaic module paving task. After aligning the target point cloud set and the standard point cloud set according to the normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected, the method further includes: determining the paving parameters of the target photovoltaic module based on the target pose of the photovoltaic support to be detected, the paving parameters including the paving position, paving angle, and paving gap of the target photovoltaic module on the photovoltaic support to be detected; generating a first control command for the robotic arm based on the paving parameters to control the robotic arm to complete the paving task for the target photovoltaic module according to the paving parameters.

[0093] In a specific embodiment, the spatial coordinates (X, Y, Z) of the target pose of the bracket are used as a reference. Combined with the component size (length, width, thickness) and the bracket installation node position (such as the beam spacing), the precise landing point coordinates of the photovoltaic module on the photovoltaic bracket to be tested are determined (such as the specific position of the bracket corresponding to the four corners of the component). According to the target attitude of the bracket (including roll angle, pitch angle, yaw angle), the installation angle of the photovoltaic module is calculated in reverse compensation. Combined with the bracket structural design requirements (such as drainage gap and heat dissipation gap between components) and the position of adjacent installed components, the lateral / longitudinal gap between the current photovoltaic module and the surrounding components is calculated.

[0094] Optional, please refer to Figure 4 , Figure 4This is a flowchart of a method for determining the target pose of a photovoltaic support according to an embodiment of this application. Step S260, which involves aligning the target point cloud set and the standard point cloud set according to a normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected, may include... Figure 4 The steps shown are as follows: S41. Determine the set of target valid matching pairs with the highest FPFH histogram feature similarity among the multiple valid matching pair sets.

[0095] S42. Calculate the rigid transformation matrix between the standard point cloud set and the target point cloud set based on the set of valid matching pairs.

[0096] Specifically, the formula for calculating the rigid transformation matrix [R|t] is as follows: E(R,t)= ; Where E(R,t) is the alignment error (sum of squared distances) between the standard point cloud set and the target point cloud set, R is the rotation matrix (a 3×3 matrix describing the rotation of the standard point cloud set), and t is the translation vector (a 3×1 vector describing the translation of the standard point cloud set). For points in a standard point cloud set, Let M be the set of points in the target point cloud, and M be the set of valid matching pairs. Let R be the squared Euclidean distance between the vectors. Then, by minimizing the sum of the squared distances between the standard point cloud set and the target point cloud set after rotation and translation, the optimal R and t are obtained by using the singular value decomposition (SVD) algorithm, thus obtaining the initial rigid transformation matrix [R|t].

[0097] Furthermore, after the initial alignment (matching) and obtaining the initial rigid transformation matrix, the initial alignment result needs to be evaluated, and the matching error is calculated using the following formula: e= ; Where e is the average matching error, To effectively match the number of elements in set M.

[0098] It can be seen that the logic for calculating the matching error is to calculate the average distance of all matching pairs until the error is less than the convergence threshold. If this is confirmed, the initial alignment operation is successful, and the initial rigid transformation matrix obtained is valid.

[0099] S43. The rigid transformation matrix is ​​corrected according to the normal distribution transformation algorithm to obtain the target rigid transformation matrix, which is used to align the target point cloud set with the standard point cloud set.

[0100] The core of NDT (Normal Transform) fine registration is to transform the "geometric alignment" of point clouds into "probability distribution matching". By dividing the points in the target point cloud set into voxel units and establishing a Gaussian model, the standard point cloud set is transformed to fit as closely as possible to the probability distribution of the target point cloud set, ultimately achieving high-precision pose alignment. Specifically, the NDT algorithm includes the following steps: Step 1: Divide the target point cloud set into multiple 3D voxel units. The point cloud within each unit will be modeled as a multidimensional Gaussian distribution, with the following formulas: ; ; in, This represents the probability that point p belongs to the k-th voxel unit; Let be the mean vector of the set of points within the k-th voxel unit. The number of points within a cell. Let k be a 3D point cloud coordinate point contained within the k-th voxel unit. Let T be the covariance matrix of the set of points within the k-th voxel unit (describing the dispersion of the point distribution), and let T be the transpose operator, indicating that the vector... Converting column vectors to row vectors is used to meet the dimensional requirements of matrix multiplication.

[0101] Step 2: Adjust the rotation matrix R and translation vector t of the standard point cloud set so that the transformed standard point cloud set has the highest probability of appearing under the probability distribution of the target point cloud set. Since the joint probability of multiple points is the product of the probabilities of individual points, taking the logarithm transforms it into a summation (to avoid numerical underflow). The optimized formula is: ; in, This represents the points after a rigid transformation of a standard point cloud set; , Representing points respectively The mean and covariance of the voxel unit. Let R and t be points in the standard point cloud set, respectively, and let R and t be the rotation matrix and translation vector before registration. , These are the optimal rotation matrix and translation vector obtained after fine registration using the NDT algorithm. It can be seen that by adjusting R and t, maximizing the sum of the log-likelihoods of all standard point cloud sets after point transformation is equivalent to making the spatial distribution of the standard point cloud best match that of the target point cloud set.

[0102] Step 3: Transform maximizing the likelihood into minimizing the negative log-likelihood using the following formula, and then perform gradient descent optimization: ; Here, E(R,t) represents the objective function of the negative log-likelihood.

[0103] It can be seen that the gradient descent process involves updating the pose parameters. (3 translations) , , These represent the translation of the standard point cloud along the X, Y, and Z axes, and the rotation of three Euler angles, respectively. , , (These represent the rotation angles of the support around the X, Y, and Z axes, respectively). Then, the gradient is calculated in each iteration. (J is the Jacobian matrix, For the objective function In the current position The gradient vector at point T, where T is the transpose. For the standard point cloud with current pose parameters Transformed 3D coordinate points (The mean vector of the voxel unit corresponding to the target point cloud).

[0104] Next, update the pose using the Hessian matrix H: ( This is the updated pose parameter vector after the (k+1)th iteration. This is the current pose parameter vector at the k-th iteration. Let |gradient_vector_k_th_iteration| be the gradient vector at the k-th iteration, until the error change, translation, and rotation angle satisfy the convergence thresholds, i.e., ||. |< , | |< , | |< ( , , The objective functions between the two iterations are respectively The difference between the translation vector and the rotation matrix. , , (These are the threshold values ​​for error change, translation change, and rotation change, respectively), thus obtaining the optimized target rigid transformation matrix. .

[0105] S44. Obtain the original pose of the photovoltaic bracket corresponding to the standard point cloud set.

[0106] The original pose is the preset reference pose of the standard photovoltaic bracket in the template library, which includes the complete spatial position (X, Y, Z coordinates) and attitude (rotation angles around the X, Y, Z axes, i.e. roll angle, pitch angle, yaw angle). It is usually based on the "world coordinate system" or "global coordinate system of photovoltaic power station" as the reference reference and is the "anchor point" for pose calculation.

[0107] S45. Obtain the target pose of the photovoltaic bracket to be detected based on the original pose and the target rigid transformation matrix.

[0108] In a specific embodiment, the target rigid transformation matrix describes the spatial transformation (rotation, translation) required to align the standard point cloud set to the target point cloud set. By applying this transformation to the original pose of the standard support, the actual pose (target pose) of the photovoltaic support to be tested can be obtained.

[0109] As can be seen, in this embodiment, by using lidar to collect point clouds, FPFH feature matching to lock the bracket type, NDT algorithm to achieve high-precision alignment of point clouds, rigid transformation matrix to optimize pose solution, and robotic arm command generation to drive operation, the precise positioning of photovoltaic brackets and the automated, high-precision installation of photovoltaic modules are achieved, which greatly improves installation efficiency and consistency.

[0110] In this embodiment, the centroid of the fitted ground plane can be used as the origin to establish the photovoltaic support coordinate system. For example, please refer to [link to example]. Figure 5 , Figure 5 This is a schematic diagram of a photovoltaic support coordinate system provided in an embodiment of this application, as shown below. Figure 5 As shown, the origin of the coordinate system is the centroid of the point cloud fitting plane of a specific feature area on the photovoltaic support (such as the connection point between the horizontal and vertical beams, or the core mounting surface of the support). The X-axis extends along the horizontal beam of the photovoltaic support and is one of the main extension directions of the support; the Y-axis is perpendicular to the X-axis and lies within the "fitting plane" (support mounting surface), usually corresponding to the lateral width of the support; the Z-axis is perpendicular to the fitting plane and points upwards, corresponding to the height direction perpendicular to the ground.

[0111] It is known that the photovoltaic support coordinate system is the unified benchmark for all subsequent spatial calculations: the point cloud data and pose parameters (position and angle) of the support are defined based on this coordinate system, avoiding coordinate ambiguity in different stages; the installation position, angle and other parameters of the photovoltaic modules are also calculated with reference to this coordinate system to ensure the spatial alignment accuracy between the modules and the support; the coordinate system is bound to the core structure of the support (beams, feature areas), so that subsequent point cloud processing and feature extraction focus on the key areas of the support, improving the calculation efficiency and the reliability of the results.

[0112] Please see Figure 6 , Figure 6 This is a schematic diagram of a local coordinate system for FPFH feature vectors provided in an embodiment of this application, as shown below. Figure 6 As shown, a local coordinate system (with coordinate axes u, v, w) is established with point P1 as the center. This is the basic framework for FPFH feature extraction. The establishment of the local coordinate system eliminates the influence of global pose on angle calculation.

[0113] Wherein, point P1 is the target point for feature extraction, point P2 is any neighboring point in the neighborhood of point P1 (obtained through the K-nearest neighbor algorithm), n1 is the normal vector of point P1 (a vector perpendicular to the local plane containing point P1), n2 is the normal vector of point P2, and d is the vector connecting P1 and P2. The coordinate axes are u, v, and w. The u-axis is aligned with the direction of the normal vector n1 of P1 (i.e., u = n1); the v-axis lies in the plane perpendicular to u and is aligned with the projection direction of the connecting vector d; the w-axis is obtained by the cross product of u and v, forming a right-handed coordinate system.

[0114] It can be seen that FPFH quantifies the three angles (α, α, α) between P1 and P2. Angles α, θ, and y are used to characterize local geometric structures. These three angles are the core quantitative indicators of the features. Angle α: The angle between the normal vector n1 of P1 and the normal vector n2 of P2, reflecting the "normal difference" between the local planes containing the two points (e.g., the normal angle between a horizontal beam and a vertical beam is usually 90°); The angle between the normal vector n1 of P1 and the connecting vector d reflects the "spatial orientation" of neighboring points relative to the target point (e.g., the neighborhood points of points on the beam surface). (Usually close to 90°); Angle θ: The angle between the projection of the line vector d connecting points P1 and P2 onto the normal vector n1 and the normal vector n2, reflecting the "directional difference" of the normal vectors of neighboring points in the local plane.

[0115] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a photovoltaic bracket identification and positioning device 700 provided in an embodiment of this application. The photovoltaic bracket identification and positioning device 700 includes: a scanning unit 701 and a processing unit 702, wherein: The scanning unit 701 is used to scan the photovoltaic support to be detected using a lidar to obtain an initial point cloud set. The processing unit 702 is configured to perform preprocessing operations on the initial point cloud set to obtain a target point cloud set; retrieve a pre-stored photovoltaic support template library, which stores multiple support point cloud sets corresponding to various types of photovoltaic supports; calculate the similarity of multiple FPFH histogram features between the target point cloud set and the multiple support point cloud sets; and select the standard point cloud set with the highest similarity from the multiple support point cloud sets, wherein the photovoltaic support type corresponding to the standard point cloud set matches the photovoltaic support to be detected; and align the target point cloud set and the standard point cloud set according to a normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected.

[0116] In one embodiment, the preprocessing operation performed on the initial point cloud set to obtain the target point cloud set includes: performing a voxel filtering operation on the initial point cloud set to obtain a downsampled first point cloud set; performing plane fitting on the first point cloud set using a preset plane fitting algorithm to obtain a ground plane, and calculating the distance between each point in the first point cloud set and the ground plane to obtain multiple distances; determining points whose distances are less than a first preset distance as ground point clouds located within the ground plane; if the proportion of all ground point clouds in the first point cloud set is greater than a preset proportion, then removing all ground point clouds from the first point cloud set to obtain a second point cloud set; and performing noise reduction processing on the second point cloud set using the DBSCAN density clustering algorithm to remove discrete noise points to obtain the target point cloud set.

[0117] In one embodiment, calculating the similarity of multiple FPFH histogram features between the target point cloud set and the multiple support point cloud sets includes: obtaining the plane normal vector and principal direction vector of the ground plane, and a preset reference point located on the ground plane; establishing a three-dimensional coordinate system based on the cross product relationship with the preset reference point as the origin, the direction of the plane normal vector as the positive Z-axis, and the direction of the principal direction vector as the positive X-axis; performing plane fitting on the target point cloud set based on a preset K-nearest neighbor algorithm to obtain multiple local planes, and determining the multiple local planes using the PCA principal component analysis algorithm. Multiple local direction vectors corresponding to the plane are used; multiple intersection points are determined based on the multiple local direction vectors, a preset three-dimensional straight line fitting algorithm, and the three-dimensional coordinate system. The multiple intersection points represent the point cloud corresponding to the connection position of the photovoltaic bracket in the photovoltaic bracket to be detected; based on the three-dimensional coordinate system, the area around the intersection point at a second preset distance is taken as the first feature area to obtain multiple first feature areas; the points corresponding to the multiple first feature areas are determined according to the target point cloud set to obtain a third point cloud set; the similarity of multiple FPFH histogram features between the third point cloud set and the multiple bracket point cloud sets is calculated.

[0118] In one embodiment, determining multiple intersection points based on the multiple local direction vectors, a preset three-dimensional line fitting algorithm, and the three-dimensional coordinate system includes: determining multiple axial extension directions of the multiple local direction vectors in the three-dimensional coordinate system, the axial extension directions including the X-axis extension direction and the Y-axis extension direction; merging multiple local planes with consistent and continuous axial extension directions into a support beam point cloud region, obtaining multiple support beam point cloud regions, each support beam point cloud region corresponding to a single support beam structure, the single support beam structure including a single support crossbeam and a single support vertical beam; determining points corresponding to the multiple support beam point cloud regions based on the target point cloud set, obtaining multiple fourth point cloud sets; extracting centerlines from the multiple fourth point cloud sets and performing three-dimensional line fitting based on the preset three-dimensional line fitting algorithm, obtaining multiple spatial centerlines; determining the multiple intersection points between the multiple spatial centerlines, the multiple intersection points including coplanar intersection points and skew intersection points, the skew intersection point being the midpoint of the common perpendicular of multiple spatial centerlines that are skew and have a spatial connection relationship.

[0119] In one embodiment, calculating the similarity of multiple FPFH histogram features between the third point cloud set and the multiple support point cloud sets includes: determining a first target FPFH feature set corresponding to the third point cloud set, wherein the first target FPFH feature set includes multiple first target FPFH histogram feature vectors corresponding one-to-one with the multiple first feature regions; obtaining multiple second target FPFH feature sets pre-stored in the photovoltaic support template library, wherein the multiple second target FPFH feature sets correspond one-to-one with the multiple support point cloud sets, wherein a single support point cloud set includes multiple second feature regions, and a single second target FPFH feature set includes multiple second target FPFH histogram feature vectors in the single support point cloud set that correspond one-to-one with the multiple second feature regions; and calculating the similarity of the multiple FPFH histogram features between the first target FPFH feature set and the multiple second target FPFH feature sets respectively.

[0120] In one embodiment, the first target FPFH feature set includes m first feature regions and their corresponding m first target FPFH histogram feature vectors, and the second target FPFH feature set includes n second feature regions and their corresponding n second target FPFH histogram feature vectors, where n and m are both positive integers; the step of calculating the similarity of the multiple FPFH histogram features between the first target FPFH feature set and the multiple second target FPFH feature sets includes: calculating the similarity between the m first target FPFH histogram feature vectors and the multiple second target FPFH feature sets based on a preset Euclidean distance similarity algorithm. Multiple similarities are calculated among the n second target FPFH histogram feature vectors; the second target FPFH histogram feature vector with the highest similarity to each first target FPFH histogram feature vector is determined as a valid matching pair, resulting in multiple valid matching pairs; and a set of valid matching pairs is determined based on the multiple valid matches; multiple sets of valid matching pairs are obtained corresponding to the first target FPFH feature set and the multiple second target FPFH feature sets, respectively; and the similarity of the multiple FPFH histogram features corresponding to the multiple sets of valid matching pairs is calculated based on a preset set similarity algorithm.

[0121] In one embodiment, the standard point cloud set is the set of support point clouds with the highest similarity to the FPFH histogram features among the plurality of support point cloud sets; aligning the target point cloud set and the standard point cloud set according to the normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected includes: determining the target effective matching pair set with the highest similarity to the FPFH histogram features among the plurality of effective matching pair sets; calculating the rigid transformation matrix between the standard point cloud set and the target point cloud set according to the effective matching pair set; correcting the rigid transformation matrix according to the normal distribution transformation algorithm to obtain the target rigid transformation matrix, the target rigid transformation matrix being used to align the target point cloud set with the standard point cloud set; obtaining the original pose of the photovoltaic support corresponding to the standard point cloud set; and obtaining the target pose of the photovoltaic support to be detected according to the original pose and the target rigid transformation matrix.

[0122] In one embodiment, the lidar is mounted on a paving robot, which performs the photovoltaic module paving task. After aligning the target point cloud set and the standard point cloud set according to the normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected, the method further includes: determining the paving parameters of the target photovoltaic module based on the target pose of the photovoltaic support to be detected, the paving parameters including the paving position, paving angle, and paving gap of the target photovoltaic module on the photovoltaic support to be detected; generating a first control command for the robotic arm based on the paving parameters to control the robotic arm to complete the paving task for the target photovoltaic module according to the paving parameters.

[0123] As can be seen, in this embodiment, the photovoltaic support identification and positioning scheme can accurately adapt to the regular beam structure of the photovoltaic support, effectively avoid the interference and dispersion of ground point clouds on site, and realize the automatic identification and high-precision spatial pose calculation of the photovoltaic support.

[0124] In specific implementations, the photovoltaic bracket identification and positioning device 700 described in the embodiments of the present invention can also execute other implementations described in the photovoltaic bracket identification and positioning method based on point cloud matching provided in the embodiments of the present invention, which will not be repeated here.

[0125] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include a processor, a memory, a communication interface, and one or more programs. The processor, memory, and communication interface can be interconnected via a bus. The one or more programs are stored in the memory and configured to be executed by the processor. In this embodiment, the programs include instructions for performing the following steps: The photovoltaic support under test is scanned using lidar to obtain an initial point cloud set; Perform preprocessing operations on the initial point cloud set to obtain the target point cloud set; Retrieve a pre-stored photovoltaic support template library, which stores multiple support point cloud sets corresponding to various types of photovoltaic supports; Calculate the similarity of multiple FPFH histogram features between the target point cloud set and the multiple support point cloud sets, and select the standard point cloud set with the highest similarity from the multiple support point cloud sets, wherein the photovoltaic support type corresponding to the standard point cloud set matches the photovoltaic support to be detected; The target point cloud set and the standard point cloud set are aligned using a normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected.

[0126] In a specific embodiment, the above procedure further includes causing the electronic device to perform some or all of the steps of any of the methods described in the above method embodiments.

[0127] This application also provides a computer-readable storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0128] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0129] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0130] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0131] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0132] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0133] The steps of the methods or algorithms described in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, EPROM, electrically erasable programmable read-only memory (EEPROM), registers, hard disk, portable hard disk, read-only optical disk (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Furthermore, the ASIC can reside in a terminal device or management device. Alternatively, the processor and storage medium can exist as discrete components in the terminal device or management device.

[0134] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in the embodiments of this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated.

[0135] The aforementioned computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media.

[0136] The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0137] The modules / units included in the various devices and products described in the above embodiments can be software modules / units, hardware modules / units, or a combination of both. For example, for devices and products applied to or integrated into a chip, all modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits. For devices and products applied to or integrated into a chip module, all modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The implementation is achieved through a software program that runs on the processor integrated within the chip module. The remaining modules / units (if any) can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into terminal equipment, each of their modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components within the terminal equipment. Alternatively, at least some modules / units can be implemented through a software program that runs on the processor integrated within the terminal equipment, while the remaining modules / units (if any) can be implemented using hardware methods such as circuits.

[0138] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above descriptions are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A photovoltaic support bracket identification and positioning method based on point cloud matching, characterized in that, include: The photovoltaic support under test is scanned using lidar to obtain an initial point cloud set; Perform preprocessing operations on the initial point cloud set to obtain the target point cloud set; Retrieve a pre-stored photovoltaic support template library, which stores multiple support point cloud sets corresponding to various types of photovoltaic supports; Obtain the plane normal vector and principal direction vector of the ground plane, as well as the preset reference point located on the ground plane; A three-dimensional coordinate system is established based on the cross product relationship, with the preset reference point as the origin, the direction of the plane normal vector as the positive Z-axis, and the direction of the principal direction vector as the positive X-axis. The target point cloud set is fitted with a plane based on a preset K-nearest neighbor algorithm to obtain multiple local planes, and multiple local direction vectors corresponding to the multiple local planes are determined by PCA principal component analysis algorithm. Multiple intersection points are determined based on the multiple local direction vectors, the preset three-dimensional straight line fitting algorithm, and the three-dimensional coordinate system. The multiple intersection points represent the point cloud corresponding to the connection position of the photovoltaic bracket in the photovoltaic bracket to be detected. The intersection points are the most distinguishable structural features of the bracket, including the intersection of the horizontal beam and the vertical beam, which are used to determine the bracket type. Based on the three-dimensional coordinate system, the area around the intersection point at a second preset distance is taken as the first feature area, and multiple first feature areas are obtained; Based on the target point cloud set, the points corresponding to the plurality of first feature regions are determined to obtain a third point cloud set; Calculate the similarity of multiple FPFH histogram features between the third point cloud set and the multiple support point cloud sets, and select the standard point cloud set with the highest similarity from the multiple support point cloud sets. The photovoltaic support type corresponding to the standard point cloud set matches the photovoltaic support to be detected. The target point cloud set and the standard point cloud set are aligned using a normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected.

2. The method according to claim 1, characterized in that, The step of performing preprocessing operations on the initial point cloud set to obtain the target point cloud set includes: Perform a voxel filtering operation on the initial point cloud set to obtain a downsampled first point cloud set; A preset plane fitting algorithm is used to perform plane fitting on the first point cloud set to obtain a ground plane, and the distance between each point in the first point cloud set and the ground plane is calculated to obtain multiple distances; Points whose distance is less than a first preset distance are identified as ground point clouds located within the ground plane; If the proportion of all ground point clouds in the first point cloud set is greater than a preset proportion, then all ground point clouds are removed from the first point cloud set to obtain the second point cloud set. The DBSCAN density clustering algorithm is used to denoise the second point cloud set, removing discrete noise points to obtain the target point cloud set.

3. The method according to claim 2, characterized in that, The step of determining multiple intersection points based on the multiple local direction vectors, a preset three-dimensional line fitting algorithm, and the three-dimensional coordinate system includes: Determine multiple axial extension directions of the plurality of local direction vectors in the three-dimensional coordinate system, wherein the axial extension directions include the X-axis extension direction and the Y-axis extension direction; Multiple local planes that are axially extended in the same direction and continuously distributed are merged into a support beam point cloud region to obtain multiple support beam point cloud regions. Each support beam point cloud region corresponds to a single support beam structure, and the single support beam structure includes a single support horizontal beam and a single support vertical beam. Based on the target point cloud set, the points corresponding to the multiple support beam point cloud regions are determined respectively, resulting in multiple fourth point cloud sets; Based on the preset three-dimensional straight line fitting algorithm, the center lines of the multiple fourth point cloud sets are extracted and three-dimensional straight line fitting is performed to obtain multiple spatial center lines; Determine the multiple intersection points between the multiple spatial center lines, including coplanar intersection points and skew intersection points, wherein the skew intersection point is the midpoint of the common perpendicular of multiple spatial center lines that are skew and have a spatial connection relationship.

4. The method according to claim 3, characterized in that, The calculation of multiple FPFH histogram feature similarities between the third point cloud set and the multiple support point cloud sets includes: Determine the first target FPFH feature set corresponding to the third point cloud set, wherein the first target FPFH feature set includes multiple first target FPFH histogram feature vectors that correspond one-to-one with the multiple first feature regions; Obtain multiple second target FPFH feature sets pre-stored in the photovoltaic support template library. The multiple second target FPFH feature sets correspond one-to-one with the multiple support point cloud sets. A single support point cloud set includes multiple second feature regions. A single second target FPFH feature set includes multiple second target FPFH histogram feature vectors in a single support point cloud set that correspond one-to-one with the multiple second feature regions. Calculate the similarity of the multiple FPFH histogram features between the first target FPFH feature set and the multiple second target FPFH feature sets.

5. The method according to claim 4, characterized in that, The first target FPFH feature set includes m first feature regions and their corresponding m first target FPFH histogram feature vectors, and the second target FPFH feature set includes n second feature regions and their corresponding n second target FPFH histogram feature vectors, where n and m are both positive integers; The step of calculating the similarity of the multiple FPFH histogram features between the first target FPFH feature set and the multiple second target FPFH feature sets includes: Based on a preset Euclidean distance similarity algorithm, multiple similarities are calculated between the m first target FPFH histogram feature vectors and the n second target FPFH histogram feature vectors; The second target FPFH histogram feature vector with the highest similarity to each first target FPFH histogram feature vector is determined as a valid matching pair, and multiple valid matching pairs are obtained; and a set of valid matching pairs is determined based on the multiple valid matches. Obtain multiple sets of valid matching pairs corresponding to the first target FPFH feature set and the multiple second target FPFH feature sets, respectively; Based on a preset set similarity algorithm, the similarity of the multiple FPFH histogram features corresponding to the multiple sets of valid matching pairs is calculated.

6. The method according to claim 5, characterized in that, The standard point cloud set is the set of support point clouds with the highest similarity to the FPFH histogram features among the multiple support point cloud sets; The step of aligning the target point cloud set and the standard point cloud set according to the normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected includes: Determine the set of target valid matching pairs with the highest FPFH histogram feature similarity among the plurality of valid matching pair sets; Calculate the rigid transformation matrix between the standard point cloud set and the target point cloud set based on the set of valid matching pairs; The rigid transformation matrix is ​​corrected according to the normal distribution transformation algorithm to obtain the target rigid transformation matrix, which is used to align the target point cloud set with the standard point cloud set. Obtain the original pose of the photovoltaic support corresponding to the standard point cloud set; The target pose of the photovoltaic bracket to be tested is obtained based on the original pose and the target rigid transformation matrix.

7. The method according to any one of claims 1-6, characterized in that, The lidar is mounted on the paving robot, which is used to perform photovoltaic module paving tasks; After aligning the target point cloud set and the standard point cloud set according to the normal distribution transformation algorithm to obtain the target pose of the photovoltaic support to be detected, the method further includes: The installation parameters of the target photovoltaic module are determined based on the target pose of the photovoltaic support to be tested. The installation parameters include the installation position, installation angle, and installation gap of the target photovoltaic module on the photovoltaic support to be tested. The first control command for the robotic arm is generated based on the paving parameters to control the robotic arm to complete the paving task for the target photovoltaic module according to the paving parameters.

8. A photovoltaic support bracket identification and positioning device based on point cloud matching, characterized in that, include: Scanning unit, processing unit, wherein: The scanning unit is used to scan the photovoltaic support to be inspected using a lidar to obtain an initial point cloud set; The processing unit is configured to perform preprocessing operations on the initial point cloud set to obtain a target point cloud set; retrieve a pre-stored photovoltaic support template library, which stores multiple support point cloud sets corresponding to various types of photovoltaic supports; obtain the plane normal vector and principal direction vector of the ground plane, as well as a preset reference point located on the ground plane; establish a three-dimensional coordinate system based on the cross product relationship, with the preset reference point as the origin, the direction of the plane normal vector as the positive Z-axis, and the direction of the principal direction vector as the positive X-axis; perform plane fitting on the target point cloud set based on a preset K-nearest neighbor algorithm to obtain multiple local planes, and determine multiple local direction vectors corresponding to the multiple local planes using a PCA principal component analysis algorithm; determine multiple intersection points based on the multiple local direction vectors, a preset three-dimensional line fitting algorithm, and the three-dimensional coordinate system. The intersection point represents the point cloud corresponding to the connection position of the photovoltaic bracket in the photovoltaic bracket to be detected. The intersection point is the most distinguishable structural feature of the bracket, including the intersection of the horizontal beam and the vertical beam, which is used to determine the bracket type. Based on the three-dimensional coordinate system, the area around the intersection point at a second preset distance is taken as the first feature area, resulting in multiple first feature areas. The points corresponding to the multiple first feature areas are determined according to the target point cloud set, resulting in a third point cloud set. The similarity of multiple FPFH histogram features between the third point cloud set and the multiple bracket point cloud sets is calculated. The standard point cloud set with the highest similarity is selected from the multiple bracket point cloud sets. The photovoltaic bracket type corresponding to the standard point cloud set matches the photovoltaic bracket to be detected. The target point cloud set and the standard point cloud set are aligned according to the normal distribution transformation algorithm to obtain the target pose of the photovoltaic bracket to be detected.

9. An electronic device, characterized in that, include: Processor, memory, communication interface, and one or more programs; The one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing the steps of the method as described in any one of claims 1-7.

Images