Photovoltaic panel pose estimation method and device and electronic equipment

Abstract

The invention discloses a photovoltaic panel pose estimation method and device and electronic equipment, and the method comprises the steps: carrying out the scanning of a to-be-detected photovoltaic panel through a laser radar, and obtaining an original point cloud set; performing data preprocessing on the original point cloud set to obtain a target point cloud set; determining an initial region point cloud set according to the target point cloud set; determining a photovoltaic panel point cloud set according to the initial region point cloud set, a conditional Euclidean algorithm and the target point cloud set; extracting a plane point cloud frame according to the photovoltaic panel point cloud set; taking the preset position of the to-be-detected photovoltaic panel as a coordinate origin, and establishing a photovoltaic panel coordinate system; and performing photovoltaic panel pose estimation based on the photovoltaic panel coordinate system and the plane point cloud frame to obtain a target pose estimation result. By adopting the embodiment of the invention, the pose estimation precision of the photovoltaic panel is improved.

Description

Technical Field

[0001] This application relates to the field of pose estimation technology, and in particular to a photovoltaic panel pose estimation method, apparatus and electronic device. Background Technology

[0002] The photovoltaic panel installation robot needs to use visual positioning to guide the robotic arm to complete the automated installation. The core challenge lies in accurately calculating the pose of the photovoltaic panels and performing the installation work. The robot needs to identify the pose of the already installed photovoltaic panels in order to calculate the pose of the photovoltaic panels to be installed.

[0003] Currently, traditional methods mainly rely on cameras and image recognition, which have significant limitations in practical applications. They are easily affected by outdoor light, have poor recognition robustness, and result in low pose estimation accuracy. Therefore, how to improve the pose estimation accuracy of photovoltaic panels has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides a photovoltaic panel pose estimation method, apparatus, and electronic device, which improves the pose estimation accuracy of photovoltaic panels.

[0005] In a first aspect, embodiments of this application provide a photovoltaic panel pose estimation method, including:

[0006] The photovoltaic panel under test is scanned using lidar to obtain the original point cloud set;

[0007] The original point cloud set is preprocessed to obtain the target point cloud set;

[0008] Determine the initial region point cloud set based on the target point cloud set;

[0009] The photovoltaic panel point cloud set is determined based on the initial region point cloud set, the conditional Euclidean algorithm, and the target point cloud set.

[0010] Extract the planar point cloud outline based on the photovoltaic panel point cloud set;

[0011] A photovoltaic panel coordinate system is established by taking the preset position of the photovoltaic panel under test as the origin; the photovoltaic panel coordinate system is a three-dimensional coordinate system.

[0012] The photovoltaic panel pose is estimated based on the photovoltaic panel coordinate system and the planar point cloud bounding box to obtain the target pose estimation result.

[0013] Secondly, embodiments of this application provide a photovoltaic panel pose estimation device, comprising: a scanning unit, a data processing unit, and a pose estimation unit, wherein:

[0014] The scanning unit is used to scan the photovoltaic panel under test using a lidar to obtain the original point cloud set;

[0015] The data processing unit is used to preprocess the original point cloud set to obtain a target point cloud set; determine an initial region point cloud set based on the target point cloud set; determine a photovoltaic panel point cloud set based on the initial region point cloud set, the conditional Euclidean algorithm, and the target point cloud set; extract the planar point cloud bounding box based on the photovoltaic panel point cloud set; and establish a photovoltaic panel coordinate system by using a preset position of the photovoltaic panel to be tested as the origin; the photovoltaic panel coordinate system is a three-dimensional coordinate system.

[0016] The pose estimation unit is used to estimate the pose of the photovoltaic panel based on the photovoltaic panel coordinate system and the planar point cloud bounding box, and obtain the target pose estimation result.

[0017] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing the steps in the first aspect of embodiments of this application.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the first aspect of embodiments of this application.

[0019] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of embodiments of this application. The computer program product may be a software installation package.

[0020] Implementing this application will have the following beneficial effects:

[0021] As can be seen, the photovoltaic panel pose estimation method described in this application scans the photovoltaic panel under test with a lidar to obtain an original point cloud set. Then, the original point cloud set is preprocessed to obtain a target point cloud set, which can remove irrelevant points (such as ground and debris) and noise points, reducing the sources of error in subsequent feature extraction and providing a clean data source for high-precision pose estimation. Next, the planar point cloud bounding box is extracted and a coordinate system is established with a preset position as the origin. The pose features of the photovoltaic panel are transformed into quantifiable coordinate parameters, which are directly associated with the rotation and translation components of the pose matrix, reducing the cumulative error of indirect calculation, thereby improving the pose estimation accuracy of the photovoltaic panel. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0023] Figure 1 This is a schematic diagram of the structure of a photovoltaic panel installation robot provided in an embodiment of this application;

[0024] Figure 2 This is a scenario application diagram of a photovoltaic panel installation robot provided in an embodiment of this application;

[0025] Figure 3 This is a flowchart of a photovoltaic panel pose estimation method provided in an embodiment of this application;

[0026] Figure 4 This is a flowchart of a method for determining a target point cloud set provided in an embodiment of this application;

[0027] Figure 5 This is a flowchart of a method for determining an initial region point cloud set provided in an embodiment of this application;

[0028] Figure 6 This is a schematic diagram of a photovoltaic panel coordinate system provided in an embodiment of this application;

[0029] Figure 7 This is a schematic diagram of the structure of a photovoltaic panel pose estimation device provided in an embodiment of this application;

[0030] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0032] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0033] It should be understood that the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, "multiple" refers to two or more.

[0034] In the embodiments of this application, "at least one item" or its similar expression refers to any combination of these items, including any combination of a single item or a plurality of items. "One or more" means one or more, while "multiple" means two or more. For example, "at least one item" of a, b, or c can represent the following seven cases: a, b, c; a and b; a and c; b and c; a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0035] In this application, the term "connection" refers to various connection methods, such as direct connection or indirect connection, to achieve communication between devices. This application does not impose any limitations on this.

[0036] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0037] The electronic devices described in this application embodiment may include smartphones (such as Android phones, iOS phones, Windows Phones, etc.), tablet computers, PDAs, laptops, video matrices, monitoring platforms, mobile internet devices (MIDs), or wearable devices, etc. The above are merely examples and not exhaustive, and include but are not limited to the above devices.

[0038] Of course, the aforementioned electronic devices may also include photovoltaic panel installation robots.

[0039] The following describes the relevant content, concepts, meanings, technical issues, technical solutions, and beneficial effects involved in the embodiments of this application.

[0040] First, let me explain some of the technical terms used in this application:

[0041] Photovoltaic panel pose estimation refers to the technique of acquiring three-dimensional spatial information of a photovoltaic panel through sensors (such as LiDAR, cameras, etc.) and calculating its position (translation parameters) and attitude (rotation parameters) in a world coordinate system (or a specified coordinate system). Its core purpose is to determine the spatial placement of the photovoltaic panel (such as tilt angle, orientation, and coordinates). It is commonly used in scenarios such as photovoltaic panel installation inspection, operation and maintenance monitoring (such as dust accumulation / shading assessment), and tracking control (such as adjusting the attitude according to the sun's angle to improve power generation efficiency).

[0042] LiDAR: A sensor that detects the three-dimensional information of a target by emitting a laser beam and receiving the reflected signal.

[0043] A point cloud is a dataset composed of a large number of discrete 3D points. Each point contains 3D coordinates, and some points may also include information such as color and reflectivity. These points collectively form the surface contour of a target object (such as a photovoltaic panel). For example, after a LiDAR scanner scans a photovoltaic panel, it generates millions of points. The distribution of these points corresponds one-to-one with the shape and position of the photovoltaic panel, forming a "digital twin" point cloud of the photovoltaic panel. Point clouds are the fundamental data format for 3D reconstruction and pose estimation.

[0044] Conditional Euclidean algorithm: a point cloud-based clustering and segmentation algorithm used to divide point clouds into multiple independent subsets (clusters) based on "spatial proximity" and "additional conditions".

[0045] Pass-through filtering is a simple point cloud preprocessing algorithm used to quickly remove invalid data (such as background noise and points outside the region of interest) from point clouds.

[0046] Voxel filtering is a point cloud downsampling algorithm used to reduce the number of point clouds (simplify the data) while preserving the overall structural features of the object.

[0047] Plane fitting is the process of using algorithms to fit a mathematical model (plane equation) from a point cloud that best approximates a target plane (such as the surface of a photovoltaic panel). The surface of a photovoltaic panel can be approximated as an ideal plane. Commonly used plane fitting methods include the least squares method (minimizing the sum of squared distances from points to the plane) and the RANSAC algorithm (random sampling consistency, strong noise resistance, suitable for noisy point clouds). The results of plane fitting can be directly used to calculate the attitude of the photovoltaic panel (the normal vector determines the tilt angle and orientation).

[0048] RANSAC (Random Sample Consensus) is an iterative algorithm for robustly estimating mathematical model parameters from data containing noise and outliers.

[0049] Latitude and Longitude Scanning Algorithm: This is a pose refinement algorithm for regular planes (such as photovoltaic panels). It extracts the edges or feature points of the plane by simulating scan lines in the "longitude" and "latitude" directions, and optimizes the pose parameters.

[0050] Rotation matrix: A 3×3 orthogonal matrix used to describe the attitude transformation of an object in three-dimensional space (e.g., rotation about the X, Y, and Z axes). In photovoltaic panel pose estimation, the rotation matrix is ​​the core representation of attitude parameters, which can be calculated from plane normal vectors or feature points, and is used to describe the rotation state of the photovoltaic panel relative to the world coordinate system (e.g., pitch angle, yaw angle, roll angle).

[0051] The pose transformation matrix is ​​a 4×4 matrix used to simultaneously describe the position (translation) and orientation (rotation) of an object in three-dimensional space. Using the pose transformation matrix, the coordinates of any point on the photovoltaic panel in its own coordinate system can be directly transformed to the world coordinate system, fully reflecting its spatial pose. It is one of the final output forms of the pose estimation results.

[0052] Please see Figure 1 , Figure 1 This is a structural schematic diagram of a photovoltaic panel installation robot provided in an embodiment of this application. It can be seen that the photovoltaic panel installation robot may include: a control module, an installation module, a movement module, a lidar, etc., which are not limited herein; wherein:

[0053] The control module is the "brain" of the robot, responsible for coordinating the operation of all modules. The control module can receive environmental data detected by sensors such as lidar (such as the location of photovoltaic brackets and terrain information), and calculate the optimal operation strategy (such as the movement route and the trajectory of the robotic arm) by combining it with the preset paving path and rules.

[0054] The installation module (e.g., a robotic arm) is the core functional module that performs the installation of photovoltaic panels. It is used to operate the photovoltaic panels. For example, the installation module can perform operations such as grabbing, moving, placing, and fixing the photovoltaic panels. No specific limitations are made here.

[0055] The mobility module is the robot's "legs," responsible for supporting the robot as a whole and enabling it to move within the space.

[0056] LiDAR is one of the "eyes" of a robot, mainly providing three-dimensional environmental perception capabilities. Its function is to scan the three-dimensional structure of photovoltaic panels, extract the point cloud information of the photovoltaic panels, and send the point cloud information to the control module. The control module calculates the spatial pose of the photovoltaic panels, thereby providing coordinate reference for the precise alignment of the installation module.

[0057] It should be noted that, in practice, lidar can be a detachable device.

[0058] Please see Figure 2 , Figure 2 This is a scenario application diagram of a photovoltaic panel installation robot provided in the embodiments of this application. It can be seen that the photovoltaic bracket, as the installation foundation of the photovoltaic panel, provides support and positioning for the photovoltaic panel, ensuring its stable placement, and is an important structural component of the photovoltaic system.

[0059] LiDAR can be installed at the front end of the robotic arm of a photovoltaic panel installation robot. By emitting and receiving lasers, it scans the surrounding environment and obtains three-dimensional spatial information of the photovoltaic support, work area, and photovoltaic panels. This information is used for environmental perception, obstacle detection, and assisting in the precise alignment and installation of photovoltaic panels.

[0060] The photovoltaic panel installation robot integrates multiple functional modules. It moves on the site using a mobile module (such as a tracked chassis), while the control module coordinates the operation. The installation module (such as a robotic arm) completes the grabbing, posture adjustment, and installation of photovoltaic panels, making it the core component for achieving automated installation.

[0061] Photovoltaic panels are the objects to be installed. Photovoltaic panel installation robots install photovoltaic panels onto photovoltaic brackets, which are then used to convert solar energy into electrical energy. They are the core power generation components of a photovoltaic system.

[0062] Figure 2 The overall presentation showcases the operation mode of photovoltaic panel installation robots using LiDAR to perceive the environment and automatically install photovoltaic panels on photovoltaic brackets. This reflects the trend of automation and intelligence in photovoltaic construction, which can improve installation efficiency and accuracy, and reduce labor costs and operational risks.

[0063] Please see Figure 3 , Figure 3 This is a flowchart of a photovoltaic panel pose estimation method provided in an embodiment of this application; the method includes, but is not limited to, the following steps:

[0064] S301. The photovoltaic panel under test is scanned by lidar to obtain the original point cloud set.

[0065] In this application embodiment, the lidar may include one of the following: mechanical lidar, solid-state lidar, triangulation lidar, etc., and is not limited thereto.

[0066] In a specific embodiment, the scanning parameters of the lidar can be set first. Specifically, the scanning parameters can be set through the lidar's software or device control panel, including the scanning frequency (i.e., the number of laser pulses emitted per unit time; a higher scanning frequency can acquire denser point cloud data, but may increase the amount of data processing), the scanning angle range (to ensure that the entire surface of the photovoltaic panel can be covered), and the laser emission power (adjusted according to the ambient light and scanning distance to ensure that the reflected signal can be effectively received), etc.

[0067] After setting the scanning parameters, the LiDAR can be activated to scan. The LiDAR will periodically emit laser pulses into space according to the set parameters to scan the photovoltaic panel under test. For example, if it is a fixedly installed LiDAR, the photovoltaic panel can be moved within the LiDAR's scanning range via a mechanical transmission device (such as a conveyor belt or slide rail) to achieve scanning of the entire photovoltaic panel surface. If the LiDAR is installed at the end of the robotic arm of a photovoltaic panel installation robot, the scanning range can be controlled by controlling the movement trajectory of the robotic arm to cover all parts of the photovoltaic panel. During the scanning process, the laser light reflected back from the photovoltaic panel surface and is captured by the LiDAR's receiver. As the LiDAR continuously emits and receives laser pulses, a large number of three-dimensional coordinate points are acquired and stored in the data acquisition device. These discrete three-dimensional points together constitute the original point cloud of the photovoltaic panel.

[0068] S302. Perform data preprocessing on the original point cloud set to obtain the target point cloud set.

[0069] In the embodiments of this application, data preprocessing may include at least one of the following: denoising processing (e.g., statistical filtering), data compression processing (e.g., voxel filtering), point cloud smoothing processing, point cloud segmentation processing (e.g., pass-through filtering), etc., without limitation.

[0070] In a specific embodiment, the original point cloud set is preprocessed to obtain the target point cloud set. For example, the original point cloud set can be denoised to remove abnormal points caused by environmental interference (such as light and dust reflection) and equipment errors, while retaining the effective points that truly reflect the photovoltaic panel and the target scene.

[0071] Optional, please refer to Figure 4 , Figure 4 This is a flowchart of a method for determining a target point cloud set according to an embodiment of this application. The scanning method of the lidar can be a non-repeating scanning method. Step S302, the data preprocessing of the original point cloud set to obtain the target point cloud set, may include... Figure 4 The steps shown are as follows:

[0072] S21. Obtain the target size information of the photovoltaic panel to be tested;

[0073] S22. Determine the estimated coordinate range based on the target size information;

[0074] S23. Based on the estimated coordinate range, perform a pass-through filter on the original point cloud set to obtain a first point cloud set;

[0075] S24. The first point cloud set is downsampled using a voxel filtering method to obtain the second point cloud set;

[0076] S25. Determine the average distance between each point in the second point cloud set and its surrounding neighboring points to obtain multiple average distances;

[0077] S26. Determine the average distance among the plurality of average distances that is greater than the first preset distance, and obtain at least one average distance;

[0078] S27. Determine the point in the second point cloud set corresponding to the at least one average distance to obtain at least one isolated noise point;

[0079] S28. Remove the at least one isolated noise point from the second point cloud set to obtain the target point cloud set.

[0080] In this embodiment of the application, the first preset distance can be preset in advance or be a default value.

[0081] In a specific embodiment, the target size information of the photovoltaic panel to be tested can be obtained first. Specifically, the model of the photovoltaic panel to be tested can be obtained, and the corresponding target size information can be obtained from a preset database based on the photovoltaic panel model. Then, the estimated coordinate range can be determined based on the target size information. Specifically, the approximate center position of the photovoltaic panel in three-dimensional space can be determined first using the radar coordinate system as a reference (which can be obtained through sensor coarse positioning, historical data, or preset installation area range) as the reference origin of the coordinate range. The target size information (length, width, and thickness) of the photovoltaic panel is converted into a range on the three-dimensional coordinate axis to obtain the estimated coordinate range. For example, assuming the length of the photovoltaic panel is 20cm, and the coordinate axis corresponding to the length is the X-axis, its coordinate range on the X-axis can be set to -10cm to +10cm. Of course, to avoid the point cloud exceeding the range due to the tilt of the photovoltaic panel or the measurement deviation of the sensor, a certain redundancy (such as 10%-20%) can be added to the original size of each axis to ensure that the estimated coordinate range can completely cover the actual point cloud.

[0082] Next, a pass-through filter can be applied to the original point cloud set based on the estimated coordinate range to obtain the first point cloud set. For example, assuming the estimated coordinate range can be [x1... min x1 max ,y1 min ,y1 max ,z1 min ,z1 max ], x1 min This represents the minimum value on the X-axis within the estimated coordinate range, x1 max This represents the maximum value on the X-axis within the estimated coordinate range, y1 min This represents the minimum value on the Y-axis within the estimated coordinate range, y1. max z1 represents the maximum value on the Y-axis within the estimated coordinate range. minThis represents the minimum value on the Z-axis within the estimated coordinate range, z1. max This represents the maximum value on the Z-axis within the estimated coordinate range. The pass-through filtering algorithm is an algorithm for filtering point clouds based on the coordinate axis range. For each point (x, y, z) in the original point cloud set, it is determined whether its coordinates are within the estimated coordinate range. Points within the estimated coordinate range are retained, and points outside the estimated coordinate range are removed. Finally, the first point cloud set containing only points within the estimated coordinate range is obtained (i.e., the point cloud of the photovoltaic panel and a small number of nearby background points).

[0083] Furthermore, a voxel filtering method can be used to downsample the first point cloud set to obtain the second point cloud set. Specifically, the point cloud space corresponding to the first point cloud set can be divided into regular voxel grids (e.g., cubes with a side length of 5cm). Only a small number of representative points (such as the center point and the average point) are retained in each voxel grid to replace all the original points in that voxel grid, thereby reducing the number of point clouds while preserving the overall shape features, thus obtaining the second point cloud set.

[0084] Then, the average distance between each point in the second point cloud and its surrounding neighboring points can be determined, resulting in multiple average distances. For example, for point p, its a nearest neighbors (e.g., if a equals 10, that is, find 10 nearest points) or radius neighborhood (e.g., if the radius is 5cm, find points within a 5cm radius), calculate the distance from point p to these neighbors, and take the average as the average distance of the point. Repeat this process multiple times to collect the average distance of each point in the second point cloud, resulting in multiple average distances.

[0085] Next, the average distances that are greater than the first preset distance among these multiple average distances can be filtered out to obtain at least one average distance; wherein, the first preset distance can be determined by statistical methods (e.g., mean plus standard deviation) instead of being preset or defaulted in advance; then, the points corresponding to at least one average distance in the second point cloud set can be found to obtain at least one isolated noise point; finally, these at least one isolated noise points can be removed from the second point cloud set to obtain the target point cloud set.

[0086] In this way, by defining the estimated coordinate range through the target size information, the background point cloud outside the photovoltaic panel (such as the ground, supports, and distant objects) is directly filtered out, reducing irrelevant data interference and avoiding redundant points in subsequent algorithms (such as ground point clouds that may account for more than 70% of the original point cloud in a photovoltaic power station scenario). This allows the calculation to focus on the photovoltaic panel itself. In addition, through pass-through filtering, the point cloud is first "coarsely screened" (filtering a large area of ​​background), then "refined" (compressing the number of point clouds) through voxel filtering, and finally "purified" (removing outliers) through isolated point deletion. This three-layer filtering can exponentially reduce the amount of point cloud data, improving computational efficiency. For example, if the original point cloud has 1 million points, after pass-through filtering, 100,000 points remain (filtering the background), after voxel filtering, 10,000 points remain (downsampling), and after isolated point deletion, 9,500 points remain (denoising). The computational workload of subsequent algorithms can be reduced by more than 99%.

[0087] S303. Determine the initial region point cloud set based on the target point cloud set.

[0088] In this embodiment of the application, an initial region can be determined first, and then points within the initial region can be extracted from the target point cloud set to obtain the initial region point cloud set.

[0089] Optionally, the lidar is mounted on the photovoltaic panel installation robot; step S303, determining the initial region point cloud set based on the target point cloud set, may include the following steps:

[0090] A1. Obtain the center position of the lidar, the first parking position of the photovoltaic panel laying robot, and the relative position of the photovoltaic panel laying robot and the photovoltaic panel to be tested;

[0091] A2. Establish a radar coordinate system with the center position as the origin; the radar coordinate system is a three-dimensional coordinate system.

[0092] A3. Based on the radar coordinate system, the area around the center position at a second preset distance is taken as the initial area;

[0093] A4. Determine the points corresponding to the initial region based on the target point cloud set to obtain the third point cloud set;

[0094] A5. Use a preset plane fitting algorithm to perform plane fitting on the third point cloud set to obtain the plane normal;

[0095] A6. Determine the initial region point cloud set based on the plane normal, the radar coordinate system, and the third point cloud set.

[0096] In this embodiment, both the second preset distance and the preset plane fitting algorithm can be preset in advance or defaulted.

[0097] In a specific embodiment, the center position of the lidar, the first parking position of the photovoltaic panel laying robot, and the relative position of the photovoltaic panel laying robot and the photovoltaic panel to be tested can be obtained first. Specifically, the photovoltaic panel laying robot can be equipped with a positioning module, which can determine the first parking position of the photovoltaic panel laying robot. Since the installation position of the lidar on the photovoltaic panel laying robot is fixed and known (through mechanical design drawings or factory calibration), if the coordinate system of the photovoltaic panel laying robot body (e.g., the center of the chassis is the origin) has been determined, the center position of the lidar can be expressed as a fixed offset in that coordinate system. The measurement, i.e., the center position, is then used. Then, the ICP algorithm (Iterative Closest Point) can be used to register the photovoltaic panel point cloud (i.e., the target point cloud set) with the "standard photovoltaic panel model point cloud" to calculate the pose transformation between the two. The transformation result is the relative position between the photovoltaic panel laying robot and the photovoltaic panel to be tested (for example, the center of the photovoltaic panel is 0.5m in front of the robot and 1.2m above it). Alternatively, since the parking point of the photovoltaic panel laying robot needs to meet the construction operation requirements, and the movement of the photovoltaic panel laying robot is manually operated, the relative position between the photovoltaic panel laying robot and the photovoltaic panel to be tested can be obtained by manual measurement by the staff.

[0098] Next, establish a radar coordinate system with the center position as the origin. For example, the radar coordinate system is a right-handed Cartesian coordinate system, and the direction of the axes needs to be unified (industry-standard rules can be adopted). The origin can be the physical center of the lidar (i.e., the center position, usually the geometric center of the laser transmitting / receiving module); the X-axis can be the "front" direction of the lidar (by default, it points to the main scanning direction of the lidar, such as the forward direction of the photovoltaic panel laying robot); the Y-axis can be the "left" direction of the lidar (perpendicular to the X-axis, on the horizontal plane); the Z-axis can be the "above" direction of the lidar (perpendicular to the XY plane, following the right-hand rule: the right thumb points to the X-axis, the index finger points to the Y-axis, and the middle finger points to the Z-axis).

[0099] Then, based on the radar coordinate system, the area around the center position at a second preset distance can be used as the initial area. For example, a circular area with the center position as the center point and the second preset distance as the radius can be used as the initial area. Alternatively, the relative position mentioned above can be used as a priori condition to set a center point for the area. This center point can be a reference point determined by the staff based on the relative position. The function of the center point is to quickly select the local area "most likely to contain the photovoltaic panel point cloud" in the global point cloud scanned by the radar. For example, assuming that the relative position of the robot and the photovoltaic panel is fixed (such as the photovoltaic panel is always within 2 meters in front of the robot and 0.5 meters to the left), the center point can be set to (2, 0.5, 0) (x = 2m, indicating 2 meters in front of the lidar, y = 0.5m, indicating 0.5 meters to the left of the radar). Combined with the area radius (i.e., the second preset distance, for example, 0.5 meters), a circular area can be selected. The point cloud in this area "probably contains the photovoltaic panel". Subsequently, only the point cloud of this small area needs to be processed, without calculating the entire point cloud, which is more efficient.

[0100] Next, the points corresponding to the initial region can be determined based on the target point cloud set to obtain the third point cloud set. Specifically, points within the initial region can be extracted from the target point cloud set to obtain the third point cloud set. Further, a preset plane fitting algorithm can be used to perform plane fitting on the third point cloud set to obtain the plane normal. For example, the preset plane fitting algorithm can be the RANSAC algorithm. By performing plane fitting on the third point cloud set using this algorithm, a first plane can be obtained. Then, the plane normal of the first plane can be calculated. Finally, the initial region point cloud set can be determined based on the plane normal, the radar coordinate system, and the third point cloud set.

[0101] Thus, by utilizing the prior condition that "the relative positions of the robot and the photovoltaic panel are fixed," the selection of the initial area becomes more targeted. Even if there are installation errors in the photovoltaic panel or changes in the scene (such as uneven ground or slight movement of the support), the effective point cloud can still be defined. For example, in outdoor photovoltaic power station scenarios, there are many interferences from light and terrain. The prior condition can significantly reduce the "probability of misjudgment" of the algorithm (such as mistaking the ground for the photovoltaic panel).

[0102] Optional, please refer to Figure 5 , Figure 5 This is a flowchart of a method for determining an initial region point cloud set according to an embodiment of this application. Step A6, determining the initial region point cloud set based on the plane normal, the radar coordinate system, and the third point cloud set, may include... Figure 5 The steps shown are as follows:

[0103] B1. Determine the angle between the plane normal and the unit vector of the Z-axis of the radar coordinate system to obtain the target angle;

[0104] B2. Determine the target included angle threshold based on the relative position;

[0105] B3. When the target angle is less than the target angle threshold, the third point cloud set is determined as the initial region point cloud set;

[0106] B4. When the target angle is greater than or equal to the target angle threshold, control the photovoltaic panel laying robot to move from the first parking position to the second parking position.

[0107] In this embodiment, the angle between the plane normal and the unit vector of the Z-axis of the radar coordinate system can be determined to obtain the target angle. Specifically, since the Z-axis direction of the radar coordinate system is fixed (e.g., vertically upward), the unit vector of the Z-axis can be denoted as (0,0,1). The angle between the plane normal and the unit vector of the Z-axis is calculated as follows:

[0108]

[0109] Here, arccos() represents the inverse cosine function, takes the absolute value of the dot product of two vectors as input, and outputs the angle between them in 0 to π / 2 radians (corresponding to 0 to 90 degrees); Δθ sr Indicates the included angle of the target. This represents the unit normal vector corresponding to the plane normal. This represents the unit vector along the Z-axis. Then, the target angle threshold can be determined based on the relative position. Specifically, the "ideal angle" of the photovoltaic panel installation can be obtained first. For example, if the photovoltaic panel is installed on a flat roof, the tilt angle is close to 0 degrees (horizontal plane), and the angle between the normal and the Z-axis is close to 0 degrees. If the photovoltaic panel is installed on a sloping roof, the tilt angle equals the roof slope (e.g., 30 degrees), and the angle between the normal and the Z-axis = 90 degrees - 30 degrees = 60 degrees. Therefore, the "ideal angle between the normal and the Z-axis" = 90 degrees - photovoltaic panel installation tilt angle. Then, the target installation scene of the photovoltaic panel to be tested can be obtained. The reference photovoltaic panel installation tilt angle can be determined based on the target installation scene. For example, a pre-stored mapping relationship between preset installation scenes and photovoltaic panel installation tilt angles can be used to determine the reference photovoltaic panel installation tilt angle corresponding to the target installation scene. However, in actual installation, errors may occur, such as robot parking errors, photovoltaic panel installation errors, point cloud fitting errors, etc. These are not specifically limited here. Therefore, it is necessary to adjust the reference photovoltaic panel installation tilt angle according to the aforementioned relative position to obtain the target photovoltaic panel installation tilt angle. For example, the ideal relative position corresponding to the target installation scene can be obtained, and the target error between the aforementioned relative position and the ideal relative position can be determined. Based on the target error, a target error factor can be determined. This can be achieved by pre-storing a mapping relationship between preset errors and error factors, and determining the target error factor corresponding to the target error based on this mapping relationship. The target error factor can take values ​​in the range of [-0.25, 0.25]. The reference photovoltaic panel installation tilt angle is then adjusted according to the target error factor, as follows:

[0110] Target photovoltaic panel installation tilt angle = Reference photovoltaic panel installation tilt angle * (1 + target error factor);

[0111] Based on the above formula, the target photovoltaic panel installation tilt angle (i.e., the target included angle threshold) can be obtained; when the target included angle is less than the target included angle threshold, the third point cloud set can be determined as the initial region point cloud set.

[0112] If the target angle is greater than or equal to the target angle threshold, it indicates that the selected initial area is not good and there may be non-photovoltaic panel point cloud planes (such as the ground). The photovoltaic panel laying robot can be controlled to move from the first parking position to the second parking position. Specifically, based on the difference between the target angle and the target angle threshold (e.g., a difference of 5 degrees), combined with the position offset of the photovoltaic panel point cloud in the radar coordinate system, the distance compensation value (e.g., moving forward / backward by 0.5m) and the direction compensation value (e.g., fine adjustment to the left / right by 0.2m) that the photovoltaic panel laying robot needs to move are calculated. Then, steps S301-S303 are re-executed to make the target angle less than the target angle threshold.

[0113] Thus, when the target angle is less than the target angle threshold, the third point cloud set is confirmed as the initial region point cloud, ensuring that subsequent processing focuses on the real photovoltaic panel point cloud and avoids interference; if the target angle is greater than or equal to the target angle threshold, the photovoltaic panel installation robot can be moved to the second parking position, and a qualified point cloud can be re-acquired by adjusting the acquisition perspective, thus solving the problem of invalid point clouds caused by inaccurate parking.

[0114] S304. Determine the photovoltaic panel point cloud set based on the initial region point cloud set, the conditional Euclidean algorithm, and the target point cloud set.

[0115] In this embodiment of the application, the conditional Euclidean algorithm can be used to continuously expand outward from the target point cloud set based on the initial regional point cloud set until all points on the photovoltaic panel plane are found, thus obtaining the segmentation result of the photovoltaic panel plane, i.e., the photovoltaic panel point cloud set.

[0116] Optionally, the target point cloud set includes n points; the initial region point cloud set includes m points, where n and m are both positive integers, and m is less than or equal to n; determining the photovoltaic panel point cloud set based on the initial region point cloud set, the conditional Euclidean algorithm, and the target point cloud set may include the following steps:

[0117] S41. Initialize the initial algorithm parameter set of the conditional Euclidean algorithm;

[0118] S42. Determine the normal vector of each point in the target point cloud set to obtain the target normal vector data;

[0119] S43. Initialize the target list and the target plane container; the target list initialization includes n identifiers, each identifier corresponding to one of the n points, and the target list is used to record the access status of each point; the target plane container is initialized to empty, and the target plane container is used to store the clustering results;

[0120] S44. Traverse each point in the initial region point cloud set, and cluster each point based on the initial algorithm parameter set, the target normal vector data, the target list, and the target plane container to obtain i clustering results; i is a positive integer less than m;

[0121] S45. Determine the fitting plane corresponding to each of the i clustering results to obtain i fitting planes;

[0122] S46. Merge the i fitting planes to obtain the photovoltaic panel fitting plane;

[0123] S47. Determine the photovoltaic panel point cloud set based on the photovoltaic panel fitting plane.

[0124] In this embodiment, the initial algorithm parameter set of the conditional Euclidean algorithm can be initialized first. Specifically, the spatial distance threshold can be set according to the point cloud density of the photovoltaic panel, usually 1.5-2 times the voxel filtering size (e.g., if the voxel size is 0.02m, then the spatial distance threshold can be set to 0.03-0.04m) to ensure that adjacent points on the same photovoltaic panel surface are grouped into one class. Then, the normal vector angle threshold can be set based on the planar characteristics of the photovoltaic panel, for example, set to 10 degrees-20 degrees to filter points with large differences in normal vectors (e.g., points between the photovoltaic panel and the ground) and retain points within the plane. Then, the minimum number of cluster points can be set to 10%-20% of the expected number of points (e.g., if the photovoltaic panel point cloud has 1000 points, then the minimum number of cluster points can be set to 100-200) to eliminate small clusters formed by noise. The initial algorithm parameter set is determined based on the spatial distance threshold, the normal vector angle threshold, and the minimum number of cluster points.

[0125] Then, the normal vector of each point in the target point cloud can be determined to obtain the target normal vector data. Specifically, for each point, we can first find all neighboring points within a certain range (such as within a fixed radius or a fixed number of nearest neighbors), analyze the spatial distribution pattern of these neighboring points, and use a plane fitting algorithm (such as the least squares method) to fit a plane that best fits the distribution of these points, thus obtaining the best fitting plane. We can then calculate the direction vector perpendicular to this best fitting plane, which is the normal vector of the plane (the normal vector of each point is determined by the plane fitted by its neighboring points). In this way, we can obtain the normal vectors of all points. Then, we can uniformly adjust the normal vectors of all points so that their directions are all pointing towards the direction of the sensor (such as LiDAR), ensuring that the normal vector directions are consistent. Finally, we obtain the normal vector corresponding to each point in the target point cloud, which is the target normal vector data.

[0126] Next, the target list and target plane container can be initialized. Then, each point in the initial region point cloud set can be traversed, and each point can be clustered based on the initial algorithm parameter set, target normal vector data, target list, and target plane container to obtain i clustering results. Next, a preset plane fitting algorithm can be used to fit each of the i clustering results to obtain i fitted planes. Then, the i fitted planes can be merged to obtain the photovoltaic panel fitted plane. Specifically, for any two planes, the first angle between their normal vectors is calculated. If the first angle is greater than or equal to the normal vector angle threshold, they are determined to be different planes and are not merged. If the first angle is less than the normal vector angle threshold, it means that the two planes are "close in direction" (almost parallel, possibly belonging to the same plane). Then, the "close in direction" planes are merged. For two planes that are "close", calculate the vertical distance between them. If the vertical distance is less than the spatial distance threshold, it means that the two planes are "spatially close" (almost coplanar) and can be merged. If the vertical distance is greater than or equal to the spatial distance threshold, they are considered separate planes and are not merged. Repeat this process until all planes are matched, forming multiple "plane groups" (planes in each group have similar directions and are close in distance). Calculate the number of point clouds (cluster points) contained in each plane group and retain the target plane group with the most points (the main point cloud of the photovoltaic panel usually accounts for the highest proportion, and the number of interfering plane points is small). For the selected target plane group, refit the plane with all the points in the target plane group to obtain the photovoltaic panel fitted plane. Finally, all the points contained in the photovoltaic panel fitted plane can be determined to obtain the photovoltaic panel point cloud set.

[0127] Thus, by first fitting a plane (local feature) to each clustering result, and then merging planes with similar orientations and close distances (excluding interference planes), the final "photovoltaic panel fitting plane" takes into account both local details and overall consistency, and its accuracy is much higher than that of directly fitting the original point cloud.

[0128] Optionally, the initial algorithm parameter set includes: spatial distance threshold, normal vector angle threshold, and minimum number of cluster points; traversing each point in the initial region point cloud set and clustering each point based on the initial algorithm parameter set, the target normal vector data, the target list, and the target plane container to obtain i clustering results may include the following steps:

[0129] First, determine the current point being traversed. Then, find the corresponding identifier of the current point in the target list. Based on the identifier, determine whether the current point has been visited. If the identifier indicates that the current point has been visited, it means that the current point has already been traversed, and the current point can be skipped. If the identifier indicates that the current point has not been visited, use the current point as the starting point of the new plane, initialize the first plane container and the first queue of points to be processed, both of which are initially empty. Then, add the new starting point of the plane to the first queue of points to be processed.

[0130] S1. Take out the first point from the head of the first pending point queue, add the first point to the first planar container; set the identifier corresponding to the first point in the target list to be visited.

[0131] S2. Based on the target point cloud set and the target list, find unvisited neighbor points within the spatial distance threshold around the first point to obtain k points; k is a natural number.

[0132] S3. Based on the target normal vector data, determine the k normal vectors corresponding to the k points, and the first normal vector corresponding to the first point;

[0133] S4. Determine the angle between each of the k normal vectors and the first normal vector to obtain k angles;

[0134] S5. Determine the angles among the k included angles that are less than or equal to the normal vector included angle threshold, to obtain h included angles; h is a natural number less than or equal to k;

[0135] S6. Add the h points corresponding to the h included angles in the k points to the first queue of points to be processed, and set the h identifiers corresponding to the h points in the target list to be visited.

[0136] S7. Determine whether the queue of the first point to be processed is empty;

[0137] S8. If the first queue of points to be processed is empty, then determine the reference clustering result based on the first planar container;

[0138] S9. If the first queue of points to be processed is not empty, repeat steps S1-S7 until the first queue of points to be processed is empty, and determine the reference clustering result based on the first planar container.

[0139] In a specific embodiment, since the first queue of points to be processed has a first-in-first-out characteristic, the first point is taken out from the first queue of points to be processed, added to the first planar container, and the identifier corresponding to the first point in the target list is set to visited. Then, based on the target point cloud set and the target list, unvisited neighboring points within the spatial distance threshold around the first point can be found to obtain k points. Specifically, all neighboring points whose distance to the first point is less than the spatial distance threshold can be found first based on the target point cloud set. Then, the identifiers of all neighboring points can be found from the target list, and the unvisited neighboring points can be determined based on these identifiers to obtain k points. Further, the k normal vectors corresponding to these k points and the first normal vector corresponding to the first point can be found from the target normal vector data. Then, the angle between each of the k normal vectors and the first normal vector can be calculated to obtain k angles. Since the method for calculating the angle between two vectors is a conventional technique, it will not be described in detail here.

[0140] Next, we can find h angles out of the k angles that are less than or equal to the normal vector angle threshold; then, we can add the h points corresponding to the h angles out of the k points to the first queue of points to be processed, and set the h identifiers corresponding to the h points in the target list to be visited; then, we can determine whether the first queue of points to be processed is empty; if the first queue of points to be processed is empty, we can determine the reference clustering result based on the first planar container, specifically, we can determine all the points contained in the first planar container, that is, the reference clustering result;

[0141] If the first queue of points to be processed is not empty, repeat steps S1-S7 until the first queue of points to be processed is empty, and determine the reference clustering result based on the first planar container.

[0142] Then, the number of points contained in the reference clustering result can be determined to obtain the reference clustering point count; when the reference clustering point count is greater than or equal to the minimum clustering point count, the reference clustering result can be used as the clustering result corresponding to the starting point of the new plane.

[0143] When the number of reference cluster points is less than the number of minimum cluster points, the reference clustering results can be discarded, and the clustering failure will be determined by setting a new starting point for the new plane.

[0144] Thus, by filtering neighbors using "spatial distance threshold" (ensuring points are close in physical location) and "normal vector angle threshold" (ensuring points belong to the same plane), it is better able to distinguish between "photovoltaic panel surface points" (close in distance and with consistent normal vectors) and "interference points" (such as support points that are close in distance but have large differences in normal vectors) than single distance clustering, reducing false clustering and improving clustering results.

[0145] S305. Extract the planar point cloud border based on the photovoltaic panel point cloud set.

[0146] In this embodiment of the application, four borders of the photovoltaic panel under test can be extracted from the photovoltaic panel point cloud set to obtain the planar point cloud borders.

[0147] Optionally, extracting the planar point cloud bounding box from the photovoltaic panel point cloud set may include the following steps:

[0148] S51. Project the photovoltaic panel point cloud set onto a two-dimensional plane to obtain a two-dimensional point cloud set;

[0149] S52. Determine the target coordinate range information based on the two-dimensional point cloud set;

[0150] S53. The two-dimensional point cloud set is processed according to the target coordinate range information and the latitude and longitude line scanning algorithm to obtain the planar point cloud bounding box.

[0151] In this embodiment, the photovoltaic panel point cloud set is projected onto a two-dimensional plane to obtain a two-dimensional point cloud set; then, the target coordinate range information can be determined based on the two-dimensional point cloud set, as follows:

[0152]

[0153]

[0154] Among them, P plane Represents a two-dimensional point cloud set, (x i ,y i ) represents P plane The coordinates of the i-th point in the array, where i is a positive integer, x min The x-axis represents the minimum value of a point in a two-dimensional point cloud set. max The x-axis represents the maximum value of a point in a 2D point cloud set, and the y-axis represents the maximum value of a point in the set. min This represents the minimum value of a point in a 2D point cloud set on the y-axis. max This represents the maximum value of a point in the 2D point cloud set on the y-axis. These four values ​​determine the target coordinate range. Then, the 2D point cloud set can be processed using the target coordinate range information and a latitude / longitude scanning algorithm to obtain a planar point cloud bounding box. Specifically, the number of scan samples in both directions can be set to N. x and N y Then the meridian scan step size Δx = (x max -x min ) / N x The latitude scan step size is Δy = (y max -y min ) / N y Then, meridian scanning and parallel scanning can be performed to obtain the planar point cloud bounding box, as shown below:

[0155] Meridian scan (X direction):

[0156] The goal of meridian scanning is to find the points with the maximum and minimum Y-axis coordinates within each sub-interval in the X-direction (i.e., the upper and lower boundary points of each sub-interval). First, the 2D point cloud set is sorted in ascending order according to the magnitude of the X-axis coordinates, resulting in the sorted point cloud set: P x =sort x (P plane ), where sort x () represents a function that sorts data in ascending order based on the x-coordinate.

[0157] For each interval index k = 0, 1, 2, ..., N x -1:

[0158] Calculate the current meridian interval

[0159] Find the points that fall within the given interval from the sorted set:

[0160] Find the upper boundary point:

[0161] Find the lower boundary point:

[0162] Where argmax represents the independent variable that maximizes the expression in the subsequent set; argmin represents the independent variable that minimizes the expression in the subsequent set.

[0163] Latitude scan (Y direction):

[0164] The goal of latitude scanning is to find the points with the maximum and minimum X-axis coordinates (i.e., the left and right boundary points of each sub-interval) within each Y-direction sub-interval. First, the 2D point cloud set is sorted in ascending order according to the magnitude of the Y-axis coordinates, resulting in the sorted point cloud set: P y =sort y (P plane ), where sort y () represents a function that sorts data in ascending order based on the y-coordinate.

[0165] For each interval index k = 0, 1, 2, ..., N x -1:

[0166] Calculate the current meridian interval

[0167] Find the points that fall within the given interval from the sorted set:

[0168] Find the left boundary point:

[0169] Find the right boundary point:

[0170] Thus, by scanning along the meridians and parallels, the final planar point cloud bounding box can be obtained: P top (The set of upper boundary point clouds obtained by meridian scanning, i.e., the upper point cloud bounding box), P bottom (The lower boundary point cloud set obtained by meridian scanning, i.e., the lower point cloud bounding box), P left (The set of left boundary point clouds obtained by latitude scanning, i.e., the left point cloud border), P right (The set of right boundary point clouds obtained by latitude scanning, i.e., the right point cloud border).

[0171] In this way, by determining the maximum / minimum values ​​of X and Y coordinates through a two-dimensional point cloud set, the effective range of the photovoltaic panel point cloud is clarified, avoiding the processing of irrelevant point clouds (such as background noise) by the latitude and longitude scanning algorithm, reducing invalid calculations, and improving the accuracy of boundary extraction.

[0172] S306. Establish a photovoltaic panel coordinate system by taking the preset position of the photovoltaic panel to be tested as the origin of the coordinate system; the photovoltaic panel coordinate system is a three-dimensional coordinate system.

[0173] In this embodiment, a photovoltaic panel coordinate system can be established by using a preset position of the photovoltaic panel under test as the origin. For example, please refer to [link to example]. Figure 6 , Figure 6 This is a schematic diagram of a photovoltaic panel coordinate system provided in an embodiment of this application, as shown below. Figure 6 As shown, the photovoltaic panel under test can be installed horizontally. The preset position is the lower left corner of the photovoltaic panel under test. Then, the normal direction of the photovoltaic panel can be taken as the Z-axis direction, the direction of the lower edge of the photovoltaic panel under test can be taken as the X-axis direction, and the direction of the left edge can be taken as the Y-axis direction.

[0174] S307. Based on the photovoltaic panel coordinate system and the planar point cloud bounding box, the photovoltaic panel pose is estimated to obtain the target pose estimation result.

[0175] In this embodiment, the planar point cloud bounding box can be substituted into the photovoltaic panel coordinate system to perform photovoltaic panel pose estimation and obtain the target pose estimation result.

[0176] Optionally, the planar point cloud bounding box includes: a lower point cloud bounding box and a left point cloud bounding box; the preset position is the intersection of the lower point cloud bounding box and the left point cloud bounding box; the step of estimating the photovoltaic panel pose based on the photovoltaic panel coordinate system and the planar point cloud bounding box to obtain the target pose estimation result may include the following steps:

[0177] S71. Determine the lower border line vector corresponding to the lower point cloud border, and the left border line vector corresponding to the left point cloud border.

[0178] S72. Determine the plane normal vector corresponding to the fitted plane of the photovoltaic panel;

[0179] S73. Determine the coordinates of the preset origin in the photovoltaic panel coordinate system based on the straight line vector of the lower border and the straight line vector of the left border;

[0180] S74. Determine the rotation matrix of the photovoltaic panel coordinate system relative to the radar coordinate system based on the lower border line vector, the left border line vector, and the plane normal vector;

[0181] S75. Determine the pose transformation matrix based on the preset origin coordinates and the rotation matrix; determine the target pose estimation result based on the pose transformation matrix.

[0182] In this embodiment of the application, the planar point cloud border may further include: an upper point cloud border and a right point cloud border.

[0183] In a specific embodiment, a straight-line fitting method can be used to fit the lower point cloud bounding box to obtain the lower bounding box straight-line vector, denoted as... Similarly, a straight-line fitting method can be used to fit the left point cloud bounding box, resulting in a straight-line vector for the left bounding box, denoted as . Then, the plane normal vector corresponding to the fitted plane of the photovoltaic panel can be calculated, denoted as . Specifically, since the method for calculating the plane normal vector is a conventional technique, it will not be elaborated here.

[0184] Then, the coordinates of the intersection point of the bottom border line vector and the left border line vector can be calculated. These intersection point coordinates are the coordinates of the preset origin, denoted as O. r (x r ,y r ,z r Then, based on the bottom border line vector, the left border line vector, and the plane normal vector, the rotation matrix of the photovoltaic panel coordinate system relative to the radar coordinate system can be determined. This rotation matrix is: The pose transformation matrix is ​​determined based on the preset origin coordinates and the rotation matrix. The pose transformation matrix (homogeneous matrix form) is as follows:

[0185]

[0186] in, The pose transformation matrix is ​​used to represent the target pose estimation result. Specifically, the pose transformation matrix can be directly used as the target pose estimation result. Alternatively, the position coordinates and attitude parameters (such as Euler angles or quaternions) of the photovoltaic panel under test in the photovoltaic panel coordinate system (or radar coordinate system) can be calculated and output based on the pose transformation matrix. The attitude parameters describe the rotation angle of the photovoltaic panel under test, thus obtaining the target pose estimation result.

[0187] As can be seen, the photovoltaic panel pose estimation method described in this application scans the photovoltaic panel under test with a lidar to obtain an original point cloud set. Then, the original point cloud set is preprocessed to obtain a target point cloud set, which can remove irrelevant points (such as ground and debris) and noise points, reducing the sources of error in subsequent feature extraction and providing a clean data source for high-precision pose estimation. Next, the planar point cloud bounding box is extracted and a coordinate system is established with a preset position as the origin. The pose features of the photovoltaic panel are transformed into quantifiable coordinate parameters, which are directly associated with the rotation and translation components of the pose matrix, reducing the cumulative error of indirect calculation, thereby improving the pose estimation accuracy of the photovoltaic panel.

[0188] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a photovoltaic panel pose estimation device 700 provided in an embodiment of this application. The photovoltaic panel pose estimation device 700 includes: a scanning unit 701, a data processing unit 702, and a pose estimation unit 703, wherein:

[0189] The scanning unit 701 is used to scan the photovoltaic panel under test using a lidar to obtain an original point cloud set;

[0190] The data processing unit 702 is used to preprocess the original point cloud set to obtain a target point cloud set; determine an initial region point cloud set based on the target point cloud set; determine a photovoltaic panel point cloud set based on the initial region point cloud set, the conditional Euclidean algorithm, and the target point cloud set; extract the planar point cloud outline based on the photovoltaic panel point cloud set; and establish a photovoltaic panel coordinate system by using a preset position of the photovoltaic panel to be tested as the origin; the photovoltaic panel coordinate system is a three-dimensional coordinate system.

[0191] The pose estimation unit 703 is used to estimate the pose of the photovoltaic panel based on the photovoltaic panel coordinate system and the planar point cloud bounding box, and obtain the target pose estimation result.

[0192] In specific implementations, the photovoltaic panel pose estimation device 700 described in the embodiments of the present invention can also execute other implementations described in the photovoltaic panel pose estimation method provided in the embodiments of the present invention, which will not be repeated here.

[0193] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include a processor, a memory, a communication interface, and one or more programs. The processor, memory, and communication interface can be interconnected via a bus. The one or more programs are stored in the memory and configured to be executed by the processor. In this embodiment, the programs include instructions for performing the following steps:

[0194] The photovoltaic panel under test is scanned using lidar to obtain the original point cloud set;

[0195] The original point cloud set is preprocessed to obtain the target point cloud set;

[0196] Determine the initial region point cloud set based on the target point cloud set;

[0197] The photovoltaic panel point cloud set is determined based on the initial region point cloud set, the conditional Euclidean algorithm, and the target point cloud set.

[0198] Extract the planar point cloud outline based on the photovoltaic panel point cloud set;

[0199] A photovoltaic panel coordinate system is established by taking the preset position of the photovoltaic panel under test as the origin; the photovoltaic panel coordinate system is a three-dimensional coordinate system.

[0200] The photovoltaic panel pose is estimated based on the photovoltaic panel coordinate system and the planar point cloud bounding box to obtain the target pose estimation result.

[0201] In a specific embodiment, the above procedure further includes causing the electronic device to perform some or all of the steps of any of the methods described in the above method embodiments.

[0202] This application also provides a computer-readable storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0203] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0204] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0205] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0206] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0207] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0208] The steps of the methods or algorithms described in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, EPROM, electrically erasable programmable read-only memory (EEPROM), registers, hard disk, portable hard disk, read-only optical disk (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Furthermore, the ASIC can reside in a terminal device or management device. Alternatively, the processor and storage medium can exist as discrete components in the terminal device or management device.

[0209] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in the embodiments of this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated.

[0210] The aforementioned computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media.

[0211] The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0212] The modules / units included in the various devices and products described in the above embodiments can be software modules / units, hardware modules / units, or a combination of both. For example, for devices and products applied to or integrated into a chip, all modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits. For devices and products applied to or integrated into a chip module, all modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The implementation is achieved through a software program that runs on a processor integrated within the chip module. The remaining modules / units (if any) can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into terminal equipment, each of their modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components within the terminal equipment. Alternatively, at least some modules / units can be implemented using a software program that runs on a processor integrated within the terminal equipment, while the remaining modules / units (if any) can be implemented using hardware methods such as circuits.

[0213] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above descriptions are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A method for estimating the pose of a photovoltaic panel, characterized in that, include: The photovoltaic panel under test is scanned using lidar to obtain the original point cloud set; The original point cloud set is preprocessed to obtain the target point cloud set; Determine the initial region point cloud set based on the target point cloud set; The photovoltaic panel point cloud set is determined based on the initial region point cloud set, the conditional Euclidean algorithm, and the target point cloud set. Extract the planar point cloud outline based on the photovoltaic panel point cloud set; A photovoltaic panel coordinate system is established by taking the preset position of the photovoltaic panel under test as the origin; the photovoltaic panel coordinate system is a three-dimensional coordinate system. The photovoltaic panel pose is estimated based on the photovoltaic panel coordinate system and the planar point cloud bounding box to obtain the target pose estimation result.

2. The method as described in claim 1, characterized in that, The LiDAR uses a non-repeating scanning method. The data preprocessing of the original point cloud set to obtain the target point cloud set includes: Obtain the target size information of the photovoltaic panel under test; Determine the estimated coordinate range based on the target size information; Based on the estimated coordinate range, the original point cloud set is subjected to pass-through filtering to obtain the first point cloud set; The first point cloud set is downsampled using a voxel filtering method to obtain the second point cloud set; Determine the average distance between each point in the second point cloud set and its surrounding neighboring points to obtain multiple average distances; Determine the average distance among the plurality of average distances that is greater than a first preset distance to obtain at least one average distance; Determine the point in the second point cloud set corresponding to the at least one average distance to obtain at least one isolated noise point; The target point cloud set is obtained by removing the at least one isolated noise point from the second point cloud set.

3. The method as described in claim 1 or 2, characterized in that, The lidar is mounted on the photovoltaic panel installation robot; the step of determining the initial region point cloud set based on the target point cloud set includes: The center position of the lidar, the first stopping position of the photovoltaic panel laying robot, and the relative position of the photovoltaic panel laying robot and the photovoltaic panel to be tested are obtained. A radar coordinate system is established with the central position as the origin; the radar coordinate system is a three-dimensional coordinate system. Based on the radar coordinate system, the area around the center position at a second preset distance is taken as the initial area; Based on the target point cloud set, the points corresponding to the initial region are determined to obtain the third point cloud set; A preset plane fitting algorithm is used to perform plane fitting on the third point cloud set to obtain the plane normal; The initial region point cloud set is determined based on the plane normal, the radar coordinate system, and the third point cloud set.

4. The method as described in claim 3, characterized in that, Determining the initial region point cloud set based on the plane normal, the radar coordinate system, and the third point cloud set includes: The target angle is obtained by determining the angle between the plane normal and the unit vector of the Z-axis of the radar coordinate system; Determine the target included angle threshold based on the relative position; When the target angle is less than the target angle threshold, the third point cloud set is determined as the initial region point cloud set; When the target angle is greater than or equal to the target angle threshold, the photovoltaic panel installation robot is controlled to move from the first parking position to the second parking position.

5. The method as described in claim 3, characterized in that, The target point cloud set includes n points; the initial region point cloud set includes m points, where n and m are both positive integers, and m is less than or equal to n; the step of determining the photovoltaic panel point cloud set based on the initial region point cloud set, the conditional Euclidean algorithm, and the target point cloud set includes: Initialize the initial parameter set of the conditional Euclidean algorithm; Determine the normal vector of each point in the target point cloud set to obtain the target normal vector data; Initialize the target list and the target plane container; the target list initialization includes n identifiers, each identifier corresponding to one of the n points, and the target list is used to record the access status of each point; the target plane container is initialized to empty, and the target plane container is used to store the clustering results; Traverse each point in the initial region point cloud set, and cluster each point based on the initial algorithm parameter set, the target normal vector data, the target list, and the target plane container to obtain i clustering results; i is a positive integer less than m; Determine the fitting plane corresponding to each of the i clustering results to obtain i fitting planes; The i fitting planes are merged to obtain the photovoltaic panel fitting plane; The photovoltaic panel point cloud set is determined based on the fitted plane of the photovoltaic panel.

6. The method as described in claim 5, characterized in that, The initial algorithm parameter set includes: spatial distance threshold, normal vector angle threshold, and minimum number of cluster points; The process involves traversing each point in the initial region point cloud set, and clustering each point based on the initial algorithm parameter set, the target normal vector data, the target list, and the target plane container to obtain i clustering results, including: Determine the point being traversed and obtain the current point; Determine whether the current point has been visited based on the target list; if it has been visited, skip the current point. If not visited, the current point is taken as the starting point of the new plane, and the first plane container and the first queue of points to be processed are initialized. The first plane container and the first queue of points to be processed are both initialized to empty. Add the new plane starting point to the first queue of points to be processed; S1. Take out the first point from the head of the first pending point queue, add the first point to the first planar container; set the identifier corresponding to the first point in the target list to be visited. S2. Based on the target point cloud set and the target list, find unvisited neighbor points within the spatial distance threshold around the first point to obtain k points; k is a natural number. S3. Based on the target normal vector data, determine the k normal vectors corresponding to the k points, and the first normal vector corresponding to the first point; S4. Determine the angle between each of the k normal vectors and the first normal vector to obtain k angles; S5. Determine the angles among the k included angles that are less than or equal to the normal vector included angle threshold, to obtain h included angles; h is a natural number less than or equal to k; S6. Add the h points corresponding to the h included angles in the k points to the first queue of points to be processed, and set the h identifiers corresponding to the h points in the target list to be visited. S7. Determine whether the queue of the first point to be processed is empty; S8. If the first queue of points to be processed is empty, then determine the reference clustering result based on the first planar container; S9. If the first queue of points to be processed is not empty, repeat steps S1-S7 until the first queue of points to be processed is empty, and determine the reference clustering result based on the first planar container. Determine the number of reference cluster points corresponding to the reference clustering result; When the number of reference cluster points is greater than or equal to the number of minimum cluster points, the clustering result corresponding to the starting point of the new plane is determined based on the reference clustering result; When the number of reference cluster points is less than the number of minimum cluster points, the clustering at the starting point of the new plane is determined to have failed.

7. The method as described in claim 5 or 6, characterized in that, The step of extracting the planar point cloud bounding box based on the photovoltaic panel point cloud set includes: The photovoltaic panel point cloud set is projected onto a two-dimensional plane to obtain a two-dimensional point cloud set; The target coordinate range information is determined based on the two-dimensional point cloud set; The two-dimensional point cloud set is processed based on the target coordinate range information and the latitude and longitude scanning algorithm to obtain the planar point cloud bounding box.

8. The method as described in claim 7, characterized in that, The planar point cloud border includes: a lower point cloud border and a left point cloud border; the preset position is the intersection of the lower point cloud border and the left point cloud border. The process of estimating the photovoltaic panel pose based on the photovoltaic panel coordinate system and the planar point cloud bounding box to obtain the target pose estimation result includes: Determine the lower border line vector corresponding to the lower point cloud border, and the left border line vector corresponding to the left point cloud border; Determine the plane normal vector corresponding to the fitted plane of the photovoltaic panel; Based on the straight line vectors of the lower border and the left border, determine the coordinates of the preset origin corresponding to the preset position in the photovoltaic panel coordinate system; The rotation matrix of the photovoltaic panel coordinate system relative to the radar coordinate system is determined based on the straight line vector of the lower border, the straight line vector of the left border, and the plane normal vector. The pose transformation matrix is ​​determined based on the preset origin coordinates and the rotation matrix; the target pose estimation result is determined based on the pose transformation matrix.

9. A photovoltaic panel pose estimation device, characterized in that, include: The unit comprises a scanning unit, a data processing unit, and a pose estimation unit, wherein: The scanning unit is used to scan the photovoltaic panel under test using a lidar to obtain the original point cloud set; The data processing unit is used to preprocess the original point cloud set to obtain a target point cloud set; determine an initial region point cloud set based on the target point cloud set; determine a photovoltaic panel point cloud set based on the initial region point cloud set, the conditional Euclidean algorithm, and the target point cloud set; extract the planar point cloud bounding box based on the photovoltaic panel point cloud set; and establish a photovoltaic panel coordinate system by using a preset position of the photovoltaic panel to be tested as the origin; the photovoltaic panel coordinate system is a three-dimensional coordinate system. The pose estimation unit is used to estimate the pose of the photovoltaic panel based on the photovoltaic panel coordinate system and the planar point cloud bounding box, and obtain the target pose estimation result.

10. An electronic device, characterized in that, include: Processor, memory, communication interface, and one or more programs; The one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing the steps of the method as described in any one of claims 1-8.

Images