Straight line detection method, positioning device and system based on photovoltaic robot

Abstract

The present application discloses a photovoltaic robot-based line detection method, photovoltaic robot positioning device, and system. The method includes: determining a first plane line based on two-dimensional data of a first edge point and a second edge point; determining the first plane line as a first candidate line when the number of inliers of the first plane line is greater than or equal to a first preset number; determining the candidate line with the largest number of inliers among multiple first candidate lines as an initial target candidate line; performing cyclic line detection based on the initial target candidate line to determine the target line; when there is a gap in the point cloud region where the target line is located, dividing the point cloud region into multiple sub-point cloud regions, performing line detection in each of the different sub-point cloud regions to obtain multiple sub-target lines; and selecting a new target line from the multiple sub-target lines. In this way, the line where the target structure of the photovoltaic support is located can be accurately detected.

Description

[0001] This case is a divisional application based on the invention patent with application date of September 24, 2024, application number "202411328439.7", and name "Positioning method, device and system for photovoltaic robot" as the parent case. Technical Field

[0002] The present application relates to the field of photovoltaic technology, and in particular to a straight line detection method, positioning device and system based on a photovoltaic robot. Background Art

[0003] Photovoltaic panels convert solar energy into electricity. They are mounted on mounting brackets, which secure and support the panels, maximizing their ability to absorb and convert solar energy. If a robot is used to automatically install the panels, any abnormal positioning of the robot relative to the mounting bracket could cause the panels to deviate from their original position, impacting the robot's efficiency. In worse cases, the robot could collide with the panels, potentially damaging them.

[0004] Therefore, how to accurately position the photovoltaic robot is very important. Summary of the Invention

[0005] In order to solve the above technical problems, the present application provides a positioning method, device and system for a photovoltaic robot, so as to accurately position the photovoltaic robot.

[0006] Specifically, the technical solution of this application is as follows:

[0007] In a first aspect, a positioning method for a photovoltaic robot is provided, which is used for the photovoltaic robot to install a photovoltaic panel on a photovoltaic bracket. The positioning method is used to determine the position information of the photovoltaic robot during the installation of the photovoltaic panel. The positioning method includes:

[0008] Obtain three-dimensional point cloud data of the photovoltaic bracket; the three-dimensional point cloud data is obtained through the sensing device;

[0009] Based on the three-dimensional point cloud data, a target straight line is determined; the target straight line includes a straight line where the target structure of the photovoltaic bracket is located, and the target straight line is located in the plane where the photovoltaic bracket is located;

[0010] Position information is determined based on a first rotation matrix between a first coordinate system of the sensing device and a second coordinate system of the photovoltaic robot, as well as the target straight line; the position information is used to indicate a relative positional relationship between the photovoltaic robot and the target structure.

[0011] In this way, the relative positional relationship between the photovoltaic robot and the photovoltaic rack is determined based on the 3D point cloud data of the photovoltaic rack. This allows the photovoltaic robot to be accurately positioned during the installation of photovoltaic panels, allowing for timely detection of posture anomalies in the photovoltaic robot. Furthermore, the above positioning method determines the target line based on the 3D point cloud data, where the target line is a planar line. By converting the 3D point cloud data into a 2D planar line, dimensionality reduction can be used to optimize the data processing process and improve positioning efficiency.

[0012] In one implementation, the above determines the target straight line based on three-dimensional point cloud data, including: extracting edge point cloud data from the three-dimensional point cloud data; the three-dimensional point cloud data includes the three-dimensional position coordinates of multiple spatial points, and the edge point cloud data includes the three-dimensional position coordinates of multiple edge points; multiple edge points constitute the point cloud of the geometric shape boundary of the photovoltaic bracket; two-dimensionally processing the edge point cloud data to obtain two-dimensional data of multiple edge points; and determining the target straight line based on the two-dimensional data of multiple edge points.

[0013] In this way, edge detection is used to extract the point cloud of the photovoltaic bracket's geometric shape boundary from the point cloud of the photovoltaic bracket, thereby obtaining edge point cloud data. Based on this edge point cloud data, the target line corresponding to the target structure is accurately identified.

[0014] In one implementation, determining the target line based on the two-dimensional data of multiple edge points includes:

[0015] Selecting a first edge point and a second edge point from a plurality of edge points; determining a first plane line based on two-dimensional data of the first edge point and the second edge point; and determining the first plane line as a first candidate line when the number of inner points of the first plane line is greater than or equal to a first preset number;

[0016] Traversing multiple edge points, when there are multiple first candidate lines, determining the candidate line with the largest number of inliers among the multiple first candidate lines as the initial target candidate line;

[0017] Selecting a third edge point and a fourth edge point from the initial edge point set; determining a second plane line based on the two-dimensional data of the third edge point and the fourth edge point; determining the second plane line as a current target candidate line when the number of inliers of the second plane line is greater than or equal to a second preset number; the initial edge point set includes the edge points remaining after removing the inliers of the initial target candidate line from the plurality of edge points;

[0018] Repeat the following iterative process until the number of target candidate lines is greater than or equal to a third preset number:

[0019] Selecting a fifth edge point and a sixth edge point from the current edge point set; determining a third plane line based on the two-dimensional data of the fifth edge point and the sixth edge point; updating the current target candidate line based on the third plane line when the number of inliers of the third plane line is greater than or equal to a fourth preset number; the current edge point set includes the edge points remaining after removing the inliers of the current target candidate line from the initial edge point set;

[0020] When the number of target candidate lines is greater than or equal to a third preset number, the target candidate line with the largest number of inliers is determined as the target line.

[0021] In this way, the initial candidate target lines are iterated in a loop to determine the final target line. This iterative approach improves the accuracy of target line detection.

[0022] In one implementation, the multiple inliers of the target line are divided into multiple inlier sets, and the inliers of the target line that meet the first adjacency condition belong to the same inlier set. The above positioning method further includes: performing line fitting based on the multiple inlier sets to determine multiple fourth-plane lines; determining multiple first-plane lines based on the line with the largest number of inliers among the multiple fourth-plane lines, and re-determining the multiple target lines; and selecting a new target line from the re-determined multiple target lines. The new target line includes the line with the longest length and / or the line with the largest number of inliers among the re-determined multiple target lines.

[0023] In one implementation, the above positioning method also includes: determining a first distance between the multiple edge points and the first plane straight line based on the two-dimensional data of the multiple edge points; when the first distance is less than or equal to the first preset distance, determining the edge point corresponding to the first distance from the multiple edge points as an inner point of the first plane straight line.

[0024] In one implementation, multiple spatial points constitute a first point cloud region; the multiple spatial points include the first point, the first point also belongs to a second point cloud region, the second point cloud region is smaller than or equal to the first point cloud region; and the spatial points in the second point cloud region meet a second adjacent condition; the second point cloud region also includes the second point. The above method of extracting edge point cloud data of a photovoltaic bracket from three-dimensional point cloud data includes:

[0025] Determine the curvature information of the first point based on the three-dimensional position coordinates of the second point cloud area; the curvature information includes the curvature value of the first point and / or the angle value between the normal of the first point and the second point; when the curvature information meets the preset curvature condition, the first point is used as an edge point.

[0026] In one implementation, the above determining the curvature information of the first point based on the three-dimensional position coordinates of the second point cloud area includes: performing surface fitting with the first point and the second point as center points, respectively, according to the three-dimensional position coordinates of the second point cloud area, to obtain the first surface and the second surface; determining the curvature value of the first point and the angle value between the normal of the first point and the normal of the second point based on the feature information of the first surface and the second surface.

[0027] In one implementation, the method further includes: performing filtering processing on the three-dimensional point cloud data; the filtering processing includes one or more of straight-through filtering, voxel filtering, and outlier filtering.

[0028] In one implementation, the position information includes a first angle between the photovoltaic robot and the target structure. Determining the position information based on a first rotation matrix between a first coordinate system of the sensing device and a second coordinate system of the photovoltaic robot, as well as the target line, includes: determining a second angle between the target line and the first direction; determining a second rotation matrix between the first coordinate system and a third coordinate system based on the second angle; the third coordinate system including the coordinate system of the target structure; determining a third rotation matrix between the second coordinate system and the third coordinate system based on the first rotation matrix and the second rotation matrix; and determining the first angle based on the third rotation matrix.

[0029] In a second aspect, a positioning device for a photovoltaic robot is provided, comprising a processor for calling instructions stored in a memory, wherein when the instructions are called by the processor, the processor executes the positioning method implemented in the first aspect or any one of the first aspects.

[0030] In a third aspect, a photovoltaic robot positioning system is provided, comprising:

[0031] A sensing device is installed on the photovoltaic robot and is used to collect sensing data of the photovoltaic support, where the sensing data includes three-dimensional point cloud data, or the sensing data is used to determine the three-dimensional point cloud data;

[0032] The positioning device of the second aspect is coupled to a sensing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The following is a brief introduction to the drawings used in describing the embodiments of this application:

[0034] Figure 1 is a schematic diagram of a photovoltaic panel installation scenario provided in an embodiment of the present application;

[0035] Figure 2 This is a schematic structural diagram of a photovoltaic robot positioning system provided in an embodiment of the present application;

[0036] Figure 3 This is a flow chart of a photovoltaic robot positioning method provided in an embodiment of the present application;

[0037] Figure 4 This is a flow chart of determining a target straight line based on three-dimensional point cloud data of a photovoltaic bracket provided in an embodiment of the present application;

[0038] Figure 5 Schematic diagram of a target straight line determination process provided in an embodiment of the present application;

[0039] Figure 6 It is a structural schematic diagram of a positioning device for a photovoltaic robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the specific implementation methods of the present application will be described below with reference to the accompanying drawings. The drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative work. Adjustments and improvements made without departing from the concept of the present application are all within the scope of protection of the present application.

[0041] To simplify the drawings, the drawings in the embodiments of this application schematically illustrate only the portions relevant to the present application and do not represent the actual structure of the product. Furthermore, to simplify the drawings and facilitate understanding, in some drawings, only a portion of components with the same structure or function are schematically depicted; in practice, more or fewer components with the same structure or function may exist.

[0042] In this application, unless otherwise expressly specified and limited, ordinal numbers such as "first", "second", etc. are only used to distinguish and describe related objects, and cannot be understood as indicating or implying the relative importance or order between related objects; in addition, they do not represent the number of related objects. "Multiple" includes two or more, and other quantifiers are similar. " / " is used to describe the relationship between related objects, which indicates the "or" relationship between related objects. "One or more" or "at least one" of multiple objects refers to any object or any combination of multiple objects, such as "one or more of a1, a2, a3" or "at least one of a1, a2, a3" includes: "alone a1", "alone a2", "alone a3", "a1 and a2", "a1 and a3", "a2 and a3" or "a1, a2 and a3".

[0043] The terms "installation" and "connection" should be interpreted broadly. For example, "installation" can mean direct installation or installation through other components; "connection" can mean fixed connection, detachable connection, or integral connection; it can be mechanical connection or electrical connection; it can be direct connection or indirect connection through an intermediate medium, and it can mean internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in this application in specific circumstances.

[0044] In photovoltaic power generation scenarios, photovoltaic panels (also known as photovoltaic modules) are fixed and supported by photovoltaic brackets. The photovoltaic brackets allow the photovoltaic panels to face the sun at the optimal angle to maximize the absorption of solar energy and convert it into electricity. Photovoltaic panels are installed by photovoltaic robots. Furthermore, photovoltaic robots also include mobile devices to facilitate the movement of photovoltaic panels to different locations for installation. Please refer to Figure 1 , which is a schematic diagram of a photovoltaic panel installation scenario provided in an embodiment of the present application. Figure 1 As shown, assuming Figure 1 The direction of the middle arrow is the travel direction of the photovoltaic robot 110. The photovoltaic robot 110 moves relative to the photovoltaic support 120 along this travel direction and installs the photovoltaic panels on the corresponding positions of the photovoltaic support 120 during the travel process, as shown by the shadows in the figure.

[0045] The photovoltaic robot has an initial posture. This initial posture maintains a certain relative position relationship with the photovoltaic bracket, such as being perpendicular to the ground, flush with the tilt angle of the photovoltaic bracket, etc. If the photovoltaic robot maintains this initial posture or is within the error range of this initial posture during the installation process, it can accurately install the photovoltaic panel to the appropriate position. However, during the movement of the photovoltaic robot, the ground may be uneven or the control error of the photovoltaic robot may occur, causing the posture of the photovoltaic robot to change, such as Figure 1 The dashed boxes in the figure represent several possible changes in the robot's posture. These are examples only and do not represent the actual posture changes of the PV robot. Such changes can cause the PV robot's current posture to deviate from its initial posture, leading to deviations in the installation position of the PV panels and preventing accurate installation on the PV mounts. As the PV robot continues to advance, these deviations accumulate, ultimately requiring manual adjustments and increasing maintenance costs.

[0046] It can be seen that changes in the posture of the photovoltaic robot may affect the installation of photovoltaic panels. In view of the above problems, the embodiments of the present application propose a photovoltaic robot positioning method, device, and other solutions, which accurately position the photovoltaic robot and facilitate timely detection of abnormal posture of the photovoltaic robot, so that the posture of the photovoltaic robot can be controlled and adjusted in time, reducing deviations during the photovoltaic panel installation process, which is conducive to accurately installing the photovoltaic panels in the appropriate position.

[0047] The following description is given with reference to the accompanying drawings.

[0048] Please refer to Figure 2 , which is a schematic diagram of the structure of a photovoltaic robot positioning system provided in an embodiment of the present application. Figure 2 As shown, the positioning system 200 includes a photovoltaic robot 210, a sensing device 220, and a positioning device 230. The sensing device 220 is used to collect sensing data of the photovoltaic bracket. The sensing data includes, for example, three-dimensional point cloud data; or the sensing data is used to determine three-dimensional point cloud data, and the sensing data is, for example, a depth image. The sensing device 220 can be installed on the photovoltaic robot 210 to collect sensing data of the photovoltaic bracket in real time during the installation of the photovoltaic panel. For example, the sensing device 220 can be installed on the robot's robotic arm to obtain a better sensing field of view. In one implementation, the sensing device 220 can be installed at the end of the robotic arm and maintain a preset vertical distance (for example, 1-2 meters) from the photovoltaic bracket, so as to collect sensing data of the photovoltaic bracket from a bird's-eye view. The sensing device 220 can also be installed on the photovoltaic bracket, for example, by installing the sensing device 220 at a designated position on the photovoltaic bracket to collect sensing data of the photovoltaic bracket under a fixed field of view. The embodiments of the present application do not limit the number and type of sensing devices 220. The positioning system 200 may include one or more sensing devices 220, such as depth cameras or lidar. The positioning device 230 is coupled to the sensing device 220 and is used to determine the position information of the photovoltaic robot 210 during the installation of the photovoltaic panels. The positioning device 230 can also obtain three-dimensional point cloud data of the photovoltaic support through the sensing device 220, for example, directly obtaining the three-dimensional point cloud data collected by the sensing device 220, or obtaining the three-dimensional point cloud data obtained by converting the depth image data. The positioning device 230 determines the position information of the photovoltaic robot 210 based on the obtained three-dimensional point cloud data. In the embodiments of the present application, the photovoltaic support can be a tracking photovoltaic support such as a flat single-axis tracking support or an inclined single-axis tracking support, or a fixed photovoltaic support such as an adjustable tilt fixed support or a maximum tilt fixed support. The embodiments of the present application do not limit the type of photovoltaic support.

[0049] The positioning method of the photovoltaic robot is described below with reference to the accompanying drawings.

[0050] Please refer to Figure 3 , which shows a flow chart of a photovoltaic robot positioning method provided in some embodiments of the present application. The method is executed by the above positioning device. The positioning device can be installed on the photovoltaic robot; or it can be set independently of the photovoltaic robot, such as being located in a server or terminal device. The positioning method is used to determine the position information of the photovoltaic robot during the installation of photovoltaic panels. Figure 3 As shown, the positioning method includes at least the following steps:

[0051] S310: Acquire three-dimensional point cloud data of the photovoltaic bracket; the three-dimensional point cloud data is obtained by a sensing device;

[0052] S320: Determine a target straight line based on the three-dimensional point cloud data; the target straight line includes a straight line where the target structure of the photovoltaic bracket is located, and the target straight line is located in the plane where the photovoltaic bracket is located;

[0053] S330: Determine the position information of the photovoltaic robot based on the first rotation matrix between the first coordinate system of the sensing device and the second coordinate system of the photovoltaic robot, as well as the target straight line; the position information is used to indicate the relative position relationship between the photovoltaic robot and the target structure.

[0054] The above positioning method uses a sensing device to acquire 3D point cloud data of the photovoltaic bracket and uses this data to determine a target line. This target line is the line where the target structure of the photovoltaic bracket is located. The target structure is used to locate the installation position of the photovoltaic panel on the photovoltaic bracket. For example, the photovoltaic bracket's rotation axis, crossbeam, etc. can serve as a target structure. The embodiments of this application do not limit the choice of target structure.

[0055] Therefore, the relative positional relationship between the photovoltaic robot and the target structure is determined based on the 3D point cloud data of the photovoltaic support. This allows the photovoltaic robot to be accurately positioned during the installation of photovoltaic panels, facilitating the timely detection of posture anomalies. Furthermore, the above positioning method determines the target line based on the 3D point cloud data, where the target line is a planar line. By converting the 3D point cloud data into a 2D planar line, dimensionality reduction can be used to optimize the data processing process and improve positioning efficiency.

[0056] The photovoltaic robot's position information includes the position information of one or more reference points on the robot. The reference point can be, for example, any point on the robot. For example, the reference point can include the robot's center of mass or center. For example, the photovoltaic robot includes a robotic arm that is used to grasp and place photovoltaic panels. In this case, the reference point can be any point on the robotic arm, such as its center of mass or center, to further improve positioning accuracy.

[0057] In one implementation, the above position information includes a first angle between the photovoltaic robot and the target structure. For example, the angle between a line containing one or more reference points on the photovoltaic robot and the target line. Furthermore, determining the position information of the photovoltaic robot based on the first rotation matrix and the target line includes at least the following steps:

[0058] S331: Determine a second angle between the target straight line and the first direction;

[0059] S332: Determine a second rotation matrix between the first coordinate system and a third coordinate system according to the second included angle; the third coordinate system includes a coordinate system of the target structure;

[0060] S333: Determine a third rotation matrix between the second coordinate system and the third coordinate system according to the first rotation matrix and the second rotation matrix;

[0061] S334: Determine a first angle based on the third rotation matrix.

[0062] In the embodiment of the present application, the target straight line is located in the plane where the photovoltaic bracket is located, that is, the target straight line is a plane straight line. After determining the target straight line, its straight line equation y=ax+b can be obtained. The slope a of the target straight line represents the angle θ (i.e., the first angle) between it and the first direction. The first direction is, for example, the positive direction or negative direction of any coordinate axis of the photovoltaic array coordinate system. In one implementation, such as Figure 1 As shown, the photovoltaic array plane coordinate system is defined as the photovoltaic array width direction is the X axis direction, the photovoltaic array direction is the Y axis direction, and the vertical photovoltaic panel upward is the positive direction of the Z axis. For example, the first direction can be Figure 1 The positive direction of the center X axis.

[0063] The angle θ can be calculated using the inverse tangent function:

[0064] θ = arctan(a);

[0065] The angle value is converted into a rotation matrix, that is, the second rotation matrix R from the first coordinate system of the sensing device to the third coordinate system. The rotation matrix can be represented by the Euler angle of the rotation transformation. For the matrix rotating around the a-axis (perpendicular to the ground), it can be obtained by the following formula:

[0066]

[0067] Since the first rotation matrix R' of the second coordinate system of the photovoltaic robot to the first coordinate system of the sensing device can be obtained by pre-calibration (that is, R' is known), the third rotation matrix R between the second coordinate system and the third coordinate system can be determined by multiplying the above two rotation matrices. T :

[0068] R T =R·R';

[0069] Through the rotation matrix R T The above position information can be determined to indicate the relative position relationship between the photovoltaic robot and the target structure, for example, to determine the above first angle. In one implementation, the first angle can be determined by: determining the rotation matrix R T The eigenvalue λ of ; taking the eigenvalue λ as the independent variable, the first angle is determined by the inverse cosine function.

[0070] In one implementation, the above position information includes the distance between the photovoltaic robot and the target structure. For example, the distance between a first reference point on the photovoltaic robot and a second reference point on the target structure. The first reference point is, for example, any point on the photovoltaic robot, and the second reference point is, for example, any point on the target structure. Based on the coordinates of the first reference point in the first coordinate system, the rotation matrix R T As well as the straight line equation of the target straight line where the target structure is located, the distance between the first reference point and the second reference point can be determined.

[0071] Furthermore, in one implementation, if the position information is outside a preset error range, it can be corrected. For example, if the first angle exceeds the error range, the PV robot's position can be controlled to adjust the angle. The preset error range can be, for example, the error range of the PV robot's initial position. By correcting the PV robot's position, errors caused by abnormal PV robot position during the PV panel installation process can be further reduced.

[0072] The following describes the process of determining the target straight line based on the three-dimensional point cloud data of the photovoltaic bracket in the above step S320.

[0073] A point cloud is a collection of multiple spatial points. A point cloud includes multiple discrete points, each of which can correspond to a spatial position, which can be represented by three-dimensional position coordinates (x, y, z). For example, in the coordinate system of the sensing device, the front of the sensing device is the positive direction of the X-axis, the left side of the sensing device is the positive direction of the Y-axis, and the Z-axis is perpendicular to the ground. For another example, in the coordinate system of a photovoltaic robot, the front of the photovoltaic robot is the positive direction of the X-axis, the left side of the photovoltaic robot is the positive direction of the Y-axis, and the Z-axis is perpendicular to the ground. The above description of three-dimensional position coordinates is for example only. The embodiments of the present application do not limit the type of coordinate system or the selection of coordinate axes, as long as the spatial position of the point in the point cloud can be indicated.

[0074] Please refer to Figure 4 , which is a flow chart of determining a target straight line based on three-dimensional point cloud data of a photovoltaic bracket provided in an embodiment of the present application. Figure 4 As shown, the above step S320 may include:

[0075] S321: extracting edge point cloud data from the three-dimensional point cloud data; the three-dimensional point cloud data includes the three-dimensional position coordinates of multiple spatial points, and the edge point cloud data includes the three-dimensional position coordinates of multiple edge points; the multiple edge points constitute a point cloud of the geometric shape boundary of the photovoltaic bracket;

[0076] S322: Perform two-dimensional processing on the edge point cloud data to obtain two-dimensional data of multiple edge points;

[0077] S323: Determine a target straight line based on the two-dimensional data of the multiple edge points.

[0078] The data for a spatial point in a point cloud can include the three-dimensional position coordinates of that point. The three-dimensional position coordinates of multiple spatial points constitute the three-dimensional point cloud data. The three-dimensional point cloud data of a photovoltaic bracket is the three-dimensional point cloud data collected by a sensing device within a sensing field of view. This sensing field includes the photovoltaic bracket and the environment in which it is located. The three-dimensional point cloud data of a photovoltaic bracket (also known as raw point cloud data) includes not only the point cloud data of the photovoltaic bracket but may also include point cloud data of the environment. Therefore, edge detection methods can be used to extract the point cloud of the photovoltaic bracket's geometric boundary from the photovoltaic bracket's point cloud, thereby obtaining edge point cloud data. Edge point cloud data includes the three-dimensional position coordinates of multiple edge points, which together constitute the point cloud of the photovoltaic bracket's geometric boundary. Furthermore, the edge point cloud data is converted into two-dimensional data of multiple edge points through two-dimensional processing. Based on the two-dimensional data of these edge points, a target line can be identified. Two-dimensional processing can include, for example, setting the Z-axis coordinate value of an edge point to 1 or projecting the three-dimensional position coordinates of an edge point onto a two-dimensional plane to obtain the two-dimensional position coordinates of the edge point. In one implementation, the original point cloud data may be first converted into a two-dimensional plane image, and edge detection may be performed on the two-dimensional plane image to determine the target straight line.

[0079] In some implementations of the edge detection process ( S321 ), edge points may be extracted using curvature information of the point cloud. For example, the curvature information of the first point may be determined based on the three-dimensional position coordinates of the second point cloud region.

[0080] The original point cloud data includes the three-dimensional position coordinates of multiple spatial points (original point clouds). The multiple spatial points constitute the first point cloud area. Among them, the multiple spatial points include point (first point), that is, the first point belongs to the first point cloud area. The first point also belongs to the second point cloud area, the second point cloud area is smaller than or equal to the first point cloud area, and the spatial points in the second point cloud area meet the preset adjacent condition (for the convenience of distinction, it is called the second adjacent condition). The second adjacent condition is, for example, the distance between any spatial point in the second point cloud area and the first point is less than or equal to the preset distance (for the convenience of distinction, it is called the second preset distance). That is, the point and points in its neighborhood Constitute the second point cloud area.

[0081] In one implementation, the curvature information may include the point The curvature value can be determined by surface fitting. For example, according to the point and points in its neighborhood The surface is fitted by the least square method to obtain the first surface; according to the characteristic information of the first surface, the point The curvature value κ (of the first point) is:

[0082]

[0083] The characteristic information includes the eigenvalues ​​λ0, λ1, and λ2 of the covariance matrix of the first surface.

[0084] In one implementation, the curvature information may include the point Normal and point The angle value between the normal line of the point. This angle value can also be determined by surface fitting. and point (Second point) as an example, point Belong to the second point cloud area. and point With the first point as the center point, surface fitting is performed to obtain a first surface and a second surface. In this case, the feature information may include a normal vector. The normal vector of the first point can be obtained from the first surface, and the normal vector of the second point can be obtained from the second surface. Thus, the angle between the two normal vectors can be determined, which is the angle between the normal of the first point and the normal of the second point.

[0085] When the above curvature information meets the preset curvature condition, the first point is taken as the edge point. The preset curvature condition can include any of the following: When the curvature value of is greater than or equal to the curvature threshold; Normal and point The angle between the normals of point is greater than or equal to the angle threshold; or The curvature value of point P→ is greater than the curvature threshold, and the normal of point P→ i The angle between the normals of is greater than the angle threshold.

[0086] For each spatial point in the original point cloud, the above edge detection process can be used to determine whether it is an edge point. Thus, an edge point cloud is extracted from the original point cloud. The edge point cloud comprises multiple edge points, and the edge point cloud data includes the 3D position coordinates of these multiple edge points. Furthermore, through 2D processing, the edge point cloud data is converted into 2D data of the multiple edge points. Based on this 2D data of the multiple edge points, a target line can be determined. The process of determining the target line is described below.

[0087] The multiple edge points include a first edge point Q1 (x1, y1, z1) and a second edge point Q2 (x2, y2, z2). Q1 and Q2 are any two edge points in the edge point cloud. The values ​​of z1 and z2 in the three-dimensional position coordinates of Q1 and Q2 are both set to 1, obtaining two-dimensional data for Q1 and Q2, namely Q1 (x1, y1, 1) and Q2 (x2, y2, 1). Linear fitting based on these two coordinates can determine a line L1, which can be considered a first plane line. Similarly, the above method can be used to obtain two-dimensional data for other edge points in the multiple edge points. Based on the two-dimensional data of the multiple edge points, an inlier of line L1 can be selected from the multiple edge points. In one implementation, based on the two-dimensional data of the multiple edge points, an inlier of line L1 can be determined from the multiple edge points: a first distance between the multiple edge points and line L1 is determined based on the two-dimensional data of the multiple edge points; when the first distance is less than or equal to a first preset distance, the edge point corresponding to the first distance is determined from the multiple edge points as an inlier of the first plane line. That is, the inner point of the line L1 is determined by the distance between each edge point and the line L1. That is, among the multiple edge points, the edge point whose distance d from the line L1 is less than or equal to the first preset distance is the inner point of the line L1.

[0088] When the number of inliers in line L1 is greater than or equal to a first preset number, line L1 is determined to be the first candidate line. In one implementation, a set number of iterations, N, can be set to traverse multiple edge points, select Q1 and Q2 multiple times, and repeat the above process to determine the first candidate line. If there are multiple first candidate lines, the candidate line with the largest number of inliers among the multiple first candidate lines is determined as the initial target candidate line. In the initial implementation, the initial target candidate line can be used as the target line.

[0089] Alternatively, a loop iteration is performed based on the initial target candidate line to determine the final target line. The accuracy of target line detection is improved through loop iteration. Please refer to Figure 5 , which is a schematic diagram of a process of cyclic straight line detection provided in an embodiment of the present application. Figure 5 As shown, after removing the inner points of the initial target candidate line from the multiple edge points, the remaining edge points form the initial edge point set. A third edge point Q3 (x3, y3, z3) and a fourth edge point Q4 (x4, y4, z4) are selected from the initial edge point set. Q3 and Q4 are any two edge points in the initial edge point set. Based on the two-dimensional data of Q3 and Q4, a straight line L2 can be determined by linear fitting. This straight line L2 can be regarded as the second plane straight line. When the number of inner points of straight line L2 is greater than or equal to the second preset number, straight line L2 is determined to be the current target candidate line.

[0090] Repeat the following iterative process until the number of target candidate lines is greater than or equal to a third preset number:

[0091] The edge points remaining after the initial edge point set removes the inliers of the current target candidate line constitute the current edge point set. For example, the current edge point set includes the edge points remaining after the inliers of line L2 are removed from the initial edge point set. The fifth edge point Q5 (x5, y5, z5) and the sixth edge point Q6 (x6, y6, z6) are selected from the current edge point set. Q5 and Q6 are any two edge points of the current edge point set. A straight line L3 can be determined by performing straight line fitting based on the two-dimensional data of Q5 and Q6. This straight line L3 can be regarded as a third plane straight line. When the number of inliers of line L3 is greater than or equal to the fourth preset number, a new current target candidate line is determined based on line L3. The above cycle process is repeated until the number of target candidate lines is greater than or equal to the third preset number, and the target candidate line with the largest number of inliers is determined to be the target line L4. The above first preset number, second preset number, and third preset number can be the same or different.

[0092] In one implementation, to further improve accuracy and prevent interrupted lines from being identified as a single line, the target line L4 can be verified. Target line L4 includes multiple interior points, where the interior points that meet a first adjacency condition belong to the same point set. For example, the first adjacency condition requires that the distance between any two interior points is less than or equal to a third preset distance. The first, second, or third preset distances can be the same or different.

[0093] In this way, the inliers of the target line L4 can be classified according to the distance between them to obtain at least one inlier set, each of which includes at least two inliers. Take the inliers of the target line L4 as an example, including A1, A2, and A3. Assume that A1, A2, and A3 correspond to labels a1, a2, and a3, respectively, and each label corresponds to an independent category. Traverse A1, A2, and A3, and for each inlier, calculate the distance between it and the other inliers. If the distance is less than or equal to the third preset distance, the two points are considered to meet the first adjacent condition. Assume that among A1, A2, and A3, A1 and A2 are adjacent. Then A1 and A2 are merged into the same category, and A1 and A2 constitute an inlier set. Similarly, when the target line L4 has more inliers, repeat the above classification process until all inliers of the target line L4 are classified into appropriate categories. Through the above classification, the point cloud area where the target line L4 fitted in the previous step is located is detected for intervals. If the number of internal point sets after classification is one, the point cloud area where the target line L4 is located can be considered to have no gaps, and the detected target line L4 meets the accuracy requirements. If the number of internal point sets after classification is multiple, the point cloud area where the target line L4 is located can be considered to have gaps. The point cloud area can be divided into multiple sub-point cloud areas, and line detection is performed in different sub-point cloud areas to obtain multiple sub-target lines. A line that meets the preset conditions is selected from the multiple sub-target lines as the final target line. Figure 5 As shown, line fitting is performed based on the multiple inlier point sets 1-n (n is an integer greater than or equal to 2) obtained after classification to determine multiple fourth-plane straight lines 1-n. The multiple inlier point sets correspond one-to-one to the multiple fourth-plane straight lines. When there are multiple fourth-plane straight lines, multiple straight lines L1 are determined based on the multiple fourth-plane straight lines. Target line detection is then re-performed based on the multiple straight lines L1 to re-determine the multiple target straight lines 1-n (i.e., re-determine the multiple sub-target straight lines); this process can be referred to the process for determining target line L4 described above. A new target line is selected from the re-determined multiple target straight lines 1-n. The new target line includes the line with the longest length and / or the largest number of inliers among the re-determined multiple target straight lines. For example, if three inlier point sets are obtained after classification, each of these three inlier point sets can be fitted to determine a straight line L1, resulting in three straight lines L1. Based on these three straight lines L1, three target lines can be determined. These three target lines are compared, and the line with the largest number of inliers or the longest length is selected as the new target line L4.

[0094] In one implementation, the 3D point cloud data in any of the above embodiments can be filtered to remove unnecessary point cloud data and reduce point cloud density, thereby improving the efficiency and accuracy of target line recognition. This filtering process, for example, includes one or more of pass filtering, voxel filtering, and outlier filtering. The following describes the processes of these filtering processes.

[0095] (1) Through-filtering: Assuming that the three-dimensional position coordinates of each spatial point in the point cloud are represented as (x, y, z), the ranges of these three coordinate values ​​can be set separately. For example, the range of x is (-1, 2), the range of y is (-3, 3), and the range of z is (0.1, 2), with the unit being meters. The x, y, and z values ​​of each point in the point cloud are traversed, and the point clouds outside the range are eliminated. The above x, y, and z ranges include the coordinate value range of the photovoltaic bracket in the first coordinate system when the photovoltaic robot is working.

[0096] (2) Voxel filtering: In one implementation, the point cloud obtained by depth image conversion is relatively dense and requires a large amount of computation. Therefore, the point cloud can be downsampled by voxel filtering to reduce the amount of computation. For example, the length, width, and height of the voxel volume are set to the same value (e.g., 0.005m), and for each point P (x, y, z) in the point cloud, they are divided into voxels of a three-dimensional grid. The calculation process is as follows:

[0097]

[0098] Among them, i, j, k represent the index of the point in the three-dimensional grid, and the symbol For all points (P1, P2, ..., Pn) in each voxel, calculate the centroid C (V ijk ) as the representative point, the process is as follows:

[0099]

[0100] Among them, V ijk represents the voxel with index (i, j, k), n is the number of points in the voxel, (x m ,y m ,z m ) is the voxel V ijk The coordinates of the mth point in . Perform the above operation on all voxels, and finally get the point set C(V 000 ),C(V 001 ),...,C(V ijk ) constitutes the point cloud after voxel filtering.

[0101] (3) Outlier filtering: used to remove points that appear abnormal compared to surrounding points. For each point P in the point cloud i , calculate the average distance μ between it and all other points i and standard deviation σ i :

[0102]

[0103] Where k is P iThe number of neighbors, \lVert·\rVert refers to the Euclidean distance. By setting the threshold c, a range value [μ i -c·σ i ,μ i +c·σ i ], if it exceeds this range, point P i Remove points from the point cloud as outliers.

[0104] Based on the same technical concept, an embodiment of the present application also provides a positioning device for a photovoltaic robot. Figure 6 FIG. 1 shows a schematic diagram of a photovoltaic robot positioning device provided in an embodiment of the present application. Figure 6 As shown, the positioning device 600 includes a processor 610, which is used to call instructions stored in a memory 620. When the instructions are called by the processor 610, the processor 610 executes any one of the positioning methods in the above embodiments.

[0105] It should be noted that the above embodiments can be freely combined as needed. The above are only preferred implementations of the present application. It should be noted that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application, and these improvements and modifications should also be considered as the scope of protection of the present application.

Claims

1. A straight line detection method based on a photovoltaic robot, characterized in that: The photovoltaic robot is used to install a photovoltaic panel on a photovoltaic bracket. A sensing device is installed on the photovoltaic robot. The sensing device is used to collect three-dimensional point cloud data of the photovoltaic bracket during the installation of the photovoltaic panel. The three-dimensional point cloud data includes edge point cloud data. The edge point cloud data includes three-dimensional position coordinates of multiple edge points. The multiple edge points constitute a point cloud of the geometric shape boundary of the photovoltaic bracket. The straight line detection method includes: Selecting a first edge point and a second edge point from the plurality of edge points; determining a first plane straight line based on two-dimensional data of the first edge point and the second edge point; When the number of inner points of the first plane straight line is greater than or equal to a first preset number, determining the first plane straight line as a first candidate straight line; When there are multiple first candidate straight lines, the candidate straight line with the largest number of inliers among the multiple first candidate straight lines is determined as the initial target candidate straight line; The initial target candidate straight line is used as a target straight line; or, a cyclic straight line detection is performed based on the initial target candidate straight line to determine the target straight line; the target straight line includes a straight line where a target structure of the photovoltaic support is located; the target structure is used to locate the installation position of the photovoltaic panel on the photovoltaic support; When there is a gap in the point cloud area where the target line is located, the point cloud area is divided into multiple sub-point cloud areas, and line detection is performed in different sub-point cloud areas to obtain multiple sub-target lines; A straight line with the longest length and / or the largest number of inner points is selected from the plurality of sub-target straight lines as a new target straight line.

2. The straight line detection method according to claim 1, characterized in that: Also includes: Classifying the interior points of the target straight line according to the distances between the interior points of the target straight line to obtain at least one interior point set; If the number of internal point sets after classification is one, the point cloud region can be considered to have no gaps; If the number of internal point sets after classification is multiple, the point cloud region can be considered to have gaps.

3. The straight line detection method according to claim 2, characterized in that: Each of the interior point sets includes at least two interior points, and the interior points of the target straight line are classified according to the distances between the interior points of the target straight line to obtain at least one interior point set, including: Traversing the interior points of the target straight line and calculating the distance between each interior point and other interior points; Merging the interior points of the target straight line that meet the first adjacent condition into the same category; and forming an interior point set from the plurality of interior points merged into the same category; The first adjacent condition includes that the distance between any two inner points of the target straight line is less than or equal to a third preset distance.

4. The straight line detection method according to claim 1, wherein: The step of performing cyclic iteration based on the initial target candidate straight line to determine the target straight line includes: Selecting a third edge point and a fourth edge point from the initial edge point set; determining a second plane line based on the two-dimensional data of the third edge point and the fourth edge point; determining the second plane line as a current target candidate line when the number of inliers of the second plane line is greater than or equal to a second preset number; the initial edge point set includes the edge points remaining after removing the inliers of the initial target candidate line from the plurality of edge points; Repeat the following iterative process until the number of target candidate lines is greater than or equal to a third preset number: Selecting a fifth edge point and a sixth edge point from the current edge point set; determining a third plane line based on the two-dimensional data of the fifth edge point and the sixth edge point; determining a new current target candidate line based on the third plane line when the number of inliers of the third plane line is greater than or equal to a fourth preset number; the current edge point set includes the edge points remaining after removing the inliers of the current target candidate line from the initial edge point set; When the number of the target candidate lines is greater than or equal to a third preset number, the target candidate line with the largest number of inliers is determined as the target line.

5. The straight line detection method according to any one of claims 1 to 4, characterized in that: Also includes: Performing filtering processing on the three-dimensional point cloud data; The filtering process includes one or more of pass-through filtering, voxel filtering and outlier filtering.

6. The straight line detection method according to claim 1, wherein: Also includes: Converting the edge point cloud data into two-dimensional data of the plurality of edge points through two-dimensional processing; The two-dimensional processing includes setting the Z-axis coordinate values ​​of the multiple edge points to 1, or mapping the three-dimensional position coordinates of the multiple edge points to a two-dimensional plane by projection to obtain the two-dimensional position coordinates of the multiple edge points.

7. A photovoltaic robot positioning device, characterized in that: The method comprises a processor configured to call instructions stored in a memory, wherein when the instructions are called by the processor, the processor executes the straight line detection method according to any one of claims 1 to 6.

8. A photovoltaic robot positioning system, characterized in that: include: A sensing device, mounted on the photovoltaic robot, for collecting sensing data of the photovoltaic support, wherein the sensing data includes three-dimensional point cloud data, or the sensing data is used to determine the three-dimensional point cloud data; The positioning device according to claim 7, coupled to the sensing device.

Images