Linear detection method, positioning device and system based on photovoltaic robot

Abstract

The invention discloses a linear detection method based on a photovoltaic robot, and a positioning device and system of the photovoltaic robot. The method comprises the following steps: determining a first plane straight line according to two-dimensional data of a first edge point and a second edge point; when the number of the inner points of the first plane straight line is greater than or equal to a first preset number, determining the first plane straight line as a first candidate straight line; determining the candidate straight line with the largest number of inner points in the plurality of first candidate straight lines as an initial target candidate straight line; performing cyclic straight line detection based on the initial target candidate straight line, and determining a target straight line; when an interval exists in the point cloud area where the target straight line is located, dividing the point cloud area into a plurality of sub-point cloud areas, and performing straight line detection in different sub-point cloud areas to obtain a plurality of sub-target straight lines; and selecting a new target straight line from the plurality of sub-target straight lines. Therefore, the straight line where the target structure of the photovoltaic support is located can be accurately detected.

Description

[0001] This case is a divisional application based on the invention patent with application date of September 24, 2024, application number "202411328439.7", and name "Positioning method, device and system for photovoltaic robot" as the parent case. Technical Field

[0002] The present application relates to the field of photovoltaic technology, and in particular to a straight line detection method, positioning device and system based on a photovoltaic robot. Background Art

[0003] Photovoltaic panels can convert solar energy into electrical energy. Photovoltaic panels are installed on photovoltaic brackets, which are used to fix and support photovoltaic panels so that the photovoltaic panels can maximize the absorption and conversion of solar energy. If photovoltaic panels are automatically installed by photovoltaic robots, if the position of the photovoltaic robot relative to the photovoltaic bracket is abnormal during the installation process, it may cause the installation position of the photovoltaic panel to deviate, thereby affecting the efficiency of the photovoltaic robot's automatic installation. Even worse, the photovoltaic panel may collide with it and be damaged.

[0004] Therefore, it is very important to accurately position the photovoltaic robot. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a photovoltaic robot positioning method, device and system to facilitate accurate positioning of the photovoltaic robot.

[0006] Specifically, the technical solution of this application is as follows:

[0007] In a first aspect, a photovoltaic robot positioning method is provided, which is used for the photovoltaic robot to install a photovoltaic panel on a photovoltaic bracket. The positioning method is used to determine the position information of the photovoltaic robot during the installation of the photovoltaic panel. The positioning method includes:

[0008] Obtain three-dimensional point cloud data of the photovoltaic bracket; the three-dimensional point cloud data is obtained through the sensing device;

[0009] Based on the three-dimensional point cloud data, a target straight line is determined; the target straight line includes a straight line where a target structure of the photovoltaic bracket is located, and the target straight line is located in the plane where the photovoltaic bracket is located;

[0010] The position information is determined based on the first rotation matrix between the first coordinate system of the sensing device and the second coordinate system of the photovoltaic robot, as well as the target straight line; the position information is used to indicate the relative position relationship between the photovoltaic robot and the target structure.

[0011] In this way, the relative position relationship between the photovoltaic robot and the photovoltaic bracket is determined based on the three-dimensional point cloud data of the photovoltaic bracket, and the photovoltaic robot can be accurately positioned during the process of installing photovoltaic panels by the photovoltaic robot, so as to detect abnormal posture of the photovoltaic robot in time. In addition, the above positioning method determines the target straight line based on the three-dimensional point cloud data, where the target straight line is a plane straight line. In this way, the three-dimensional point cloud data is converted into a two-dimensional plane straight line. Through dimensionality reduction processing, the data processing process can be optimized to improve the positioning efficiency.

[0012] In one implementation, the above determines the target straight line based on the three-dimensional point cloud data, including: extracting edge point cloud data from the three-dimensional point cloud data; the three-dimensional point cloud data includes the three-dimensional position coordinates of multiple spatial points, and the edge point cloud data includes the three-dimensional position coordinates of multiple edge points; multiple edge points constitute the point cloud of the geometric shape boundary of the photovoltaic bracket; two-dimensionally processing the edge point cloud data to obtain two-dimensional data of multiple edge points; and determining the target straight line based on the two-dimensional data of multiple edge points.

[0013] In this way, the point cloud of the geometric shape boundary of the photovoltaic bracket is extracted from the point cloud of the photovoltaic bracket by the edge detection method, thereby obtaining edge point cloud data. Based on the edge point cloud data, the target straight line corresponding to the target structure is accurately identified.

[0014] In one implementation, determining the target straight line based on the two-dimensional data of multiple edge points includes:

[0015] Selecting a first edge point and a second edge point from a plurality of edge points; determining a first plane straight line according to two-dimensional data of the first edge point and the second edge point; and determining the first plane straight line as a first candidate straight line when the number of inner points of the first plane straight line is greater than or equal to a first preset number;

[0016] Traversing multiple edge points, when there are multiple first candidate straight lines, determining the candidate straight line with the largest number of inner points among the multiple first candidate straight lines as the initial target candidate straight line;

[0017] Select a third edge point and a fourth edge point from the initial edge point set; determine a second plane straight line according to the two-dimensional data of the third edge point and the fourth edge point; when the number of inner points of the second plane straight line is greater than or equal to a second preset number, determine the second plane straight line as the current target candidate straight line; the initial edge point set includes the edge points remaining after removing the inner points of the initial target candidate straight line from the plurality of edge points;

[0018] Repeat the following iterative process until the number of target candidate lines is greater than or equal to a third preset number:

[0019] Select a fifth edge point and a sixth edge point from the current edge point set; determine a third plane straight line based on the two-dimensional data of the fifth edge point and the sixth edge point; when the number of inner points of the third plane straight line is greater than or equal to a fourth preset number, update the current target candidate straight line based on the third plane straight line; the current edge point set includes the edge points remaining after the initial edge point set removes the inner points of the current target candidate straight line;

[0020] When the number of target candidate lines is greater than or equal to a third preset number, the target candidate line with the largest number of inliers is determined as the target line.

[0021] In this way, the initial target candidate line is iterated in a loop to determine the final target line. The accuracy of target line detection is improved through loop iteration.

[0022] In one implementation, the multiple inner points of the target straight line are divided into multiple inner point sets, and the inner points that meet the first adjacent condition among the multiple inner points of the target straight line belong to the same inner point set. The above positioning method also includes: performing straight line fitting based on the multiple inner point sets to determine multiple fourth plane straight lines; determining multiple first plane straight lines based on the straight line with the largest number of inner points among the multiple fourth plane straight lines, and re-determining multiple target straight lines; selecting a new target straight line from the re-determined multiple target straight lines. The new target straight line includes the straight line with the longest length and / or the largest number of inner points among the re-determined multiple target straight lines.

[0023] In one implementation, the above positioning method also includes: determining a first distance between multiple edge points and a first plane straight line based on two-dimensional data of multiple edge points; when the first distance is less than or equal to a first preset distance, determining an edge point corresponding to the first distance from the multiple edge points as an inner point of the first plane straight line.

[0024] In one implementation, a plurality of spatial points constitute a first point cloud region; the plurality of spatial points include the first point, the first point also belongs to the second point cloud region, the second point cloud region is less than or equal to the first point cloud region; and the spatial points in the second point cloud region satisfy the second adjacent condition; the second point cloud region also includes the second point. The above method of extracting edge point cloud data of a photovoltaic bracket from three-dimensional point cloud data includes:

[0025] Determine the curvature information of the first point according to the three-dimensional position coordinates of the second point cloud area; the curvature information includes the curvature value of the first point and / or the angle value between the normal of the first point and the second point; when the curvature information meets the preset curvature condition, the first point is used as an edge point.

[0026] In one implementation, the above determining the curvature information of the first point based on the three-dimensional position coordinates of the second point cloud area includes: performing surface fitting with the first point and the second point as center points, respectively, according to the three-dimensional position coordinates of the second point cloud area, to obtain the first surface and the second surface; determining the curvature value of the first point and the angle value between the normal of the first point and the normal of the second point according to the feature information of the first surface and the second surface.

[0027] In one implementation, it also includes: filtering the three-dimensional point cloud data; the filtering process includes one or more of through filtering, voxel filtering and outlier filtering.

[0028] In one implementation, the position information includes a first angle between the photovoltaic robot and the target structure. The above method determines the position information based on the first rotation matrix between the first coordinate system of the sensing device and the second coordinate system of the photovoltaic robot, and the target straight line, including: determining the second angle between the target straight line and the first direction; determining the second rotation matrix between the first coordinate system and the third coordinate system based on the second angle; the third coordinate system includes the coordinate system of the target structure; determining the third rotation matrix between the second coordinate system and the third coordinate system based on the first rotation matrix and the second rotation matrix; and determining the first angle based on the third rotation matrix.

[0029] In a second aspect, a photovoltaic robot positioning device is provided, comprising a processor for calling instructions stored in a memory, wherein when the instructions are called by the processor, the processor executes the positioning method implemented in the first aspect or any one of the first aspects.

[0030] In a third aspect, a photovoltaic robot positioning system is provided, comprising:

[0031] A sensing device is installed on the photovoltaic robot and is used to collect sensing data of the photovoltaic support, where the sensing data includes three-dimensional point cloud data, or the sensing data is used to determine the three-dimensional point cloud data;

[0032] The positioning device of the second aspect is coupled to a sensing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The following is a brief introduction to the drawings used in describing the embodiments of the present application:

[0034] Figure 1 is a schematic diagram of a photovoltaic panel installation scenario provided in an embodiment of the present application;

[0035] Figure 2 is a schematic structural diagram of a photovoltaic robot positioning system provided in an embodiment of the present application;

[0036] Figure 3 is a flow chart of a photovoltaic robot positioning method provided in an embodiment of the present application;

[0037] Figure 4 It is a flow chart of determining a target straight line based on three-dimensional point cloud data of a photovoltaic bracket provided in an embodiment of the present application;

[0038] Figure 5 is a schematic diagram of a target straight line determination process provided in an embodiment of the present application;

[0039] Figure 6 It is a structural schematic diagram of a photovoltaic robot positioning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the specific implementation methods of the present application will be described below with reference to the accompanying drawings. The accompanying drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other accompanying drawings and other implementation methods can be obtained based on these drawings without creative work. Adjustments and improvements made without departing from the concept of the present application are all within the scope of protection of the present application.

[0041] In order to simplify the drawings, the drawings in the embodiments of the present application only schematically show the parts related to the present application, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some drawings, only parts with the same structure or function are schematically shown, and there may be more or fewer parts with the same structure or function in reality.

[0042] In this application, unless otherwise clearly specified and limited, ordinal numbers, such as "first", "second", etc., are only used to distinguish and describe associated objects, and cannot be understood as indicating or implying the relative importance or order between associated objects; in addition, they do not represent the number of associated objects. "Multiple" includes two or more, and other quantifiers are similar. " / " is used to describe the relationship between associated objects, which indicates the "or" relationship between associated objects. "One or more" or "at least one" of multiple objects refers to any object or any combination of multiple objects, for example, "one or more of a1, a2, a3" or "at least one of a1, a2, a3" includes: "alone a1", "alone a2", "alone a3", "a1 and a2", "a1 and a3", "a2 and a3" or "a1, a2 and a3".

[0043] The terms "installation" and "connection" should be understood in a broad sense. For example, "installation" can be direct installation or installation through other components; "connection" can be fixed connection, detachable connection, or integral connection; it can be mechanical connection or electrical connection; it can be direct connection or indirect connection through an intermediate medium, or it can be internal communication between two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0044] In photovoltaic power generation scenarios, photovoltaic panels (also known as photovoltaic modules) are fixed and supported by photovoltaic brackets. Photovoltaic brackets enable photovoltaic panels to face the sun at the best angle to maximize the reception of solar energy and convert it into electrical energy. Photovoltaic panels are installed by photovoltaic robots. Furthermore, photovoltaic robots also include mobile devices to facilitate the carrying of photovoltaic panels to different locations for installation. Please refer to Figure 1 , which is a schematic diagram of a photovoltaic panel installation scenario provided in an embodiment of the present application. Figure 1 As shown, assuming Figure 1 The direction of the middle arrow is the traveling direction of the photovoltaic robot 110. The photovoltaic robot 110 moves along the traveling direction relative to the photovoltaic support 120, and installs the photovoltaic panel on the corresponding position of the photovoltaic support 120 during the traveling process, as shown by the shadow in the figure.

[0045] The photovoltaic robot has an initial posture. This initial posture maintains a certain relative position relationship with the photovoltaic bracket, such as being perpendicular to the ground, flush with the inclination angle of the photovoltaic bracket, etc. If the photovoltaic robot maintains this initial posture or is within the error range of this initial posture during the installation process, the photovoltaic panel can be accurately installed in the appropriate position. However, during the movement of the photovoltaic robot, the ground may be uneven or the control error of the photovoltaic robot may occur, causing the posture of the photovoltaic robot to change, such as Figure 1 As shown in the dashed box, the dashed box represents several possible situations in which the robot's posture changes. It is only an example and does not represent the actual posture change of the photovoltaic robot. This situation will cause the current posture of the photovoltaic robot to deviate from the initial posture, resulting in a deviation in the installation position of the photovoltaic panel, which will cause the photovoltaic panel to be unable to be accurately installed on the photovoltaic bracket. And as the photovoltaic robot continues to move forward, this deviation will continue to accumulate, and eventually manual adjustment will be required, increasing the operation and maintenance costs.

[0046] It can be seen that the change of the posture of the photovoltaic robot may affect the installation of the photovoltaic panel. Considering the above problems, the embodiment of the present application proposes a photovoltaic robot positioning method, device and other solutions, which can accurately position the photovoltaic robot, facilitate timely detection of abnormal posture of the photovoltaic robot, so that the posture of the photovoltaic robot can be controlled and adjusted in time, reduce the deviation in the photovoltaic panel installation process, and facilitate accurate installation of the photovoltaic panel in the appropriate position.

[0047] The following is a description with reference to the accompanying drawings.

[0048] Please refer to Figure 2 , which is a schematic diagram of the structure of a photovoltaic robot positioning system provided in an embodiment of the present application. Figure 2 As shown, the positioning system 200 includes a photovoltaic robot 210, a sensing device 220, and a positioning device 230. The sensing device 220 is used to collect sensing data of the photovoltaic bracket. The sensing data includes, for example, three-dimensional point cloud data; or the sensing data is used to determine the three-dimensional point cloud data, and the sensing data is, for example, a depth image. The sensing device 220 can be installed on the photovoltaic robot 210 to collect the sensing data of the photovoltaic bracket in real time during the installation of the photovoltaic panel. For example, the sensing device 220 can be installed on the robot's mechanical arm to obtain a better sensing field of view. In one implementation, the sensing device 220 can be installed at the end of the mechanical arm and maintain a preset vertical distance (for example, 1-2 meters) from the photovoltaic bracket, so as to collect the sensing data of the photovoltaic bracket from a bird's-eye view. The sensing device 220 can also be installed on the photovoltaic bracket, for example, the sensing device 220 is installed at a designated position of the photovoltaic bracket to collect the sensing data of the photovoltaic bracket under a fixed field of view. The embodiment of the present application does not limit the number and type of the sensing device 220. The positioning system 200 may include one or more sensing devices 220, and the sensing device 220 may include, for example, a depth camera or a laser radar. The above positioning device 230 is coupled to the sensing device 220 to determine the position information of the photovoltaic robot 210 during the installation of the photovoltaic panel. The positioning device 230 can also obtain the three-dimensional point cloud data of the photovoltaic bracket through the sensing device 220, for example, directly obtain the three-dimensional point cloud data collected by the sensing device 220, or obtain the three-dimensional point cloud data obtained after the depth image data is converted. The positioning device 230 determines the position information of the photovoltaic robot 210 based on the obtained three-dimensional point cloud data. In the embodiment of the present application, the photovoltaic bracket can be a tracking photovoltaic bracket such as a flat single-axis tracking bracket, an inclined single-axis tracking bracket, or a fixed photovoltaic bracket such as an inclination adjustable fixed bracket and a maximum inclination fixed bracket. The embodiment of the present application does not limit the type of photovoltaic bracket.

[0049] The positioning method of the photovoltaic robot is described below with reference to the accompanying drawings.

[0050] Please refer to Figure 3 , which shows a flow chart of a photovoltaic robot positioning method provided in some embodiments of the present application. The method is performed by the above positioning device. The positioning device can be installed on the photovoltaic robot; or it can be set independently of the photovoltaic robot, such as being located in a server or terminal device. The positioning method is used to determine the position information of the photovoltaic robot during the installation of photovoltaic panels. Figure 3 As shown, the positioning method at least includes the following steps:

[0051] S310: Acquire three-dimensional point cloud data of the photovoltaic bracket; the three-dimensional point cloud data is obtained by a sensing device;

[0052] S320: determining a target straight line based on the three-dimensional point cloud data; the target straight line includes a straight line where a target structure of the photovoltaic bracket is located, and the target straight line is located on a plane where the photovoltaic bracket is located;

[0053] S330: Determine the position information of the photovoltaic robot according to the first rotation matrix between the first coordinate system of the sensing device and the second coordinate system of the photovoltaic robot, and the target straight line; the position information is used to indicate the relative position relationship between the photovoltaic robot and the target structure.

[0054] The above positioning method obtains the three-dimensional point cloud data of the photovoltaic bracket based on the sensing device, and uses the three-dimensional point cloud data to determine the target straight line. The target straight line is the straight line where the target structure of the photovoltaic bracket is located. The target structure is used to locate the installation position of the photovoltaic panel on the photovoltaic bracket. For example, the rotation axis, beam, etc. of the photovoltaic bracket can be used as the target structure. The embodiment of the present application does not limit the selection of the target structure.

[0055] Therefore, the relative position relationship between the photovoltaic robot and the target structure is determined based on the three-dimensional point cloud data of the photovoltaic bracket, and the photovoltaic robot can be accurately positioned during the process of installing photovoltaic panels by the photovoltaic robot, so as to detect abnormal posture of the photovoltaic robot in time. In addition, the above positioning method determines the target straight line based on the three-dimensional point cloud data, where the target straight line is a plane straight line. In this way, the three-dimensional point cloud data is converted into a two-dimensional plane straight line. Through dimensionality reduction processing, the data processing process can be optimized to improve the positioning efficiency.

[0056] The position information of the photovoltaic robot includes the position information of one or more reference points on the photovoltaic robot. The reference point is, for example, any point on the photovoltaic robot. For example, the reference point may include the centroid or center of the photovoltaic robot. For example, the photovoltaic robot includes a robotic arm, which is used to grab and place the photovoltaic panel. In this case, the reference point is, for example, any point on the robotic arm, such as the centroid or center of the robotic arm, to further improve the positioning accuracy.

[0057] In one implementation, the above position information includes a first angle between the photovoltaic robot and the target structure. For example, the angle between a straight line where one or more reference points on the photovoltaic robot are located and the target straight line. Further, the above determination of the position information of the photovoltaic robot based on the first rotation matrix and the target straight line includes at least the following steps:

[0058] S331: Determine a second angle between the target straight line and the first direction;

[0059] S332: determining a second rotation matrix between the first coordinate system and a third coordinate system according to the second angle; the third coordinate system includes a coordinate system of the target structure;

[0060] S333: Determine a third rotation matrix between the second coordinate system and the third coordinate system according to the first rotation matrix and the second rotation matrix;

[0061] S334: Determine a first angle based on the third rotation matrix.

[0062] In the embodiment of the present application, the target straight line is located in the plane where the photovoltaic bracket is located, that is, the target straight line is a plane straight line. After determining the target straight line, its straight line equation y=ax+b can be obtained. The slope a of the target straight line represents the angle θ (i.e., the first angle) between it and the first direction. The first direction is, for example, the positive direction or negative direction of any coordinate axis of the photovoltaic array coordinate system. In one implementation, Figure 1 As shown, the photovoltaic array plane coordinate system is defined as the photovoltaic array width direction as the X-axis direction, the photovoltaic array direction as the Y-axis direction, and the vertical photovoltaic panel upward as the positive direction of the Z-axis. For example, the first direction can be Figure 1 The positive direction of the X axis.

[0063] The angle θ can be calculated by the inverse tangent function:

[0064] θ = arctan(a);

[0065] The angle value is converted into a rotation matrix, that is, the second rotation matrix R from the first coordinate system of the sensing device to the third coordinate system. The rotation matrix can be represented by the Euler angle of the rotation transformation. For the matrix rotating around the a-axis (perpendicular to the ground), it can be obtained by the following formula:

[0066]

[0067] Since the first rotation matrix R' from the second coordinate system of the photovoltaic robot to the first coordinate system of the sensing device can be obtained by pre-calibration (that is, R' is known), the third rotation matrix R between the second coordinate system and the third coordinate system can be determined by multiplying the above two rotation matrices. T :

[0068] R T =R·R';

[0069] Through the rotation matrix R T The above position information can be determined to indicate the relative position relationship between the photovoltaic robot and the target structure, for example, to determine the above first angle. In one implementation, the first angle can be determined by: determining the rotation matrix R T The eigenvalue λ of ; taking the eigenvalue λ as the independent variable, the first angle is determined by the arc cosine function.

[0070] In one implementation, the above position information includes the distance between the photovoltaic robot and the target structure. For example, the distance between a first reference point on the photovoltaic robot and a second reference point on the target structure. The first reference point is, for example, any point on the photovoltaic robot, and the second reference point is, for example, any point on the target structure. Based on the coordinates of the first reference point in the first coordinate system, the rotation matrix R T As well as the straight line equation of the target straight line where the target structure is located, the distance between the first reference point and the second reference point can be determined.

[0071] Furthermore, in one implementation, if the above position information is outside the preset error range, the position information can be corrected. For example, if the first angle exceeds the error range, the position of the photovoltaic robot can be controlled to adjust the angle. The preset error range is, for example, the error range of the initial position of the photovoltaic robot. By correcting the position of the photovoltaic robot, the error caused by the abnormal position of the photovoltaic robot during the installation of photovoltaic panels can be further reduced.

[0072] The process of determining the target straight line based on the three-dimensional point cloud data of the photovoltaic support in the above step S320 is described below.

[0073] A point cloud is a collection of multiple spatial points. A point cloud includes multiple discrete points, each of which can correspond to a spatial position, which can be represented by three-dimensional position coordinates (x, y, z). For example, in the coordinate system of a sensing device, the front of the sensing device is the positive direction of the X-axis, the left side of the sensing device is the positive direction of the Y-axis, and the Z-axis is perpendicular to the ground. For another example, in the coordinate system of a photovoltaic robot, the front of the photovoltaic robot is the positive direction of the X-axis, the left side of the photovoltaic robot is the positive direction of the Y-axis, and the Z-axis is perpendicular to the ground. The above description of three-dimensional position coordinates is for example only. The embodiments of the present application do not limit the type of coordinate system or the selection of coordinate axes, as long as the spatial position of a point in the point cloud can be indicated.

[0074] Please refer to Figure 4 , which is a flow chart of determining a target straight line based on three-dimensional point cloud data of a photovoltaic bracket provided in an embodiment of the present application. Figure 4 As shown, the above step S320 may include:

[0075] S321: extracting edge point cloud data from the three-dimensional point cloud data; the three-dimensional point cloud data includes the three-dimensional position coordinates of multiple spatial points, and the edge point cloud data includes the three-dimensional position coordinates of multiple edge points; the multiple edge points constitute a point cloud of the geometric shape boundary of the photovoltaic bracket;

[0076] S322: Perform two-dimensional processing on the edge point cloud data to obtain two-dimensional data of multiple edge points;

[0077] S323: Determine a target straight line based on the two-dimensional data of multiple edge points.

[0078] The data of a spatial point in the point cloud may include the three-dimensional position coordinates of the point, and the three-dimensional position coordinates of multiple spatial points constitute the three-dimensional point cloud data. The three-dimensional point cloud data of the photovoltaic bracket is the three-dimensional point cloud data collected by the sensing device under a sensing field of view. The sensing field of view includes the photovoltaic bracket and the environment in which the photovoltaic bracket is located. The three-dimensional point cloud data of the photovoltaic bracket (also called the original point cloud data) includes not only the point cloud data of the photovoltaic bracket, but also the point cloud data of the environment. Therefore, the point cloud of the geometric shape boundary of the photovoltaic bracket can be extracted from the point cloud of the photovoltaic bracket by the edge detection method, so as to obtain the edge point cloud data. The edge point cloud data includes the three-dimensional position coordinates of multiple edge points, and these edge points constitute the point cloud of the geometric shape boundary of the photovoltaic bracket. Further, the edge point cloud data is converted into the two-dimensional data of multiple edge points by two-dimensional processing. The target straight line can be identified based on the two-dimensional data of these edge points. For example, the two-dimensional processing sets the Z-axis coordinate value of the edge point to 1, or maps the three-dimensional position coordinates of the edge point to a two-dimensional plane by projection to obtain the two-dimensional position coordinates of the edge point, etc. In one implementation, the original point cloud data may be first converted into a two-dimensional plane image, and edge detection may be performed on the two-dimensional plane image to determine the target straight line.

[0079] In some implementations of the edge detection process (S321), edge points may be extracted using curvature information of the point cloud. For example, the curvature information of the first point may be determined based on the three-dimensional position coordinates of the second point cloud region.

[0080] The original point cloud data includes the three-dimensional position coordinates of multiple spatial points (original point clouds). The multiple spatial points constitute the first point cloud area. Among them, the multiple spatial points include point (first point), that is, the first point belongs to the first point cloud area. The first point also belongs to the second point cloud area, the second point cloud area is smaller than or equal to the first point cloud area, and the spatial points in the second point cloud area meet the preset adjacent condition (for the convenience of distinction, it is called the second adjacent condition). The second adjacent condition is, for example, the distance between any spatial point in the second point cloud area and the first point is less than or equal to the preset distance (for the convenience of distinction, it is called the second preset distance). That is, point and points in its neighborhood Constitute the second point cloud area.

[0081] In one implementation, the curvature information may include the point The curvature value can be determined by surface fitting. For example, according to the point and points in its neighborhood The surface is fitted by the least square method to obtain a first surface; based on the characteristic information of the first surface, the point The curvature value κ of (the first point) is:

[0082]

[0083] The characteristic information includes eigenvalues ​​λ0, λ1, and λ2 of the covariance matrix of the first surface.

[0084] In one implementation, the curvature information may include the point The normal and point The angle between the normal lines of the points. This angle can also be determined by surface fitting. and Point (Second point) as an example, point Belong to the second point cloud area. and Point As the center point, surface fitting is performed to obtain the first curved surface and the second curved surface. In this case, the feature information may include a normal vector. The normal vector of the first point can be obtained through the first curved surface, and the normal vector of the second point can be obtained through the second curved surface, so that the angle value between the two normal vectors can be determined, which is the angle between the normal line of the first point and the normal line of the second point.

[0085] When the above curvature information meets the preset curvature condition, the first point is taken as the edge point. The preset curvature condition can include any of the following: When the curvature value of is greater than or equal to the curvature threshold; The normal and point The angle between the normals of the points is greater than or equal to the angle threshold; or The curvature value of is greater than the curvature threshold, and the normal of point P→ is equal to that of point P→ i The angle value between the normal lines of is greater than the angle threshold.

[0086] For each spatial point in the original point cloud, the above edge detection process can be used to determine whether it is an edge point, so that the edge point cloud is extracted from the original point cloud. The edge point cloud includes multiple edge points, and the edge point cloud data includes the three-dimensional position coordinates of multiple edge points. Further, the edge point cloud data is converted into two-dimensional data of multiple edge points through two-dimensional processing. Based on the two-dimensional data of multiple edge points, the target straight line can be determined. The process of determining the target straight line is described below.

[0087] The plurality of edge points include a first edge point Q1 (x1, y1, z1) and a second edge point Q2 (x2, y2, z2). Q1 and Q2 are any two edge points in the edge point cloud. The values ​​of z1 and z2 in the three-dimensional position coordinates of Q1 and Q2 are set to 1 to obtain the two-dimensional data of Q1 and Q2, i.e., Q1 (x1, y1, 1) and Q2 (x2, y2, 1). A straight line L1 can be determined by linear fitting based on these two coordinates, and the straight line L1 can be regarded as a first plane straight line. Similarly, the two-dimensional data of other edge points in the plurality of edge points can also be obtained in the above manner. According to the two-dimensional data of the plurality of edge points, the inner points of the straight line L1 can be screened out from the plurality of edge points. In one implementation, according to the two-dimensional data of the plurality of edge points, the inner points of the straight line L1 can be determined from the plurality of edge points: according to the two-dimensional data of the plurality of edge points, the first distance between the plurality of edge points and the straight line L1 is determined; when the first distance is less than or equal to the first preset distance, the edge points corresponding to the first distance are determined from the plurality of edge points as the inner points of the first plane straight line. That is, the inner point of the straight line L1 is determined by the distance between each edge point and the straight line L1. That is, among the plurality of edge points, the edge point whose distance d from the straight line L1 is less than or equal to the first preset distance is the inner point of the straight line L1.

[0088] When the number of inner points of the line L1 is greater than or equal to the first preset number, the line L1 is determined to be the first candidate line. In one implementation, the number of iterations N can be set, multiple edge points can be traversed, Q1 and Q2 can be selected multiple times, and the above process can be repeated to determine the first candidate line. When there are multiple first candidate lines, the candidate line with the largest number of inner points among the multiple first candidate lines is determined as the initial target candidate line. In the initial implementation, the initial target candidate line can be used as the target line.

[0089] Alternatively, a loop iteration is performed based on the initial target candidate line to determine the final target line. The accuracy of target line detection is improved by loop iteration. Please refer to Figure 5 , which is a schematic diagram of a process of cyclic straight line detection provided in an embodiment of the present application. Figure 5 As shown, after removing the inner points of the initial target candidate straight line from the multiple edge points, the remaining edge points form the initial edge point set. The third edge point Q3 (x3, y3, z3) and the fourth edge point Q4 (x4, y4, z4) are selected from the initial edge point set. Q3 and Q4 are any two edge points of the initial edge point set. A straight line L2 can be determined by linear fitting based on the two-dimensional data of Q3 and Q4, and the straight line L2 can be regarded as the second plane straight line. When the number of inner points of the straight line L2 is greater than or equal to the second preset number, the straight line L2 is determined to be the current target candidate straight line.

[0090] Repeat the following iterative process until the number of target candidate lines is greater than or equal to a third preset number:

[0091] The edge points remaining after the initial edge point set removes the inner points of the current target candidate line constitute the current edge point set. For example, the current edge point set includes the edge points remaining after the inner points of line L2 are removed from the initial edge point set. The fifth edge point Q5 (x5, y5, z5) and the sixth edge point Q6 (x6, y6, z6) are selected from the current edge point set. Q5 and Q6 are any two edge points of the current edge point set. According to the two-dimensional data of Q5 and Q6, a straight line L3 can be determined by linear fitting, and the straight line L3 can be regarded as a third plane straight line. When the number of inner points of line L3 is greater than or equal to the fourth preset number, a new current target candidate line is determined based on line L3. Repeat the above cycle until the number of target candidate lines is greater than or equal to the third preset number, and determine that the target candidate line with the largest number of inner points is the target line L4. The above first preset number, second preset number, and third preset number can be the same or different.

[0092] In one implementation, in order to further improve the accuracy and prevent the interrupted straight line from being identified as a straight line, the target straight line L4 may be verified. The target straight line L4 includes a plurality of interior points, wherein the interior points satisfying the first adjacent condition belong to the same point set. The first adjacent condition is, for example, that the distance between any two interior points is less than or equal to a third preset distance. The first preset distance, the second preset distance, or the third preset distance may be the same or different.

[0093] In this way, these inner points can be classified according to the distance between the inner points of the target line L4 to obtain at least one inner point set, each of which includes at least two inner points. Take the inner points of the target line L4 as an example, including A1, A2, and A3. Assume that A1, A2, and A3 correspond to labels a1, a2, and a3, respectively, and each label corresponds to an independent category. Traverse A1, A2, and A3, and for each inner point, calculate the distance between it and other inner points. If the distance is less than or equal to the third preset distance, it is considered that the two points meet the first adjacent condition. Assume that among A1, A2, and A3, A1 and A2 are adjacent. Then A1 and A2 are merged into the same category, and A1 and A2 constitute an inner point set. Similarly, when the target line L4 has more inner points, repeat the above classification process until all the inner points of the target line L4 are classified into appropriate categories. Through the above classification, the point cloud area where the target line L4 fitted in the previous step is located is interval detected. If the number of internal point sets after classification is one, the point cloud region where the target line L4 is located can be considered to have no gaps, and the detected target line L4 meets the accuracy requirements. If the number of internal point sets after classification is multiple, the point cloud region where the target line L4 is located can be considered to have gaps. The point cloud region can be divided into multiple sub-point cloud regions, and line detection is performed in different sub-point cloud regions to obtain multiple sub-target lines. A line that meets the preset conditions is selected from the multiple sub-target lines as the final target line. Figure 5 As shown, based on the multiple inlier point sets 1-n (n is an integer greater than or equal to 2) obtained after classification, straight line fitting is performed to determine multiple fourth plane straight lines 1-n. The multiple inlier point sets correspond to the multiple fourth plane straight lines one by one: when the number of the fourth plane straight lines includes multiple, multiple straight lines L1 are determined based on the multiple fourth plane straight lines. The target straight line detection is re-performed based on the multiple straight lines L1, and the multiple target straight lines 1-n are re-determined (that is, the multiple sub-target straight lines are re-determined); the process can refer to the determination process of the above-mentioned target straight line L4. A new target straight line is selected from the multiple re-determined target straight lines 1-n. The new target straight line includes the straight line with the longest length and / or the largest number of inliers among the multiple re-determined target straight lines. For example, after classification, 3 inlier point sets are obtained, and these 3 inlier point sets can respectively fit and determine a straight line L1, thereby obtaining 3 straight lines L1. Based on these 3 straight lines L1, 3 target straight lines can be determined. Compare these 3 target straight lines, and select the straight line with the largest number of inliers or the longest length as the new target straight line L4.

[0094] In one implementation, the three-dimensional point cloud data in any of the above embodiments may be filtered to filter out unnecessary point cloud data and reduce the point cloud density, thereby improving the recognition efficiency and accuracy of the target line. The filtering process, for example, includes one or more of straight-through filtering, voxel filtering, and outlier filtering. The above filtering processes are described below.

[0095] (1) Pass-through filtering: Assuming that the three-dimensional position coordinates of each spatial point in the point cloud are expressed as (x, y, z), the ranges of these three coordinate values ​​can be set separately. For example, the range of x is (-1, 2), the range of y is (-3, 3), and the range of z is (0.1, 2), and the unit is meter. Traverse the x, y, and z values ​​of each point in the point cloud and remove the point clouds outside the range. The above x, y, and z ranges include the coordinate value range of the photovoltaic bracket in the first coordinate system when the photovoltaic robot is working.

[0096] (2) Voxel filtering: In one implementation, the point cloud obtained by depth image conversion is relatively dense and requires a lot of computation. Therefore, the point cloud can be downsampled by voxel filtering to reduce the amount of computation. For example, the length, width, and height of the voxel volume are set to the same value (e.g., 0.005m), and for each point P (x, y, z) in the point cloud, they are divided into voxels of a three-dimensional grid. The calculation process is as follows:

[0097]

[0098] Among them, i, j, and k represent the index of the point in the three-dimensional grid, and the symbol For all points (P1, P2, ..., Pn) in each voxel, the centroid C (V ijk ) as the representative point, the process is as follows:

[0099]

[0100] Among them, V ijk represents the voxel with index (i, j, k), n is the number of points in the voxel, (x m ,y m ,z m ) is the voxel V ijk The coordinates of the mth point in . The above operation is performed on all voxels, and the final point set C(V 000 ),C(V 001 ),...,C(V ijk ) constitutes the point cloud after voxel filtering.

[0101] (3) Outlier filtering: used to remove points that appear abnormal compared to surrounding points. For each point P in the point cloud i , calculate the average distance μ between it and all other points i and standard deviation σ i :

[0102]

[0103] Where k is P iThe number of neighbors, \lVert·\rVert refers to the Euclidean distance. By setting the threshold c, a range value [μ i -c·σ i ,μ i +c·σ i ], if it exceeds this range, point P i Remove points from the point cloud as outliers.

[0104] Based on the same technical concept, an embodiment of the present application also provides a positioning device for a photovoltaic robot. Figure 6 FIG. 1 is a schematic diagram showing a photovoltaic robot positioning device provided in an embodiment of the present application. Figure 6 As shown, the positioning device 600 includes a processor 610, which is used to call instructions stored in a memory 620. When the instructions are called by the processor 610, the processor 610 executes any one of the positioning methods in the above embodiments.

[0105] It should be noted that the above embodiments can be freely combined as needed. The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application, and these improvements and modifications should also be regarded as the protection scope of the present application.

Claims

1. A straight line detection method based on a photovoltaic robot, characterized in that: The photovoltaic robot is used to install the photovoltaic panel on the photovoltaic bracket, and the photovoltaic robot is equipped with a sensing device, and the sensing device is used to collect three-dimensional point cloud data of the photovoltaic bracket during the installation of the photovoltaic panel; the three-dimensional point cloud data includes edge point cloud data, and the edge point cloud data includes three-dimensional position coordinates of multiple edge points, and the multiple edge points constitute a point cloud of the geometric shape boundary of the photovoltaic bracket. The straight line detection method includes: Selecting a first edge point and a second edge point from the plurality of edge points; determining a first plane straight line according to two-dimensional data of the first edge point and the second edge point; When the number of inner points of the first plane straight line is greater than or equal to a first preset number, determining the first plane straight line as a first candidate straight line; When there are multiple first candidate straight lines, the candidate straight line with the largest number of inliers among the multiple first candidate straight lines is determined as the initial target candidate straight line; The initial target candidate straight line is used as the target straight line; or, a cyclic straight line detection is performed based on the initial target candidate straight line to determine the target straight line; the target straight line includes a straight line where a target structure of the photovoltaic bracket is located; the target structure is used to locate the installation position of the photovoltaic panel on the photovoltaic bracket; When there is a gap in the point cloud area where the target straight line is located, the point cloud area is divided into a plurality of sub-point cloud areas, and straight line detection is performed in different sub-point cloud areas respectively to obtain a plurality of sub-target straight lines; A straight line with the longest length and / or the largest number of inner points is selected from the plurality of sub-target straight lines as a new target straight line.

2. The straight line detection method according to claim 1, characterized in that: Also includes: Classifying the interior points of the target straight line according to the distances between the interior points of the target straight line to obtain at least one interior point set; If the number of internal point sets after classification is one, the point cloud region can be regarded as having no interval; If the number of internal point sets after classification is multiple, the point cloud region can be considered to have intervals.

3. The straight line detection method according to claim 2, characterized in that: Each of the interior point sets includes at least two interior points, and the interior points of the target straight line are classified according to the distances between the interior points of the target straight line to obtain at least one interior point set, including: Traversing the interior points of the target straight line, and calculating the distance between each interior point and other interior points; Merging the interior points of the target straight line that satisfy the first adjacent condition into the same category; and forming an interior point set by merging multiple interior points into the same category; The first adjacent condition includes that the distance between any two inner points of the target straight line is less than or equal to a third preset distance.

4. The straight line detection method according to claim 1, characterized in that: The step of performing cyclic iteration based on the initial target candidate straight line to determine the target straight line includes: Select a third edge point and a fourth edge point from the initial edge point set; determine a second plane straight line according to the two-dimensional data of the third edge point and the fourth edge point; when the number of inner points of the second plane straight line is greater than or equal to a second preset number, determine the second plane straight line as the current target candidate straight line; the initial edge point set includes the edge points remaining after the inner points of the initial target candidate straight line are removed from the plurality of edge points; Repeat the following iterative process until the number of target candidate lines is greater than or equal to a third preset number: Select a fifth edge point and a sixth edge point from the current edge point set; determine a third plane straight line according to the two-dimensional data of the fifth edge point and the sixth edge point; when the number of inner points of the third plane straight line is greater than or equal to a fourth preset number, determine a new current target candidate straight line based on the third plane straight line; the current edge point set includes the edge points remaining after the initial edge point set removes the inner points of the current target candidate straight line; When the number of the target candidate lines is greater than or equal to a third preset number, the target candidate line with the largest number of inliers is determined as the target line.

5. The straight line detection method according to any one of claims 1 to 4, characterized in that: Also includes: Performing filtering processing on the three-dimensional point cloud data; The filtering process includes one or more of pass-through filtering, voxel volume filtering and outlier filtering.

6. The straight line detection method according to claim 1, characterized in that: Also includes: Converting the edge point cloud data into two-dimensional data of the plurality of edge points through two-dimensional processing; The two-dimensional processing includes setting the Z-axis coordinate values ​​of the multiple edge points to 1, or mapping the three-dimensional position coordinates of the multiple edge points to a two-dimensional plane by projection to obtain the two-dimensional position coordinates of the multiple edge points.

7. A photovoltaic robot positioning device, characterized in that: The method comprises a processor, which is used to call instructions stored in a memory, and when the instructions are called by the processor, the processor executes the straight line detection method according to any one of claims 1 to 6.

8. A photovoltaic robot positioning system, characterized in that: include: A sensing device, installed on the photovoltaic robot, for collecting sensing data of the photovoltaic support, wherein the sensing data includes three-dimensional point cloud data, or the sensing data is used to determine the three-dimensional point cloud data; The positioning device as described in claim 7 is coupled to the sensing device.

Images