Automatic positioning method, system and computer-readable medium for mounting holes

Abstract

The present invention provides an automatic positioning method, system and computer-readable medium for mounting holes, which are used for a photovoltaic support. The method includes the following steps: obtaining three-dimensional point cloud of an installation environment through a lidar component; extracting planar point cloud of a photovoltaic module from the three-dimensional point cloud; determining whether the number of the planar point cloud of the photovoltaic module is less than a first threshold, and if so, extracting first lateral support point cloud of the photovoltaic support from the three-dimensional point cloud, and fitting a first lateral support line based on the first lateral support point cloud; searching for the rightmost point cloud from the first lateral support point cloud along the direction of the first lateral support line, and using the rightmost point cloud as a first reference positioning point; positioning the position of the mounting hole according to the first reference positioning point and the first lateral support line. The present invention strips three-dimensional point cloud data layer by layer through a point cloud processing algorithm, extracts features of each dimension of the support respectively to position the position of the mounting hole, has a wide range of applicable scenarios and high positioning accuracy.

Description

Technical Field

[0001] The present invention mainly relates to the field of image recognition and positioning, and particularly relates to an automatic positioning method, system and computer-readable medium for mounting holes. Background Art

[0002] In the construction process of centralized photovoltaic power stations, the current installation of photovoltaic modules is basically completely manual. Workers determine the positions of the mounting holes of the photovoltaic modules on the photovoltaic brackets, and then install the photovoltaic modules on the photovoltaic brackets. This requires a large amount of manpower, and the inconsistencies caused by manual installation may lead to a high probability of hidden crack risks for the photovoltaic modules.

[0003] Some solutions adopt a target recognition and positioning method based on point cloud for automatic positioning of mounting holes to achieve automatic installation of photovoltaic modules. The target recognition and positioning method based on point cloud includes the following two solutions: Solution one is to obtain images and point cloud data based on a depth camera, process the images through deep learning or traditional image recognition methods to obtain the position information of the target in the image, and then back-project the point cloud onto the image to correspondingly obtain the point cloud data of the target, so as to obtain the three-dimensional position information of the target. Solution two is to obtain point cloud data based on a lidar, and directly obtain the point cloud data of the target by segmenting the target from the point cloud through deep learning methods, so as to obtain the three-dimensional position information of the target. Both of the above two solutions have certain defects. For example, in solution one, since it obtains information through a depth camera, the detection distance of the depth camera is short, and the error caused by the adaptation of back-projecting the point cloud onto the image is large, making it difficult to control the measurement accuracy. In solution two, since it adopts a deep learning point cloud segmentation solution, the current accuracy of point cloud segmentation is insufficient, a large amount of data needs to be collected for training and the training period is long. When the shape of the target changes, deep learning needs to be retrained, and the adaptability is poor. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an automatic positioning method, system and computer-readable medium for mounting holes, so as to solve the problems of poor adaptability and low positioning accuracy of the existing automatic positioning method for mounting holes.

[0005] To solve the above technical problems, the present invention provides an automatic positioning method for installation holes for a photovoltaic support, which includes the following steps: obtaining a three-dimensional point cloud of the installation environment through a lidar component; extracting a planar point cloud of the photovoltaic module from the three-dimensional point cloud; determining whether the number of the planar point clouds of the photovoltaic module is less than a first threshold, and if so, extracting a first lateral support point cloud of the photovoltaic support from the three-dimensional point cloud, and fitting a first lateral support line based on the first lateral support point cloud; searching for the rightmost point cloud from the first lateral support point cloud along the direction of the first lateral support line, and using the rightmost point cloud as a first reference positioning point; positioning the position of the installation hole according to the first reference positioning point and the first lateral support line.

[0006] Optionally, the step of extracting a first lateral support point cloud of the photovoltaic support from the three-dimensional point cloud and fitting a first lateral support line based on the first lateral support point cloud includes: Step a: extracting a line from the three-dimensional point cloud through Random Sample Consensus (RANSAC); Step b: determining whether the number of the line point clouds corresponding to the line is greater than a second threshold and the direction of the line is lateral, and if so, using the line point cloud as the first lateral support point cloud and the line as the first lateral support line.

[0007] Optionally, the step of extracting a first lateral support point cloud of the photovoltaic support from the three-dimensional point cloud and fitting a first lateral support line based on the first lateral support point cloud further includes: Step c: removing the extracted first lateral support point cloud from the three-dimensional point cloud, and extracting the first lateral support line from the remaining point cloud; Step d: repeating Steps a-c N times, where N is a positive integer greater than or equal to 1; Step e: determining whether the number of the first lateral support lines is N, and if so, determining the position of the installation hole according to the N first lateral support lines and the N first reference positioning points.

[0008] Optionally, the rightmost point cloud is searched from the first lateral support point cloud by interval sampling.

[0009] Optionally, before extracting the planar point cloud of the photovoltaic module from the three-dimensional point cloud, it further includes: removing the point cloud in the three-dimensional point cloud that is more than a first distance away from the origin, where the origin is the optical center of the lidar component.

[0010] Optionally, the lidar component includes a lidar and a pitching pan-tilt head.

[0011] Optionally, the step of obtaining a three-dimensional point cloud of the installation environment through the lidar component includes: controlling the pitching pan-tilt head to drive the lidar to rotate, and splicing the two-dimensional point cloud collected by the lidar and the rotation angle of the pitching pan-tilt head to obtain the three-dimensional point cloud.

[0012] Optionally, the positioning method further includes: if the number of the photovoltaic module plane point clouds is greater than or equal to the first threshold, extracting the left longitudinal edge point cloud from the photovoltaic module plane point clouds, and fitting a left longitudinal edge line based on the left longitudinal edge point cloud; removing the extracted photovoltaic module plane point clouds from the three-dimensional point cloud, extracting the second transverse support point cloud of the photovoltaic support from the remaining point clouds, and fitting a second transverse support line based on the second transverse support point cloud; calculating an intersection point between the left longitudinal edge line and the second transverse support line, and using the intersection point as a second reference positioning point; and positioning the position of the mounting hole according to the second reference positioning point and the second transverse support line.

[0013] Optionally, the steps of extracting the second transverse support point cloud of the photovoltaic support from the remaining point clouds and fitting a second transverse support line based on the second transverse support point cloud include: Step A: extracting a line from the remaining point clouds by using Random Sample Consensus (RANSAC); Step B: determining whether the number of the line point clouds corresponding to the line is greater than a second threshold and the direction of the line is transverse. If so, using the line point clouds as the second transverse support point cloud and the line as the second transverse support line.

[0014] Optionally, the steps of extracting the second transverse support point cloud of the photovoltaic support from the remaining point clouds and fitting a second transverse support line based on the second transverse support point cloud further include: Step C: removing the extracted second transverse support point cloud from the remaining point clouds, and extracting the second transverse support line from the remaining point clouds; Step D: repeating Steps A - C for N times, where N is a positive integer greater than or equal to 1; Step E: determining whether the number of the second transverse support lines is N. If so, calculating the intersection points between the N second transverse support lines and the left longitudinal edge line respectively to obtain N second reference positioning points, and determining the position of the mounting hole according to the N second transverse support lines and the N second reference positioning points.

[0015] Optionally, extracting the photovoltaic module plane point cloud from the three-dimensional point cloud by using Random Sample Consensus (RANSAC).

[0016] To solve the above technical problems, the present invention provides an automatic positioning system for mounting holes, which is used for a photovoltaic support and includes: a lidar component for obtaining a three-dimensional point cloud of the installation environment; an identification and positioning module for extracting a planar point cloud of a photovoltaic module from the three-dimensional point cloud, determining whether the number of the planar point clouds of the photovoltaic module is less than a first threshold, and if so, extracting a first lateral support point cloud of the photovoltaic support from the three-dimensional point cloud, fitting a first lateral support line based on the first lateral support point cloud, searching for the rightmost point cloud from the first lateral support point cloud along the direction of the first lateral support line, using the rightmost point cloud as a first reference positioning point, and positioning the position of the mounting hole according to the first reference positioning point and the first lateral support line.

[0017] Optionally, the lidar component includes: a lidar for collecting a two-dimensional point cloud of the installation environment; a pitch pan-tilt for driving the lidar to rotate and splicing the two-dimensional point cloud and the rotation angle of the pitch pan-tilt to obtain the three-dimensional point cloud.

[0018] Optionally, the identification and positioning module is further configured to: if the number of the planar point clouds of the photovoltaic module is greater than or equal to the first threshold, extract a left longitudinal edge point cloud from the planar point cloud of the photovoltaic module, and fit a left longitudinal edge line based on the left longitudinal edge point cloud; remove the extracted planar point cloud of the photovoltaic module from the three-dimensional point cloud, extract a second lateral support point cloud of the photovoltaic support from the remaining point cloud, and fit a second lateral support line based on the second lateral support point cloud; calculate an intersection point between the left longitudinal edge line and the second lateral support line, and use the intersection point as a second reference positioning point; and position the position of the mounting hole according to the second reference positioning point and the second lateral support line.

[0019] To solve the above technical problems, the present invention provides a computer-readable medium storing computer program code, and the computer program code, when executed by a processor, implements the positioning method as described above.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] The automatic positioning method of the mounting holes of the present invention uses a point cloud processing algorithm to layer and strip three-dimensional point cloud data, extracts the plane features, straight line features, and positioning reference points of the bracket respectively, and automatically locates the positions of the mounting holes on the bracket according to the straight line features and positioning reference points. Compared with the deep learning solution, it can adapt to the transformed bracket structure as long as some parameters are adjusted without re-training the data. Compared with the depth camera solution, it can achieve higher positioning accuracy; by using the point cloud processing algorithm to layer and strip three-dimensional point cloud data, extracting the plane features, edge straight line features, and horizontal straight line features of the bracket respectively, and using the intersection point of the edge straight line and the horizontal straight line of the bracket as the reference positioning point, it is possible to continue to achieve automatic positioning of the mounting holes in the scenario where some photovoltaic modules have been installed, and the positioning accuracy of the mounting holes is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following provides a detailed description of the specific embodiments of the present invention with reference to the accompanying drawings, where:

[0023] Figure 1 is a schematic structural diagram of a photovoltaic bracket according to an embodiment of the present invention;

[0024] Figure 2 is a flowchart of the automatic positioning method of the mounting holes according to an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of obtaining the three-dimensional point cloud of the installation environment according to an embodiment of the present invention;

[0026] Figure 4 is Figure 2 a schematic flowchart of step a in

[0027] Figure 5 is Figure 2 a schematic flowchart of step b in

[0028] Figure 6 is a system block diagram of the automatic positioning system of the mounting holes according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] To more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the following drawings are only some examples or embodiments of the present application. For those of ordinary skill in the art, without creative efforts, the present application can also be applied to other similar scenarios based on these drawings. Unless obvious from the language context or otherwise stated, the same reference numerals in the figures represent the same structure or operation.

[0030] As shown in this application and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0031] Unless otherwise specifically stated, the relative arrangements of the components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of this application. At the same time, it should be understood that, for the sake of convenience of description, the dimensions of the various parts shown in the drawings are not drawn in actual proportional relationships. Technologies, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such technologies, methods, and devices should be regarded as part of the authorization specification. In all the examples shown and discussed here, any specific value should be construed as merely exemplary and not as a limitation. Therefore, other examples of the exemplary embodiments may have different values. It should be noted that: like reference numerals and letters denote like items in the following drawings, and thus, once an item is defined in one drawing, further discussion thereof is not required in subsequent drawings.

[0032] In the description of this application, it should be understood that the orientation or positional relationships indicated by orientation words such as "front, back, up, down, left, right", "lateral, vertical, perpendicular, horizontal", and "top, bottom" are usually based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing this application and simplifying the description. Without contrary explanation, these orientation words do not indicate and imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the scope of protection of this application; the orientation words "inside, outside" refer to the inside and outside relative to the contour of each component itself.

[0033] For ease of description, spatial relative terms, such as "above", "over", "on the upper surface", "upper", etc., may be used herein to describe the spatial positional relationship of one device or feature to other devices or features as shown in the figures. It should be understood that the spatial relative terms are intended to encompass different orientations in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figure is inverted, the device described as "above" or "over" other devices or structures will then be positioned "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both the orientations of "above" and "below". The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the corresponding explanations will be made for the spatial relative descriptions used herein.

[0034] In addition, it should be noted that the use of terms such as "first" and "second" to define components is only for the convenience of differentiating the corresponding components. Without further statement, the above terms have no special meaning, and thus should not be construed as limiting the scope of protection of this application. In addition, although the terms used in this application are selected from well-known and commonly used terms, some of the terms mentioned in the specification of this application may be selected by the applicant according to his or her judgment, and their detailed meanings are described in the relevant parts of the description herein. In addition, it is required to understand this application not only through the actual terms used, but also through the meaning implied by each term.

[0035] Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the operations before or after do not necessarily need to be executed precisely in sequence. On the contrary, various steps can be executed in reverse order or simultaneously. At the same time, other operations may be added to these processes, or one or several steps may be removed from these processes.

[0036] Figure 1 is a schematic structural diagram of a photovoltaic support according to an embodiment of the present invention. As Figure 1 shown, the photovoltaic support 100 includes a front column 11, a rear column 12, a transverse mounting channel steel 13, and a longitudinal support channel steel 14. The height of the front column 11 is lower than that of the rear column 12, and thus forms a certain angle with the ground. When automatically installing photovoltaic modules, first, the positions of the photovoltaic module mounting holes are located on the transverse mounting channel steel 13, and then the photovoltaic modules are grabbed by a robotic arm and positioned and installed according to the positions of the mounting holes obtained in the previous step. The photovoltaic modules are fixed to the photovoltaic support 100 through the mounting holes. Thus, locating the positions of the photovoltaic module mounting holes on the transverse mounting channel steel 13 is a necessary condition for automatic installation. As Figure 1As shown, in this embodiment, the photovoltaic support 100 includes four horizontally installed channel steels 13, and the position of an installation hole needs to be located on each horizontally installed channel steel 13. In other embodiments, the photovoltaic support may also include other numbers of horizontally installed channel steels 13, and this application does not limit this. The longitudinal support channel steel 14 is used to support the installed photovoltaic modules, and the number of longitudinal support channel steels 14 can be set as required, and this application does not limit this. Place the installed photovoltaic modules towards the low-latitude area and form a solar photovoltaic array in a series-parallel connection manner to achieve the purpose of solar photovoltaic power generation.

[0037] Figure 2 is a flowchart of the automatic positioning method of the installation hole in an embodiment of the present invention. This method is applicable to automatically positioning the position of the photovoltaic module installation hole on the photovoltaic support. As Figure 1 shown, the automatic positioning method 200 of the installation hole includes the following steps:

[0038] Step S21: Obtain the three-dimensional point cloud of the installation environment through the lidar component. The lidar component includes a lidar and a pitch gimbal. Figure 3 is a schematic diagram of obtaining the three-dimensional point cloud of the installation environment in an embodiment of the present invention. As Figure 3 shown, the lidar component includes a single-line lidar 31 and a pitch gimbal 32. The pitch gimbal 32 drives the single-line lidar 31 to rotate, and splices the two-dimensional point cloud collected by the single-line lidar 31 and the rotation angle of the pitch gimbal 32 to obtain the three-dimensional point cloud, realizing the three-dimensional reconstruction of the installation environment. Among them, the rotation angle of the pitch gimbal 32 is obtained through the closed-loop control of the pitch gimbal 32, that is, by inputting a control rotation signal and reading the actual rotation angle. In some embodiments, the scanning angle of the single-line lidar 31 is 190°, and the angular resolution is 0.1°. The scanning angle of the pitch gimbal 32 is 180°, and the angular resolution is 0.01°. This application does not limit the scanning angles and angular resolutions of the single-line lidar and the pitch gimbal.

[0039] Step S22: Extract the planar point cloud of the photovoltaic module from the three-dimensional point cloud. The photovoltaic module is composed of a series of solar cells arranged in different arrays, so the photovoltaic module is generally planar. Extracting the planar point cloud of the photovoltaic module from the three-dimensional point cloud lays a foundation for subsequent judgment of whether there is already a photovoltaic module on the photovoltaic support. In the present invention, the planar point cloud of the photovoltaic module is extracted from the three-dimensional point cloud by Random Sample Consensus (RANSAC). RANSAC is an iterative method for estimating the parameters of a mathematical model by using the observed data points. The parameters of the mathematical model can be a plane or a straight line, etc. Specifically, RANSAC divides the three-dimensional point cloud into "inliers" and "outliers". In a dataset containing "outliers", an iterative method is used to find the optimal plane model, and the planar point cloud corresponding to the optimal plane model is the planar point cloud of the photovoltaic module.

[0040] In some embodiments, before entering step S22, it further includes the step of removing the point cloud in the three-dimensional point cloud that is more than a first distance away from the origin. The point cloud that is more than the first distance away from the origin can be considered as invalid point cloud. Among them, the origin is the optical center of the lidar component, and the value of the first distance is related to the size of the photovoltaic support. For example, the value of the first distance can be set to 3 meters in the left and right directions, 3 meters in the forward direction, 3.5 meters in the upward direction, and 0.5 meters in the downward direction. Removing the invalid point cloud can reduce the interference in the environmental three-dimensional point cloud, which is beneficial to the subsequent extraction of the planar point cloud and the straight-line point cloud.

[0041] Step S23: Determine whether the number of the planar point cloud of the photovoltaic module is less than a first threshold. If so, enter step a. Among them, the first threshold is related to the size of the photovoltaic support. For example, the first threshold can be 20,000 points. Taking the first threshold of 20,000 points as an example, if the number of the planar point cloud of the photovoltaic module is less than 20,000 points, it means that there is no photovoltaic module in the installation environment, that is, no photovoltaic module has been installed on the photovoltaic support. Therefore, it is necessary to find the initial position on the horizontal installation channel steel where the photovoltaic module can be installed. Since in the actual installation scenario, the photovoltaic module is often installed from right to left, it is necessary to find the horizontal support straight line representing the horizontal installation channel steel and the rightmost endpoint on the horizontal support straight line as a reference point to determine the position of the photovoltaic module installation hole.

[0042] Figure 4 Yes Figure 2 Schematic diagram of the process of step a. As Figure 4 shown, step a includes:

[0043] Step a1: Extract the first lateral support point cloud of the photovoltaic support from the three-dimensional point cloud, and fit the first lateral support line based on the first lateral support point cloud. Extract a line from the three-dimensional point cloud by using the Random Sample Consensus (RANSAC). The extracted line has a certain direction, and the line point cloud forming the line also has a certain number. Determine whether the number of the line point cloud corresponding to the line is greater than a second threshold and the direction of the line is lateral. If so, take the line point cloud as the first lateral support point cloud and the line as the first lateral support line. If not, it indicates that the first lateral support line cannot be extracted, suggesting that the position of the photovoltaic support itself needs to be adjusted, and then notify the carrier to adjust its pose. The value of the second threshold is related to the size of the lateral installation channel steel. For example, the value of the second threshold can be 100 points. Determine whether the direction of the line is lateral can be determined by the angle between the line and the horizontal line or the slope of the line. If the angle or slope is less than the threshold, the line is considered to be lateral.

[0044] In some embodiments, the photovoltaic support includes multiple lateral installation channel steels. Correspondingly, the corresponding number of first lateral support lines should be extracted from the three-dimensional point cloud. Taking four lateral installation channel steels as an example, four first lateral support lines should be extracted. Specifically, first, extract the first lateral support point cloud of the photovoltaic support from the three-dimensional point cloud, and fit the first lateral support line based on the first lateral support point cloud. Then, remove the already extracted first lateral support point cloud from the three-dimensional point cloud, and continue to extract the first lateral support line from the remaining point cloud. Extract one first lateral support line each time, and repeat the extraction 4 times. Determine whether the number of the first lateral support lines is 4. If so, end the extraction of the first lateral support lines. If not, it indicates that the position of the photovoltaic support itself needs to be adjusted, and then notify the carrier to adjust its pose.

[0045] Step a2: Search for the rightmost point cloud from the first lateral support point cloud along the direction of the first lateral support line, and take the rightmost point cloud as the first reference positioning point. The rightmost point cloud can be searched from the first lateral support point cloud by interval sampling. For example, calculate the sampling interval at 10 cm intervals along the first lateral support direction, search whether there are more than 10 point clouds within a radius of 5 cm centered on the interval sampling points until no more than 10 point clouds can be searched. Then, calculate the sampling points in the reverse direction at 5 cm intervals, and then search whether there are more than 5 support point clouds within a radius of 2.5 cm centered on the interval sampling points until more than 5 point clouds are searched, and record the position as the rightmost point cloud. When there are multiple first lateral support lines, search for the rightmost point cloud from the first lateral support point cloud corresponding to each lateral support line. Continuing with the example of four first lateral support lines, four first reference positioning points are obtained.

[0046] Step a3: Locate the positions of the mounting holes on the first horizontal bracket line based on the first reference positioning point. Since the structural parameters (such as length, width, and height) of the photovoltaic module are known, on the first horizontal bracket line, a point at a certain position (e.g., 3 cm) away from the first reference positioning point is taken as the position of the mounting hole to achieve automatic positioning of the mounting hole.

[0047] Continue to return to Figure 2 As shown, in step S23, it is judged whether the number of planar point clouds of the photovoltaic module is less than the first threshold. If not, step b is entered. Taking the first threshold as 20,000 points as an example, if the number of planar point clouds of the photovoltaic module is greater than 20,000 points, it means that some photovoltaic modules have been installed on the photovoltaic bracket. Since the photovoltaic modules are often installed from right to left, it is necessary to find the intersection point of the left longitudinal edge line of the installed photovoltaic module plane and the horizontal bracket line of the photovoltaic bracket as the reference positioning point for the mounting hole. Figure 5 Yes Figure 2 is the flow schematic diagram of step b in Figure 5 As shown, step b includes:

[0048] Step b1: Extract the left longitudinal edge point cloud from the planar point cloud of the photovoltaic module, and fit the left longitudinal edge line based on the left longitudinal edge point cloud. Specifically, first extract the edge point cloud of the planar point cloud of the photovoltaic module, and then extract the left longitudinal edge line point cloud from the edge point cloud to fit the left longitudinal edge line.

[0049] Step b2: Remove the extracted planar point cloud of the photovoltaic module from the three-dimensional point cloud, extract the second horizontal bracket point cloud of the photovoltaic bracket from the remaining point cloud, and fit the second horizontal bracket line based on the second horizontal bracket point cloud. Remove the extracted planar point cloud of the photovoltaic module from the three-dimensional point cloud, and then extract the second horizontal bracket point cloud of the photovoltaic bracket to prevent the planar point cloud of the photovoltaic module from interfering with the extraction of the second horizontal bracket point cloud of the photovoltaic bracket. The steps of extracting the second horizontal bracket point cloud of the photovoltaic bracket from the remaining point cloud and fitting the second horizontal bracket line based on the second horizontal bracket point cloud include: extracting a line from the remaining point cloud, judging whether the number of line point clouds corresponding to the line is greater than the second threshold and the direction of the line is horizontal. If so, the line point cloud is used as the second horizontal bracket point cloud, and the line is used as the second horizontal bracket line. If not, it means that the second horizontal bracket line cannot be extracted, indicating that the position of the photovoltaic bracket itself needs to be adjusted, and the carrier is notified to adjust the pose. The value of the second threshold is related to the size of the horizontal installation channel steel. For example, the value of the second threshold can be 100 points. Judging whether the direction of the line is horizontal can be determined by the angle between the line and the horizontal line or the slope of the line. If the angle or slope is less than the threshold, the line is considered horizontal.

[0050] In some embodiments, the photovoltaic support includes multiple horizontally installed channel steels. Correspondingly, the corresponding number of second horizontal support lines should be extracted from the remaining point cloud. Taking four horizontally installed channel steels as an example, four second horizontal support lines should be extracted. Specifically, first, extract the second horizontal support point cloud of the photovoltaic support from the remaining point cloud, and fit the second horizontal support line based on the second horizontal support point cloud. Then, remove the extracted second horizontal support point cloud from the remaining point cloud, and continue to extract the second horizontal support line from the point cloud remaining after removing the second horizontal support point cloud. Extract one second horizontal support line each time, and repeat the extraction 4 times. Determine whether the number of second horizontal support lines is 4. If so, end the extraction of the second horizontal support line. If not, it indicates that the position of the photovoltaic support itself needs to be adjusted, and the carrier is notified to adjust its pose.

[0051] Step b3: Calculate the intersection point of the left longitudinal edge line and the second horizontal support line, and use the intersection point as the second reference positioning point. Since the left longitudinal edge line and the second horizontal support line in the fitted three-dimensional space may not have a direct intersection point, two lines can be sampled at intervals, and the Euclidean distance between the points on each line and the points on the other line can be traversed and calculated. The central point with the shortest distance between the two points is extracted as the approximate intersection point. When there are multiple second horizontal support lines, calculate the intersection point of each second horizontal support line and the left longitudinal edge line. Continuing with the example of four second horizontal support lines, four intersection points, that is, four second reference positioning points, are obtained.

[0052] Step b4: Locate the position of the mounting hole according to the second reference positioning point and the second horizontal support line. Since the structural parameters (such as length, width, and height) of the photovoltaic module are known, the position of the mounting hole of the photovoltaic module can be located through the second reference positioning point and the second horizontal support line. For example, a point at a certain distance (such as 3 cm) from the second reference positioning point can be selected on the second horizontal support line as the position of the mounting hole to achieve automatic positioning of the mounting hole.

[0053] The automatic positioning method of the mounting hole of the present invention hierarchically strips the three-dimensional point cloud data through a point cloud processing algorithm, extracts the plane features, line features, and positioning reference points of the support respectively, and automatically locates the position of the mounting hole on the support according to the line features and the positioning reference points. Compared with the deep learning solution, only some parameters need to be adjusted without re-training the data to adapt to the transformed support structure; compared with the depth camera solution, higher positioning accuracy can be achieved. In the scenario where some photovoltaic modules have been installed, by extracting the plane features, edge line features, and horizontal line features of the support respectively, and using the intersection point of the edge line and the horizontal line of the support as the reference positioning point, it is possible to continue installing photovoltaic modules near the installed part of the photovoltaic modules. The applicable scenario is wide, and the positioning accuracy of the mounting hole is higher.

[0054] Figure 6 It is a system block diagram of an automatic positioning system for mounting holes according to an embodiment of the present invention. As Figure 6 shown, the automatic positioning system 600 for mounting holes includes a lidar assembly 61 and an identification and positioning module 62. The lidar assembly 61 includes a lidar 611 and a pitch gimbal 612. The lidar 611 is used to collect two-dimensional point clouds of the installation environment. The pitch gimbal 612 is used to drive the lidar 611 to rotate and splice the two-dimensional point clouds and the rotation angle of the pitch gimbal 612 to obtain three-dimensional point clouds. The lidar 611 can be a single-line lidar.

[0055] The identification and positioning module 62 is used to extract the planar point cloud of the photovoltaic module from the three-dimensional point cloud, determine whether the number of the planar point clouds of the photovoltaic module is less than a first threshold. If so, it extracts the first horizontal support point cloud of the photovoltaic support from the three-dimensional point cloud, fits a first horizontal support line based on the first horizontal support point cloud, searches for the rightmost point cloud from the first horizontal support point cloud along the direction of the first horizontal support line, uses the rightmost point cloud as the first reference positioning point, and locates the position of the mounting hole according to the first reference positioning point and the first horizontal support line.

[0056] In some embodiments, the identification and positioning module 62 is further used to: if the number of the planar point clouds of the photovoltaic module is greater than or equal to the first threshold, extract the left longitudinal edge point cloud from the planar point cloud of the photovoltaic module and fit a left longitudinal edge line based on the left longitudinal edge point cloud; remove the extracted planar point cloud of the photovoltaic module from the three-dimensional point cloud, extract the second horizontal support point cloud of the photovoltaic support from the remaining point clouds, and fit a second horizontal support line based on the second horizontal support point cloud; calculate the intersection point of the left longitudinal edge line and the second horizontal support line, use the intersection point as the second reference positioning point; and locate the position of the mounting hole according to the second reference positioning point and the second horizontal support line.

[0057] The present invention further includes a computer-readable medium storing computer program code, and the computer program code implements the foregoing automatic positioning method for mounting holes when executed by a processor.

[0058] When the automatic positioning method for mounting holes is implemented as a computer program, it can also be stored in a computer-readable storage medium as an article of manufacture. For example, the computer-readable storage medium may include, but is not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs)), smart cards, and flash memory devices (e.g., electrically erasable programmable read-only memories (EPROMs), cards, sticks, key drives). In addition, the various storage media described herein can represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media (and / or storage media) that can store, contain, and / or carry code and / or instructions and / or data.

[0059] The basic concepts have been described above. Obviously, for those skilled in the art, the above invention disclosure is only an example and does not constitute a limitation to this application. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, so such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this application.

[0060] Meanwhile, this application uses specific terms to describe the embodiments of this application. Such as "one embodiment", "an embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that the "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application can be appropriately combined.

[0061] Some aspects of this application can be executed entirely by hardware, can be executed entirely by software (including firmware, resident software, microcode, etc.), or can be executed by a combination of hardware and software. The above hardware or software can all be referred to as "data block", "module", "engine", "unit", "component", or "system". The processor can be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or a combination thereof. In addition, aspects of this application may be embodied as a computer product located in one or more computer-readable media, and the product includes computer-readable program code. For example, the computer-readable medium may include, but is not limited to, magnetic storage devices (such as hard disks, floppy disks, magnetic tapes...), optical discs (such as compact discs CD, digital versatile discs DVD...), smart cards, and flash memory devices (such as cards, sticks, key drives...).

[0062] The computer-readable medium may contain a propagated data signal containing computer program code, such as on a baseband or as part of a carrier wave. This propagated signal may have various forms of representation, including electromagnetic form, optical form, etc., or a suitable combination of forms. The computer-readable medium can be any computer-readable medium other than a computer-readable storage medium, and this medium can be connected to an instruction execution system, device, or equipment to realize communication, propagation, or transmission for use of the program. The program code located on the computer-readable medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, radio frequency signal, or similar media, or any combination of the above media.

[0063] Similarly, it should be noted that, in order to simplify the description of the disclosure of the present application and thus help the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of the present application, sometimes multiple features are merged into one embodiment, drawing or description thereof. However, this method of disclosure does not mean that the features required by the subject matter of the present application are more than those mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiments disclosed above.

[0064] Although the present application has been described with reference to the current specific embodiments, those of ordinary skill in the art should recognize that the above embodiments are only used to illustrate the present application, and various equivalent changes or substitutions can be made without departing from the spirit of the present application. Therefore, as long as the changes and modifications of the above embodiments are within the scope of the spirit of the present application, they will fall within the scope of the claims of the present application.

Claims

1. An automatic positioning method for mounting holes, which is used for a photovoltaic support, characterized in that Comprising: Obtaining a three-dimensional point cloud of the installation environment through a lidar component; Extracting a photovoltaic module plane point cloud from the three-dimensional point cloud; Judging whether the number of the photovoltaic module plane point cloud is less than a first threshold. If so, extracting a first lateral support point cloud of a photovoltaic support from the three-dimensional point cloud, and fitting a first lateral support line based on the first lateral support point cloud; Searching for the rightmost point cloud from the first lateral support point cloud along the direction of the first lateral support line, and using the rightmost point cloud as a first reference positioning point; Positioning the position of the mounting hole according to the first reference positioning point and the first lateral support line; If the number of the photovoltaic module plane point cloud is greater than or equal to the first threshold, extracting a left longitudinal edge point cloud from the photovoltaic module plane point cloud, and fitting a left longitudinal edge line based on the left longitudinal edge point cloud; Removing the extracted photovoltaic module plane point cloud from the three-dimensional point cloud, extracting a second lateral support point cloud of the photovoltaic support from the remaining point cloud, and fitting a second lateral support line based on the second lateral support point cloud; Calculating the intersection point of the left longitudinal edge line and the second lateral support line, and using the intersection point as a second reference positioning point; Positioning the position of the mounting hole according to the second reference positioning point and the second lateral support line.

2. The method according to claim 1, wherein The steps of extracting a first lateral support point cloud of a photovoltaic support from the three-dimensional point cloud and fitting a first lateral support line based on the first lateral support point cloud include: Step a: Extracting a line from the three-dimensional point cloud by using Random Sample Consensus (RANSAC); Step b: Judging whether the number of the line point cloud corresponding to the line is greater than a second threshold and the direction of the line is lateral. If so, using the line point cloud as the first lateral support point cloud and the line as the first lateral support line.

3. The method according to claim 2, characterized in that, Also comprising: Step c: Removing the extracted first lateral support point cloud from the three-dimensional point cloud, and extracting the first lateral support line from the remaining point cloud; Step d: Repeating steps a-c N times, where N is a positive integer greater than or equal to 1; Step e: Judging whether the number of the first lateral support lines is N. If so, determining the position of the mounting hole according to the N first lateral support lines and the N first reference positioning points.

4. The method according to claim 1, characterized in that Searching for the rightmost point cloud from the first lateral support point cloud by interval sampling.

5. The method according to claim 1, wherein Before extracting the photovoltaic module plane point cloud from the three-dimensional point cloud, it also includes: removing the point cloud in the three-dimensional point cloud that is more than a first distance away from the origin, where the origin is the optical center of the lidar component.

6. The method according to claim 1, wherein The lidar component includes a lidar and a pitching pan-tilt head.

7. The method according to claim 6, characterized in that, The step of obtaining a three-dimensional point cloud of the installation environment through the lidar component includes: controlling the pitching pan-tilt head to drive the lidar to rotate, and splicing the two-dimensional point cloud collected by the lidar and the rotation angle of the pitching pan-tilt head to obtain the three-dimensional point cloud.

8. The method according to claim 1, wherein The steps of extracting a second lateral support point cloud of a photovoltaic support from the remaining point cloud and fitting a second lateral support line based on the second lateral support point cloud include: Step A: Extracting a line from the remaining point cloud by using Random Sample Consensus (RANSAC); Step B: Determine whether the number of the line point clouds corresponding to the line is greater than a second threshold and the direction of the line is horizontal. If so, use the line point cloud as the second horizontal support point cloud and use the line as the second horizontal support line.

9. The method according to claim 8, wherein It further includes: Step C: Remove the extracted second horizontal support point cloud from the remaining point clouds, and extract the second horizontal support line from the remaining point clouds. Step D: Repeat steps A - C for N times, where N is a positive integer greater than or equal to 1. Step E: Determine whether the number of the second horizontal support lines is N. If so, calculate the intersection points of the N second horizontal support lines and the left vertical edge line respectively to obtain N second reference positioning points, and determine the positions of the mounting holes according to the N second horizontal support lines and the N second reference positioning points.

10. The method according to any one of claims 1-9, characterized in that, Extract the photovoltaic module plane point cloud from the three - dimensional point cloud through Random Sample Consensus (RANSAC).

11. An automatic positioning system for mounting holes, used for a photovoltaic bracket, characterized in that It includes: A lidar assembly for acquiring the three - dimensional point cloud of the installation environment. An identification and positioning module for extracting the photovoltaic module plane point cloud from the three - dimensional point cloud, determining whether the number of the photovoltaic module plane point clouds is less than a first threshold. If so, extract the first horizontal support point cloud of the photovoltaic support from the three - dimensional point cloud, fit the first horizontal support line based on the first horizontal support point cloud, search for the right - most point cloud in the first horizontal support point cloud along the direction of the first horizontal support line, use the right - most point cloud as the first reference positioning point, and position the mounting hole according to the first reference positioning point and the first horizontal support line. If the number of the photovoltaic module plane point clouds is greater than or equal to the first threshold, extract the left vertical edge point cloud from the photovoltaic module plane point cloud, fit the left vertical edge line based on the left vertical edge point cloud, remove the extracted photovoltaic module plane point cloud from the three - dimensional point cloud, extract the second horizontal support point cloud of the photovoltaic support from the remaining point clouds, fit the second horizontal support line based on the second horizontal support point cloud, calculate the intersection point of the left vertical edge line and the second horizontal support line, use the intersection point as the second reference positioning point, and position the mounting hole according to the second reference positioning point and the second horizontal support line.

12. The system according to claim 11, wherein, The lidar assembly includes: A lidar for collecting the two - dimensional point cloud of the installation environment. A pitching pan - tilt for driving the lidar to rotate and splicing the two - dimensional point cloud and the rotation angle of the pitching pan - tilt to obtain the three - dimensional point cloud.

13. A computer - readable medium storing computer program code, where the computer program code, when executed by a processor, implements the method according to any one of claims 1 - 10.

Images