Automatic installation method, system and robot for photovoltaic module

Abstract

The invention relates to the field of photovoltaic module automatic installation, and discloses a photovoltaic module automatic installation method and system and a robot, and the method comprises the steps: shooting a target region containing a to-be-installed photovoltaic module and an installed photovoltaic module through a depth camera; acquiring image data and point cloud data of the target area; processing the image data based on an artificial intelligence model; identifying the to-be-installed photovoltaic module so as to adjust the initial pose of the to-be-installed photovoltaic module to be within a preset range; performing plane fitting on the point cloud data; obtaining a first plane corresponding to the photovoltaic module to be installed and a second plane corresponding to the installed photovoltaic module; calculating a relative deflection angle between the first plane and the second plane; and according to the relative deflection angle, the relative pose relation between the photovoltaic module to be installed and the installed photovoltaic module is adjusted, so that the installation meets the requirement. According to the invention, the angle control precision of the installation system can be improved, the installation efficiency of the photovoltaic module is improved, and the potential risk of manpower resource consumption and rework cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of photovoltaic installation technology, and in particular to a method, system and robot for automatic installation of photovoltaic components. Background Art

[0002] During the automatic installation of photovoltaic modules, the photovoltaic modules to be installed need to be kept relatively parallel to the installed photovoltaic modules, and the edges need to be aligned to ensure smooth installation. However, the installation of photovoltaic modules faces the challenge of undulating terrain. When the photovoltaic modules are pressed down, there is a tendency for the angle of placement to deviate from the installed photovoltaic modules, which may lead to installation failure and rework. Summary of the invention

[0003] In order to solve the technical problem that the automatic installation of photovoltaic modules is prone to angle deviation, the present application provides a method, system and robot for automatic installation of photovoltaic modules, which, combined with the processing of point cloud data, simulates the plane of the photovoltaic modules to be installed and the plane of the installed photovoltaic modules, can improve the accuracy of the angle control of the installation system, improve the efficiency of photovoltaic module installation, and reduce the potential risk of consuming human resources and rework costs. Specifically, the technical solution of the present application is as follows:

[0004] In a first aspect, the present application discloses a method for automatically installing a photovoltaic module, comprising the following steps:

[0005] Using a depth camera to photograph a target area including photovoltaic modules to be installed and installed photovoltaic modules; obtaining RGB image data and point cloud data of the target area;

[0006] Processing RGB image data based on an artificial intelligence model; identifying a photovoltaic module to be installed so as to adjust its initial posture relative to the installed photovoltaic module within a preset range;

[0007] Performing plane fitting on the point cloud data; obtaining a first plane corresponding to the photovoltaic assembly to be installed and a second plane corresponding to the installed photovoltaic assembly;

[0008] The relative deflection angle between the first plane and the second plane is calculated; and then the relative posture relationship between the photovoltaic assembly to be installed and the installed photovoltaic assembly is adjusted according to the relative deflection angle, so that the installation of the photovoltaic assembly to be installed meets the requirements.

[0009] In some implementations, before performing plane fitting on the point cloud data, the method further includes preprocessing the point cloud data;

[0010] Specifically, it includes: using statistical filtering to remove outliers around PV panels;

[0011] Clustering is performed on the filtered point cloud data.

[0012] In some embodiments, the method of removing outliers around a photovoltaic module using statistical filtering comprises the following steps:

[0013] For any point cloud in the point cloud data, count several neighboring point clouds within a preset radius around it;

[0014] Calculating a first distance between all the point clouds and each of the neighboring point clouds, and calculating a mean and a standard deviation of the first distance;

[0015] Based on the mean and standard deviation, point clouds whose distance from the neighborhood point clouds is greater than a first preset distance are identified as outliers, and outliers in the several neighborhood point clouds are removed; the first preset distance is the sum of the mean and several times the standard deviation.

[0016] In some implementations, clustering the filtered point cloud data includes the following steps:

[0017] Set clustering parameters; set the second clustering distance based on the initial posture of the photovoltaic module to be installed; set the point cloud quantity threshold of clustering based on the number of single-frame point clouds in the target area;

[0018] Extend the maximum clustering distance outward from any point cloud as the center, and find all neighboring points within the second clustering distance; all neighboring points corresponding to the point cloud are grouped into one cluster;

[0019] Traversing all point clouds in the target area to obtain clusters corresponding to all point clouds;

[0020] All the clusters are sorted according to the number of point clouds they contain, and the first two clusters with the largest number of point clouds are taken as the fitted plane point cloud clusters of the photovoltaic components to be installed and the installed photovoltaic components.

[0021] In some embodiments, the plane fitting of the point cloud data comprises the following steps:

[0022] Randomly selecting a number of point clouds from the point cloud data corresponding to the target area to generate a point cloud sample set;

[0023] The reference plane model is obtained by fitting the point cloud sample set;

[0024] Calculating a third distance between all point clouds in the point cloud data and the reference plane model; if the third distance is less than a second preset distance, considering the corresponding point cloud as an inner point of the reference plane model;

[0025] Counting the number of interior points of the reference plane model;

[0026] If the number of internal points is greater than the preset number, the reference plane model is considered to be a valid plane model; otherwise, the above steps are repeated until a valid plane model of the target area is obtained.

[0027] In some implementations, after performing plane fitting on the point cloud data, the following steps are further included:

[0028] Calculate the plane parameters corresponding to the first plane and the second plane respectively;

[0029] Plane parameters, including plane normal vector and center point coordinates;

[0030] The plane normal vector is used to calculate the relative deflection angle between the first plane and the second plane; the center point coordinates are used to calculate the depth difference between the first plane and the second plane.

[0031] In some embodiments, the step of calculating the relative deflection angle between the first plane and the second plane comprises the following steps:

[0032] Based on the plane normal vector, construct a first quaternion corresponding to the first plane and a second quaternion corresponding to the second plane;

[0033] Calculate the product of the first quaternion and the second quaternion to obtain a rotation matrix between the first plane and the second plane;

[0034] The relative deflection angle, including the pitch angle and the roll angle, is extracted from the rotation matrix, that is, the adjustment angle of the first plane relative to the second plane.

[0035] In a second aspect, the present application further discloses a photovoltaic module automatic installation system, the system is used to execute the photovoltaic module automatic installation method in any of the above embodiments; comprising:

[0036] A depth camera is used to photograph a target area including photovoltaic modules to be installed and installed photovoltaic modules; and obtain RGB image data and point cloud data of the target area;

[0037] The processor is used to perform the following steps: process RGB image data based on an artificial intelligence model; identify the photovoltaic assembly to be installed so as to adjust its initial posture to be within a preset range relative to the installed photovoltaic assembly; perform plane fitting on the point cloud data to obtain a first plane corresponding to the photovoltaic assembly to be installed and a second plane corresponding to the installed photovoltaic assembly; and calculate the relative deflection angle between the first plane and the second plane.

[0038] In some embodiments, the photovoltaic assembly automatic installation system further includes: a controller;

[0039] The controller is used to receive instructions from the processor and control the mechanical arm to adjust the relative posture relationship between the photovoltaic assembly to be installed and the installed photovoltaic assembly, so that the installation of the photovoltaic assembly to be installed meets the requirements.

[0040] In a third aspect, the present application further discloses an automatic installation robot, which includes the photovoltaic component automatic installation system in any one of the above-mentioned embodiments.

[0041] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0042] 1. The technical solution of the present application can realize automatic installation of photovoltaic components without human intervention; first, the image data is analyzed based on the artificial intelligence model to control the initial posture of the component to be installed within a rough range; then, combined with the processing of the point cloud data, the plane of the photovoltaic component to be installed and the plane of the installed photovoltaic component are simulated to calculate the relative deflection angle for fine installation; the present application has high automatic installation accuracy, which can improve the accuracy of the angle control of the installation system, improve the efficiency of photovoltaic component installation, and reduce the potential risk of wasting human resources and rework costs.

[0043] 2. This application has a simple structure, low cost, and a wide range of applications. All the data required for automatic installation, including RGB image data and point cloud data, can be collected through only one depth camera.

[0044] 3. This application fits the plane of the photovoltaic module based on filtering, clustering, screening and other processing of point cloud data; the fitting accuracy is high, and the plane model obtained by fitting is robust. When the light intensity changes greatly, it can still accurately fit the plane where the photovoltaic module is located. After obtaining the fitted plane, the relative deflection angle and depth difference between the photovoltaic module to be installed and the installed photovoltaic module are calculated and output to the automatic installation robot. The position of the component to be installed is adjusted by the mechanical arm so that the photovoltaic module to be installed is relatively parallel to the installed photovoltaic module, which is convenient for subsequent straight line detection, thereby achieving the purpose of improving accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The preferred implementation scheme will be described below in a clear and understandable manner with reference to the accompanying drawings to further illustrate the above-mentioned characteristics, technical features, advantages and implementation methods of the present application.

[0046] Figure 1 A flowchart of an embodiment of a method for automatically installing a photovoltaic module is disclosed in this application;

[0047] Figure 2 This is a schematic diagram of point cloud data initially obtained by a depth camera in an embodiment of the present application;

[0048] Figure 3This is a schematic diagram of point cloud data after preprocessing such as outlier filtering and clustering in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of a coordinate system corresponding to the first plane and the second plane in an embodiment of the present application;

[0050] Figure 5 This is a schematic diagram of the height difference between the first plane and the second plane in the embodiment of the present application;

[0051] Figure 6 The present application discloses a structural block diagram of an embodiment of an automatic photovoltaic assembly installation system. DETAILED DESCRIPTION

[0052] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0053] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections.

[0054] In order to simplify the drawings, only the parts related to the invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".

[0055] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0056] In this document, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0057] In a specific implementation, the terminal device described in the embodiments of the present application includes, but is not limited to, other portable devices such as mobile phones, laptop computers, tutoring machines, or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or touch pads). It should also be understood that in some embodiments, the terminal device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or touch pads).

[0058] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0059] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the specific implementation methods of the present application will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0060] With the gradual development of industrial technology, automatic installation robots have gradually replaced manual installation and become the main force in the installation of large outdoor equipment. The automatic installation robot for photovoltaic modules is designed to reduce the heavy labor intensity of on-site photovoltaic module installation. The goal is to accurately install and ensure that the photovoltaic modules are firmly fixed on the bracket to prevent movement or falling due to wind or other environmental factors. Automatic installation technology can reduce human errors, improve installation efficiency, and improve the consistency and accuracy of installation. Through mechanization and automation equipment, manual high-altitude operations can be reduced, and labor intensity and safety risks can be reduced.

[0061] During the automatic installation of photovoltaic modules, the photovoltaic modules to be installed need to be kept relatively parallel to the installed photovoltaic modules, and the modules to be installed grasped by the robot arm need to be pressed down to the position to be installed next to the installed modules, so as to ensure smooth installation. However, photovoltaic modules face the challenge of undulating terrain when they are installed. When the photovoltaic modules are pressed down, there is a tendency for the angles to deviate from those of the installed photovoltaic modules, which may lead to installation failure and rework.

[0062] In the prior art, the automatic installation of photovoltaic panels is based on the collected image data to identify the location of the photovoltaic components to be installed. However, since the photovoltaic components are installed in an outdoor environment, the light intensity is often too dark or too bright during image acquisition, resulting in the collected image being unable to accurately identify the location of the photovoltaic components to be installed, the posture and the angle difference between the installed photovoltaic components. This will lead to unsuccessful installation.

[0063] How to accurately detect the relative deflection angle of the upper and lower photovoltaic modules is the prerequisite for the automatic installation robot to perform subsequent adjustments. In order to solve the technical problem that the automatic installation of photovoltaic modules is prone to angle deviation, the present application provides a method, system and robot for automatic installation of photovoltaic modules. By combining point cloud filtering, clustering and other data processing methods, the plane of the photovoltaic modules to be installed and the plane of the installed photovoltaic modules are simulated, which can improve the accuracy of the angle control of the installation system, improve the stability of the photovoltaic module installation, and reduce the potential risk of wasting human resources and rework costs.

[0064] Reference Manual Attached Figure 1 An embodiment of a photovoltaic assembly automatic installation method provided by the present application comprises the following steps:

[0065] S100, photographing a target area including photovoltaic components to be installed and installed photovoltaic components by a depth camera; and acquiring RGB image data and point cloud data of the target area.

[0066] Specifically, in this embodiment, the depth camera includes multiple imaging modules that can capture and process different types of visual information. The depth camera uses the information of the RGB image and the depth map, combined with the camera's internal parameters (internal parameters), to calculate the three-dimensional coordinates (X, Y, Z) of any pixel in the camera coordinate system, i.e., point cloud data. This embodiment simultaneously obtains the RGB image data and point cloud data of the target area.

[0067] Among them, RGB image data: RGB image data provides x, y coordinates in the pixel coordinate system. These data contain color information and can show visual features such as image color and brightness. RGB images are captured by a standard color camera, which can record the color information of each pixel in the scene.

[0068] Point cloud data: Point cloud data provides the Z coordinate in the camera coordinate system, that is, the distance between the camera and the point. Depth cameras use different technologies (such as structured light, TOF (Time-of-Flight) or binocular vision) to measure the distance between each pixel and the camera to generate a point cloud depth map.

[0069] In this embodiment, the captured image range needs to include the installed photovoltaic components and the photovoltaic components to be installed next to them. The installed photovoltaic components are used as references for the photovoltaic components to be installed. First, the depth camera collects RGB image information and point cloud depth information, and the data is sent to the processor. Next, the processor fits the plane of the photovoltaic panel according to the point cloud data and calculates the plane parameters, thereby adjusting the relative posture relationship between the photovoltaic components to be installed and the installed photovoltaic components.

[0070] S200, processing the RGB image data based on an artificial intelligence model to identify the photovoltaic assembly to be installed so as to adjust its initial posture to be within a preset range relative to the installed photovoltaic assembly.

[0071] Preferably, the computer uses deep learning technology to identify the photovoltaic components that have been installed on the flat single axis, and presses the component to be installed grasped by the robotic arm down to the position to be installed next to the installed component.

[0072] Specifically, this step is the rough adjustment process of the installation, the purpose of which is to place the photovoltaic components to be installed in an approximate position that is convenient for subsequent fine-tuning, that is, a position slightly above the installed photovoltaic panels. Due to terrain reasons or other uncontrollable factors, the ambient light of the RGB image acquisition will change. By using deep learning to identify photovoltaic panels, the photovoltaic panels to be installed are pressed down to the position for subsequent fine-tuning. At this time, the placement and posture of the photovoltaic panels are not fixed, and only a preset range can be set to represent the initial posture for the start of fine-tuning.

[0073] S300, performing plane fitting on the point cloud data to obtain a first plane corresponding to the photovoltaic assembly to be installed and a second plane corresponding to the installed photovoltaic assembly.

[0074] Specifically, since the photovoltaic panels to be installed and those already installed are not blocked, the point cloud data acquired by the depth camera can be divided into two parts, upper and lower, through point cloud filtering and clustering, representing the plane of the photovoltaic components to be installed and the plane of the installed photovoltaic components respectively. The point clouds of the two parts are plane-fitted through a specific algorithm such as the RANSAC (Random Sample Consensus) algorithm, and the normal vector and center point coordinates of the optimal plane of the two parts of the point cloud are calculated.

[0075] S400, calculating a relative deflection angle between the first plane and the second plane, and then adjusting the relative posture relationship between the photovoltaic assembly to be installed and the installed photovoltaic assembly according to the relative deflection angle, so that the installation of the photovoltaic assembly to be installed meets the requirements.

[0076] Specifically, according to the fitted plane normal vectors and center point coordinates of the two photovoltaic modules, the angle difference between the two photovoltaic module planes in the X-axis and Y-axis directions and the depth difference in the z direction (vertical direction) are obtained.

[0077] The deflection relationship between planes can be expressed by attitude angles (Euler), including: yaw, pitch, roll, which reflects the attitude of the carrier relative to the reference plane. Pitch is a rotation around the X-axis, also called the pitch angle. When the positive half axis of the X-axis is above the horizontal plane passing through the origin of the coordinate system (head up), the pitch angle is positive, otherwise it is negative, as shown in the following figure: Yaw is a rotation around the Y-axis, also called the yaw angle. That is, the right yaw of the nose is positive, and vice versa. As shown in the following figure: Roll is a rotation around the Z-axis, also called the roll angle. The body rolls to the right is positive, and vice versa is negative.

[0078] After obtaining the absolute deflection angles of the two planes, the relative rotation angle between the two planes can be calculated, so as to adjust the posture of the photovoltaic component to be installed grasped by the robotic arm relative to the installed photovoltaic panel, so that it remains parallel to the installed photovoltaic component, which facilitates the subsequent improvement of installation accuracy.

[0079] Another embodiment of a method for automatic installation of photovoltaic components of the present application, based on an embodiment of the above method, further includes the step of S010 before executing step S300, pre-processing the point cloud data.

[0080] Specifically including: S011, using statistical filtering to remove outliers around photovoltaic modules.

[0081] S012, performing clustering processing on the filtered point cloud data.

[0082] In one implementation of this embodiment, the step S011 includes using statistical filtering to remove outliers around the photovoltaic module, specifically including:

[0083] S0111, for any point cloud in the point cloud data, count a number of neighborhood point clouds within a preset radius around it.

[0084] For details, please refer to the attached manual. Figure 2 As shown, Figure 2 The point cloud data initially obtained by the depth camera is shown. The neighborhood of each point cloud is calculated. For each point in the point cloud, the number of neighboring point clouds within a sphere formed by a certain radius around it is calculated.

[0085] S0112, calculating the first distance between all the point clouds and each of their neighborhood point clouds, and calculating the mean and standard deviation of the first distance.

[0086] Specifically, the distance between all neighboring point clouds and the point cloud is calculated; each point cloud is traversed to count the mean and standard deviation of the distance.

[0087] S0113, based on the mean and the standard deviation, confirm the point cloud whose distance from the neighborhood point cloud is greater than a first preset distance as the outlier, and remove the outlier points in the several neighborhood point clouds; the first preset distance is the sum of the mean and several times the standard deviation.

[0088] Specifically, outliers are filtered. If the distance between a certain neighborhood point cloud corresponding to a point cloud and the point cloud is greater than the mean plus k times the standard deviation, the point is considered to be an outlier, that is, a point that is irrelevant to the plane fitting of the photovoltaic module and needs to be removed from the point cloud.

[0089] It is better to set the outlier judgment standard based on the actual applicable scenario, and this application does not make any specific limitation.

[0090] Point cloud outlier filtering can remove noise and outliers and improve the overall quality of the data. After outlier removal, the point cloud distribution becomes cleaner, which helps to observe and analyze the data more clearly.

[0091] In another implementation of this embodiment, the step S012 performs clustering processing on the filtered point cloud data, including the following steps:

[0092] S0121, setting clustering parameters. Setting a second clustering distance based on the initial position of the photovoltaic assembly to be installed. Setting a point cloud quantity threshold for clustering based on the number of single-frame point clouds in the target area.

[0093] Specifically, the distance threshold of clustering is defined according to the approximate distance of the photovoltaic component plane. The minimum and maximum point cloud quantity constraints are set according to the single-frame point cloud quantity of the upper and lower photovoltaic component planes to improve the clustering accuracy.

[0094] S0122: Extend the maximum clustering distance outward from any point cloud as the center to find all neighboring points within the second clustering distance, and group all the neighboring points corresponding to the point cloud into one cluster.

[0095] Specifically, traverse the point cloud data: traverse each point and check whether it has been processed. For unprocessed points, grow a region from the point based on the distance threshold and find all neighboring points within the specified distance. Classify all neighboring points into a cluster and continue to process other unprocessed points until all points are assigned to a cluster to form a cluster.

[0096] S0123, traverse all point clouds in the target area to obtain clusters corresponding to all point clouds.

[0097] S0124, sorting all the clusters according to the number of point clouds they contain, and taking the first two clusters with the largest number of point clouds as the fitted plane point cloud clusters of the photovoltaic assembly to be installed and the installed photovoltaic assembly.

[0098] For details, please refer to the attached manual. Figure 3 As shown, Figure 3 The figure shows the point cloud data after preprocessing such as outlier filtering and clustering. After clustering, the first two clusters with the largest number of point clouds are the point cloud clusters of the photovoltaic modules to be installed and the point cloud clusters of the photovoltaic modules installed. The point cloud clusters can be distinguished based on the height of the horizontal position. The merged clusters that are too small (the point cloud clusters with a small number of point clouds) can be marked as abnormal data and removed.

[0099] The point cloud data quality is higher after preprocessing such as outlier filtering and clustering, which is helpful to the accuracy of subsequent plane fitting.

[0100] Another embodiment of the method for automatically installing photovoltaic modules in the present application, based on any one of the embodiments of the above method, the step: S300, performing plane fitting on the point cloud data, includes the following sub-steps:

[0101] S310, randomly selecting a number of point clouds from the point cloud data corresponding to the target area to generate a point cloud sample set.

[0102] S320: Use the point cloud sample set to fit and obtain a reference plane model.

[0103] S330, calculating a third distance between all point clouds in the point cloud data and the reference plane model; if the third distance is less than a second preset distance, treating the corresponding point cloud as an inner point of the reference plane model;

[0104] S340, counting the number of inner points of the reference plane model; if the number of inner points is greater than a preset number, the reference plane model is considered to be a valid plane model; otherwise, repeating the above steps until a valid plane model of the target area is obtained.

[0105] Reference Manual Attached Figure 4 , Figure 4 The coordinate system corresponding to the first plane and the second plane obtained by fitting is shown.

[0106] Preferably, the equation of the fitted plane model is ax+by+cz+d=0, wherein a, b, c, d are the parameters of the plane.

[0107] Construct the system of equations: For each point (x i ,y i ,z i ), construct the equation ax i +by i +cz i +d=0.

[0108] Construct the matrix equation: combine the equations of all points into a linear system of equations AX=0; where A is the coefficient matrix and X is the parameter vector [a,b,c,d] T ;

[0109] Calculate A T AX=A T 0, get the value of X. Extract a, b, c, d from X and determine the plane parameters.

[0110] Preferably, the maximum number of iterations of the model can be set, and the above steps S310 to S340 can be repeated until the maximum number of iterations is reached or a valid plane model is found. The valid plane model is a model whose number of inliers reaches the set inlier number threshold. If the number of inliers of the current model exceeds the best model previously recorded, the best model iteration is updated: random sampling and model fitting are repeated to find the optimal plane model.

[0111] In another implementation of the above embodiment, after step S300, the following sub-steps are further included:

[0112] S350, calculating the plane parameters corresponding to the first plane and the second plane, respectively. The plane parameters include a plane normal vector and a center point coordinate.

[0113] Specifically, the plane normal vector is used to calculate the relative deflection angle between the first plane and the second plane. The center point coordinates are used to calculate the depth difference between the first plane and the second plane.

[0114] Preferably, the first plane normal vector: n1 = (a1, b1, c1); the second plane normal vector: n2 = (a2, b2, c2)

[0115] Coordinates of the center point of the first plane: centre1 = (x1, y1, z1); coordinates of the center point of the second plane: centre2 = (x2, y2, z2).

[0116] Reference Manual Attached Figure 4 As shown by Figure 4In the figure, it can be seen that the relative deflection angle between the coordinate systems corresponding to the first plane and the second plane is schematically shown. According to the normal vectors of the two planes, the relative deflection angle between the two photovoltaic panels can be calculated, including the pitch angle and the roll angle, that is, the adjustment angle of the first plane relative to the second plane. Among them, the pitch angle represents the angle between the Y-axis of a coordinate system corresponding to the first plane and the horizontal plane (XY plane) of the coordinate system (reference coordinate system) corresponding to the second plane. If the angle between the Y-axis and the horizontal plane is above the horizontal plane, the pitch angle is positive; conversely, if the angle is below the horizontal plane, the pitch angle is negative. The roll angle represents the angle between the X-axis of a coordinate system corresponding to the first plane and the horizontal plane (XY plane) of the coordinate system (reference coordinate system) corresponding to the second plane. If the axis is tilted to the right, the roll angle is positive; if it is tilted to the left, the roll angle is negative.

[0117] Reference Manual Attached Figure 5 As shown, Figure 5 Schematic diagram of the height difference between the first plane and the second plane. After determining the pitch angle and the roll angle, the height difference between the two photovoltaic panels can be obtained according to the center point.

[0118] Another embodiment of the method for automatically installing a photovoltaic module of the present application, based on an embodiment of the above method, the step: S400, calculating the relative deflection angle between the first plane and the second plane, includes the following sub-steps:

[0119] S410: Construct a first quaternion corresponding to the first plane and a second quaternion corresponding to the second plane based on the plane normal vector.

[0120] S420: Calculate the product of the first quaternion and the second quaternion to obtain a rotation matrix between the first plane and the second plane.

[0121] S430, extracting the relative deflection angle from the rotation matrix, including the pitch angle and the roll angle, that is, the adjustment angle of the first plane relative to the second plane.

[0122] This application calculates the relative deflection angle between two planes, such as the pitch angle and the roll angle, based on the normal vectors of the two planes. The following steps may also be used:

[0123] 1. The angle θ between the two plane normal vectors can be obtained by calculating the dot product of the two normal vectors;

[0124]

[0125] 2. The rotation axis direction ν of the two plane normal vectors can be obtained by calculating the cross product of the two plane normal vectors;

[0126] v = n1 × n2 = (vx ,v y ,v z )

[0127] The unit vector is

[0128] 3. Use the Rodriguez rotation formula to calculate the rotation matrix R between the two plane normal vectors;

[0129] R=I+sin(θ)K+(1-cos(θ))K 2

[0130] Where I is the identity matrix and K is the rotation axis V unit The antisymmetric matrix of

[0131]

[0132] 4. The roll and pitch angles can be extracted from the rotation matrix.

[0133] pitch=arcsin(v y )

[0134] roll=actan2(v x ,0)

[0135] The difference in the center point coordinates can be used to determine the positional relationship between the two planes. If the positional relationship and deflection angle between the two planes are known, the posture adjustment scheme of the photovoltaic module to be installed can be reversed.

[0136] Preferably, in another implementation of this embodiment, the step S400 further includes:

[0137] S440: Based on the relative deflection angle between the first plane and the second plane, and the depth difference, generate a control instruction for controlling the posture adjustment of the photovoltaic assembly to be installed. The control instruction is used to adjust the relative posture relationship between the photovoltaic assembly to be installed and the installed photovoltaic assembly, so that the installation of the photovoltaic assembly to be installed meets the requirements.

[0138] The target requirement is that during the pressing process of the photovoltaic module to be installed, the corresponding plane always remains parallel to the plane corresponding to the installed photovoltaic module. When the depth difference (height difference) between the two is 0, the pressing stops.

[0139] At this point, the two photovoltaic panels have been adjusted to be relatively parallel, and the straightness of the photovoltaic panels can be subsequently detected, and the posture of the photovoltaic panels to be installed can be further adjusted to improve the installation accuracy of the photovoltaic panels.

[0140] Based on the same technical concept, the present application also discloses a photovoltaic module automatic installation system, which can be used to implement any of the above photovoltaic module automatic installation methods. Specifically, an embodiment of the photovoltaic module automatic installation system of the present application is shown in the attached specification. Figure 6 As shown, including:

[0141] The depth camera is used to photograph a target area including photovoltaic modules to be installed and photovoltaic modules that have been installed, and obtain RGB image data and point cloud data of the target area.

[0142] Specifically, in this embodiment, the depth camera includes multiple imaging modules that can capture and process different types of visual information. The depth camera uses the information of the RGB image and the depth map, combined with the camera's internal parameters (internal parameters), to calculate the three-dimensional coordinates (X, Y, Z) of any pixel in the camera coordinate system, i.e., point cloud data. This embodiment simultaneously obtains the RGB image data and point cloud data of the target area.

[0143] Among them, RGB image data: RGB image data provides x, y coordinates in the pixel coordinate system. These data contain color information and can show visual features such as image color and brightness. RGB images are captured by a standard color camera, which can record the color information of each pixel in the scene.

[0144] Point cloud data: Point cloud data provides the Z coordinate in the camera coordinate system, that is, the distance between the camera and the point. Depth cameras use different technologies (such as structured light, TOF (Time-of-Flight) or binocular vision) to measure the distance between each pixel and the camera to generate a point cloud depth map.

[0145] In this embodiment, the captured image range needs to include the installed photovoltaic components and the photovoltaic components to be installed next to them. The installed photovoltaic components are used as references for the photovoltaic components to be installed. First, the depth camera collects RGB image information and point cloud depth information, and the data is sent to the processor. Next, the processor fits the plane of the photovoltaic panel according to the point cloud data and calculates the plane parameters, thereby adjusting the relative posture relationship between the photovoltaic components to be installed and the installed photovoltaic components.

[0146] The processor is configured to perform the following steps: processing the RGB image data based on an artificial intelligence model. Identifying the photovoltaic assembly to be installed so as to adjust its initial posture to be within a preset range relative to the installed photovoltaic assembly. Performing plane fitting on the point cloud data to obtain a first plane corresponding to the photovoltaic assembly to be installed and a second plane corresponding to the installed photovoltaic assembly. Calculating a relative deflection angle between the first plane and the second plane.

[0147] Preferably, the computer uses deep learning technology to identify the photovoltaic components that have been installed on the flat single axis, and presses the components to be installed grasped by the robot arm to the position to be installed next to the installed components. This step is the coarse adjustment process of the installation, and the purpose is to place the photovoltaic components to be installed in an approximate position that is convenient for subsequent fine-tuning, that is, a position slightly above the installed photovoltaic panels. Due to terrain reasons or other uncontrollable factors, the ambient light of RGB image acquisition will change. By using deep learning to identify photovoltaic panels, the photovoltaic panels to be installed are pressed down to the position for subsequent fine-tuning. At this time, the placement position and posture of the photovoltaic panels are not fixed, and only a preset range can be set to represent the initial posture at the beginning of fine-tuning.

[0148] Since the photovoltaic panels to be installed and those already installed are not blocked, the point cloud data acquired by the depth camera can be divided into two parts, upper and lower, through point cloud filtering and clustering, representing the plane of the photovoltaic components to be installed and the plane of the installed photovoltaic components respectively. The point clouds of the two parts are plane-fitted by a specific algorithm such as the RANSAC (Random Sample Consensus) algorithm, and the normal vectors and center point coordinates of the optimal planes of the two parts of the point cloud are calculated. Based on the plane normal vectors and center point coordinates of the two photovoltaic components fitted, the angle difference between the planes of the two photovoltaic components in the X-axis and Y-axis directions and the depth difference in the z direction (vertical direction) are obtained.

[0149] Another embodiment of a photovoltaic component automatic installation system provided by the present application, based on the above system embodiment, the photovoltaic component automatic installation system further includes: a controller.

[0150] The controller is used to receive instructions from the processor and control the mechanical arm to adjust the relative posture relationship between the photovoltaic assembly to be installed and the installed photovoltaic assembly, so that the installation of the photovoltaic assembly to be installed meets the requirements.

[0151] Specifically, the target requirement is that during the downward pressing process of the photovoltaic module to be installed, the corresponding plane is always parallel to the plane corresponding to the installed photovoltaic module. When the depth difference (height difference) between the two is 0, the downward pressing is stopped. The difference based on the center point coordinates can be used to determine the positional relationship between the two planes. If the positional relationship and deflection angle between the two planes are known, the posture adjustment scheme of the photovoltaic module to be installed can be inferred.

[0152] In other embodiments of the present application, the photovoltaic module automatic installation system can also be used for the automatic installation of other outdoor equipment; different installation requirements can be set based on different projects, and the present application does not make specific limitations. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0153] Based on the same technical concept, the present application also discloses an automatic installation robot, which includes the photovoltaic component automatic installation system described in any one of the above embodiments.

[0154] Preferably, the automatic installation robot also includes a mechanical arm, including a mechanical arm and a mechanical hand, which is controlled by a controller: the mechanical arm is responsible for rough positioning and rapid movement to a suitable installation space, and more detailed and specific installation work is completed by the installation mechanical hand.

[0155] In other embodiments, the automatic installation robot further includes a power supply module, a walking module, etc. These structures work together to enable the photovoltaic panel automatic installation robot to efficiently and accurately complete the installation of photovoltaic panels.

[0156] The photovoltaic module automatic installation method, system and robot of the present application have the same technical concept, and the technical details of the embodiments of the two are applicable to each other. In order to reduce repetition, they will not be repeated here.

[0157] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned program modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program units or modules to complete all or part of the functions described above. The program modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a processing unit, and the above-mentioned integrated unit can be implemented in the form of hardware or in the form of software program units. In addition, the specific names of the program modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0158] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

Claims

1. A method for automatically installing photovoltaic modules, characterized in that: The steps include: Using a depth camera to photograph a target area including photovoltaic modules to be installed and installed photovoltaic modules; obtaining RGB image data and point cloud data of the target area; Processing the RGB image data based on an artificial intelligence model; identifying the photovoltaic assembly to be installed so as to adjust its initial posture relative to the installed photovoltaic assembly within a preset range; Performing plane fitting on the point cloud data; obtaining a first plane corresponding to the photovoltaic assembly to be installed and a second plane corresponding to the installed photovoltaic assembly; The relative deflection angle between the first plane and the second plane is calculated; and then the relative posture relationship between the photovoltaic assembly to be installed and the installed photovoltaic assembly is adjusted according to the relative deflection angle, so that the installation of the photovoltaic assembly to be installed meets the requirements.

2. A method for automatically installing photovoltaic components according to claim 1, characterized in that: Before performing plane fitting on the point cloud data, the step further includes preprocessing the point cloud data; Specifically comprising: using statistical filtering to remove outliers around the photovoltaic module; The filtered point cloud data is clustered.

3. A method for automatically installing photovoltaic modules as claimed in claim 2, characterized in that: The method of removing outliers around the photovoltaic module by using statistical filtering comprises the following steps: For any point cloud in the point cloud data, counting a number of neighboring point clouds within a preset radius around it; Calculating a first distance between all the point clouds and each of the field point clouds, and calculating a mean and a standard deviation of the first distance; Based on the mean and the standard deviation, point clouds whose distance from the neighborhood point clouds is greater than a first preset distance are identified as outliers, and outliers in the several neighborhood point clouds are removed; the first preset distance is the sum of the mean and several times the standard deviation.

4. A method for automatically installing photovoltaic modules as claimed in claim 2, characterized in that: The clustering process of the filtered point cloud data comprises the following steps: Setting clustering parameters; setting a second clustering distance based on the initial position of the photovoltaic assembly to be installed; setting a point cloud quantity threshold for clustering based on the number of single-frame point clouds in the target area; Extending the maximum clustering distance outward from any point cloud as the center, searching for all neighboring points within the second clustering distance; and grouping all the neighboring points corresponding to the point cloud into one cluster; Traversing all point clouds in the target area to obtain clusters corresponding to all point clouds; All the clusters are sorted according to the number of point clouds they contain, and the first two clusters with the largest number of point clouds are taken as the fitted plane point cloud clusters of the photovoltaic assembly to be installed and the installed photovoltaic assembly.

5. A method for automatically installing a photovoltaic module according to any one of claims 1 to 4, characterized in that: The plane fitting of the point cloud data comprises the following steps: Randomly selecting a number of point clouds from the point cloud data corresponding to the target area to generate a point cloud sample set; Using the point cloud sample set to fit a reference plane model; Calculating a third distance between all point clouds in the point cloud data and the reference plane model; if the third distance is less than a second preset distance, considering the corresponding point cloud as an inner point of the reference plane model; Counting the number of interior points of the reference plane model; If the number of the inner points is greater than a preset number, the reference plane model is considered to be a valid plane model; otherwise, the above steps are repeated until a valid plane model of the target area is obtained.

6. A method for automatically installing photovoltaic modules as claimed in claim 5, characterized in that: After performing plane fitting on the point cloud data, the following steps are also included: Calculating plane parameters corresponding to the first plane and the second plane respectively; The plane parameters include the plane normal vector and the center point coordinates; The plane normal vector is used to calculate the relative deflection angle between the first plane and the second plane; The center point coordinates are used to calculate the depth difference between the first plane and the second plane.

7. A method for automatically installing a photovoltaic module according to any one of claim 6, characterized in that: The calculating of the relative deflection angle between the first plane and the second plane comprises the following steps: Based on the plane normal vector, construct a first quaternion corresponding to the first plane and a second quaternion corresponding to the second plane; Calculate the product of the first quaternion and the second quaternion to obtain a rotation matrix between the first plane and the second plane; The relative deflection angle, including the pitch angle and the roll angle, is extracted from the rotation matrix, that is, the adjustment angle of the first plane relative to the second plane.

8. A photovoltaic module automatic installation system, characterized in that: The system is used to execute the photovoltaic assembly automatic installation method according to any one of claims 1 to 7; comprising: A depth camera is used to photograph a target area including photovoltaic modules to be installed and installed photovoltaic modules; and obtain RGB image data and point cloud data of the target area; A processor is used to perform the following steps: processing the RGB image data based on an artificial intelligence model; identifying the photovoltaic component to be installed so as to adjust its initial posture within a preset range relative to the installed photovoltaic component; performing plane fitting on the point cloud data to obtain a first plane corresponding to the photovoltaic component to be installed and a second plane corresponding to the installed photovoltaic component; and calculating a relative deflection angle between the first plane and the second plane.

9. A photovoltaic module automatic installation system as claimed in claim 8, characterized in that: Also includes: Controller; The controller is used to receive instructions from the processor and control the mechanical arm to adjust the relative posture relationship between the photovoltaic assembly to be installed and the installed photovoltaic assembly so that the installation meets the requirements.

10. An automatic installation robot, characterized in that: The automatic installation robot comprises the photovoltaic component automatic installation system according to any one of claims 8-9.

Images