Photovoltaic panel paving method and device, storage medium and robot

Abstract

The invention discloses a photovoltaic panel paving method and device, a storage medium and a robot, and the method comprises the steps: obtaining a first image of a target region, wherein the target region is internally provided with a target part; determining a first area of the target piece based on the first image; determining a first plane and a first axis based on the first region and a first preset equation; determining an original point, a second axis and a third axis based on the first plane and the first area, wherein the first axis, the second axis and the third axis intersect pairwise; determining a rotation matrix based on the origin, the first axis, the second axis and the third axis; and according to the original point and the rotation matrix, the photovoltaic panel to be laid is installed at the target position of the target area. The photovoltaic panel laying efficiency and accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, specifically to a photovoltaic panel installation method, apparatus, storage medium, and robot. Background Technology

[0002] Currently, solar photovoltaic (PV) panels are power generation devices that convert solar energy into electrical energy. PV panels can be made into different shapes, and they can be interconnected to generate more electricity. How to lay PV panels has become a research hotspot during the construction of photovoltaic power plants. In existing technologies, PV panels are usually moved and manually installed by workers, a method that results in low efficiency and low accuracy in PV panel installation. Summary of the Invention

[0003] This application provides a photovoltaic panel installation method, apparatus, storage medium, and robot, which improves the efficiency and accuracy of photovoltaic panel installation.

[0004] In a first aspect, this application provides a photovoltaic panel installation method, comprising: acquiring a first image of a target area, wherein the target area contains a target component; determining a first region of the target component based on the first image; determining a first plane and a first axis based on the first region and a first preset equation; determining an origin, a second axis, and a third axis based on the first plane and the first region, wherein the first axis, the second axis, and the third axis intersect each other; determining a rotation matrix based on the origin, the first axis, the second axis, and the third axis; and installing the photovoltaic panel to be installed to the target position in the target area according to the origin and the rotation matrix. Thus, by determining the first plane and the first axis based on the first region and the first preset equation, determining the origin, the second axis, and the third axis based on the first plane and the first region, determining the rotation matrix based on the origin, the first axis, the second axis, and the third axis, and installing the photovoltaic panel to be installed to the target position in the target area according to the origin and the rotation matrix, the installation of the photovoltaic panel to be installed to the target position in the target area can be achieved without manual handling, thereby improving the efficiency and accuracy of photovoltaic panel installation.

[0005] In some feasible examples, determining the first region of the target component based on the first image includes: the first image comprising a three-dimensional image and a second image; determining the position of the target component based on the second image; obtaining a first mask image of the target component based on its position; detecting contours in the first mask image; and determining the contour with the largest area in the first mask image as the first region. Thus, automatically determining the first region based on the second image improves both the speed and accuracy of obtaining the first region.

[0006] In some feasible examples, determining the first plane and the first axis based on the first region and the first preset equation includes: determining a first coordinate group of the first data point group based on the first region; determining the first plane based on the first coordinate group and the first preset equation; and determining the first axis based on the normal vector of the first plane. Thus, determining the first coordinate group of the first data point group based on the first region accurately obtains the spatial position information of the first data point group, and using the first coordinate group and the first preset equation to determine the first plane ensures the accuracy and reliability of the first plane.

[0007] In some feasible examples, determining the origin, second axis, and third axis based on the first plane and the first region, where the first axis, second axis, and third axis intersect each other pairwise, includes: determining a second set of coordinates for multiple first target points on the first and second sides based on the first region; determining a first straight line and a first residual based on the multiple second coordinate sets, a second preset equation, and the least squares method; determining a second straight line and a second residual based on the multiple second coordinate sets, a third preset equation, and the least squares method, where the first straight line and the second straight line intersect; determining the origin based on the first straight line and the second straight line; and determining the second axis and the third axis based on the first plane and the three-dimensional image. Thus, determining the second coordinate set based on the first region, determining the first straight line and the first residual based on the multiple second coordinate sets, the second preset equation, and the least squares method, determining the second straight line and the second residual based on the multiple second coordinate sets, the third preset equation, and the least squares method, determining the origin based on the first straight line and the second straight line, and determining the second axis and the third axis based on the first plane and the three-dimensional image improves the accuracy and efficiency of obtaining the second axis and the third axis.

[0008] In some feasible examples, determining the second coordinate set of multiple first target points on the first and second sides based on the first region includes: the target component includes a first photovoltaic panel or a feature component; when the target region contains the first photovoltaic panel, determining a first corner point, a second corner point, a third corner point, and a fourth corner point based on the first region; determining multiple first target points based on the first corner point, the second corner point, the third corner point, and the fourth corner point; when the target region contains the feature component, processing the first region using a key point detection algorithm to obtain multiple first target points. Thus, when the target region contains the first photovoltaic panel, determining the first corner point, the second corner point, the third corner point, and the fourth corner point based on the first region, and determining multiple first target points based on the first corner point, the first region contains the feature component, and the first region is processed using a key point detection algorithm to obtain multiple first target points. Obtaining multiple first target points according to different types of target components improves the accuracy and efficiency of obtaining multiple first target points and also expands the applicability of the method.

[0009] In some feasible examples, determining the second axis and the third axis based on the first plane and the three-dimensional image includes: determining a first endpoint and a second endpoint based on the first straight line; determining a third endpoint and a fourth endpoint based on the second straight line; determining a third straight line in a first direction based on the first endpoint and the second endpoint; determining a fourth straight line in a second direction based on the third endpoint and the fourth endpoint, wherein the first direction intersects the second direction; and determining the second axis and the third axis based on the first residual, the second residual, the third line, and the fourth line. Thus, determining the third straight line in the first direction based on the first endpoint and the second endpoint, determining the fourth straight line in the second direction based on the third endpoint and the fourth endpoint, and determining the second axis and the third axis based on the first residual, the second residual, the third line, and the fourth line improves the accuracy and efficiency of obtaining the second axis and the third axis.

[0010] In some feasible examples, determining the rotation matrix based on the origin, the first axis, the second axis, and the third axis includes: finding the three-dimensional coordinates of the origin by searching the three-dimensional image based on the two-dimensional coordinates of the origin; determining the first coordinates of the origin based on the three-dimensional coordinates of the origin and the first plane; and determining the rotation matrix based on the first coordinates, the first axis, the second axis, and the third axis. Thus, determining the rotation matrix based on the first coordinates, the first axis, the second axis, and the third axis improves the accuracy and efficiency of obtaining the rotation matrix.

[0011] Secondly, this application provides a photovoltaic panel installation apparatus, including a processor and a memory, wherein the memory is used to store computer-readable instructions, and the processor is used to call the instructions stored in the memory to execute the method provided in the first aspect or any embodiment of the first aspect.

[0012] Thirdly, this application provides a storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the method provided in the first aspect or any embodiment of the first aspect.

[0013] Fourthly, this application provides a robot, which includes the photovoltaic panel installation device described in the second aspect, for installing the photovoltaic panels.

[0014] Implementing the embodiments of this application has the following beneficial effects:

[0015] The first plane and the first axis are determined based on the first region and the first preset equation. The origin, the second axis and the third axis are determined based on the first plane and the first region. The rotation matrix is ​​determined based on the origin, the first axis, the second axis and the third axis. The photovoltaic panels to be installed are then installed at the target position in the target region according to the origin and the rotation matrix. The photovoltaic panels to be installed are installed at the target position in the target region without manual handling, which improves the efficiency and accuracy of photovoltaic panel installation. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating an application scenario of a photovoltaic panel installation method provided in an embodiment of this application;

[0018] Figure 2 A schematic flowchart illustrating a photovoltaic panel installation method provided in this application embodiment;

[0019] Figure 3 This is a schematic diagram of a mounting frame supporting a photovoltaic panel, provided in an embodiment of this application.

[0020] Figure 4 This is a structural schematic diagram of a photovoltaic panel installation device provided in an embodiment of this application. Detailed Implementation

[0021] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used only to distinguish different objects and not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0022] "At least one item" refers to one or more items; "more than one" refers to two or more items; "at least two items" refers to two or three or more items; "and / or" describes the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after it are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items. For example, at least one of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c".

[0023] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario of a photovoltaic panel installation method provided in an embodiment of this application. For example... Figure 1 As shown in the schematic diagram, this application scenario includes a construction worker 101, a photovoltaic panel installation device 102, a photovoltaic panel 103, and a server 104. Optionally, the photovoltaic panel installation device 102 includes one or more scanning units. Optionally, the photovoltaic panel installation device 102 includes one or more cameras; this application does not limit the structure of the photovoltaic panel installation device 102. Optionally, the photovoltaic panel installation device 102 can be various electronic devices; for example, the photovoltaic panel installation device 102 can be a vehicle or a robot, etc.

[0025] Optionally, server 104 can be a standalone server 104, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. Server 104 can also be implemented through a server 104 cluster composed of multiple sub-servers.

[0026] It should be noted that, Figure 1The number and form of each device in the illustrated system, as well as the number of construction workers 101, are for illustrative purposes only and do not constitute a limitation on the embodiments of this application. One construction worker 101 can operate multiple photovoltaic panel installation devices 102, and one photovoltaic panel installation device 102 can install multiple photovoltaic panels 103. The construction worker 101 can operate the photovoltaic panel installation device 102 to control it to perform corresponding actions to install the photovoltaic panels 103. The photovoltaic panel installation device 102 can be... Figure 1 The structure shown can also be other structures. Server 104 may include multiple personal computers (PCs), cloud virtual hosts, cloud databases, and cloud storage. The operating systems of the PCs, such as computers, may include, but are not limited to, Linux systems, Unix systems, and Windows series systems (such as Windows XP, Windows 7, etc.).

[0027] The following describes a photovoltaic panel installation method provided in the embodiments of this application. This method can be executed by a photovoltaic panel installation device, which can be implemented by software and / or hardware, and is generally integrated into the server 104.

[0028] Please see Figure 2 , Figure 2 This is a flowchart illustrating a photovoltaic panel installation method provided in this application. Taking the application of this method to photovoltaic panel installation as an example, the photovoltaic panel installation device may include a server or electronic equipment. The method includes the following steps S201-S206, wherein...

[0029] S201: Obtain the first image of the target area, which contains the target object.

[0030] Optionally, the first image can be acquired using the following acquisition devices: a camera, a drone, and a scanning device. The photovoltaic panel installation device includes a camera, which typically has high resolution and flexible shooting angles, suitable for image acquisition in different scenarios. The drone can communicate and transmit data with the photovoltaic panel installation device. Drones have a significant advantage in acquiring data over large installation areas or hard-to-reach areas, as they can perform aerial photography to obtain a wider field of view and more detailed first image information. The photovoltaic panel installation device includes a scanning device, which includes a scanner. The scanner can perform high-precision scanning of the target area to acquire high-quality first image data.

[0031] S202: Determine the first region of the target component based on the first image.

[0032] Determining a first region of the target component based on the first image includes the following steps:

[0033] The first image includes a three-dimensional image and a second image;

[0034] The location of the target component is determined based on the second image;

[0035] Obtain the first mask image of the target component based on its location;

[0036] Detect the contours in the first mask image and determine the contour with the largest area in the first mask image as the first region.

[0037] In this application embodiment, the second image includes a color image, which is an image of a target object of various colors generated based on different intensities of the three basic colors: red, green and blue. In red, green and blue, the numerical range of each color channel is typically 0 to 255.

[0038] The target component includes a first photovoltaic panel or a feature block. When the photovoltaic panel to be installed is first mounted on the mounting frame, a feature block is placed at a fixed position on the edge of the mounting frame. When the target component is the first photovoltaic panel, the second image can be obtained by taking a picture with a camera. After converting the second image to a grayscale image and applying filtering to reduce noise, the image of the first photovoltaic panel in the second image is segmented using an image segmentation algorithm. A separate binary mask is generated for each first photovoltaic panel as a unique identifier. The first photovoltaic panel located in the lower left corner is selected as the reference photovoltaic panel. The first photovoltaic panel includes a first boundary, a second boundary, a third boundary, and a fourth boundary. The first boundary is perpendicular to the second and fourth boundaries, and the third boundary is perpendicular to the second and fourth boundaries. Multiple solar cells are arranged in an array within the area formed by the first, second, third, and fourth boundaries.

[0039] The bounding box and corresponding binary mask of each first photovoltaic panel within the second image are annotated. The bounding box of the first photovoltaic panel includes its width and height, and the binary mask of the first photovoltaic panel is a binary image where each pixel has a value of 0 or 255. In image processing: 0 (black): represents the background; 255 (white): represents the first photovoltaic panel. The first photovoltaic panel is separated from the background using the binary mask.

[0040] The second image is segmented based on the bounding box of each first photovoltaic panel and its corresponding binary mask within the labeled second image to obtain the binary mask image of each first photovoltaic panel.

[0041] Identify the center point of each first photovoltaic panel. Assuming the coordinate system of the second image has the origin at the bottom left corner, the X-axis to the right, and the Y-axis upward, find the center point with the smallest Y-coordinate, which is the bottommost first photovoltaic panel. If multiple first photovoltaic panels have the same smallest Y-coordinate, select the one with the smallest X-coordinate, which is the leftmost first photovoltaic panel. The first photovoltaic panel obtained in this way is the one located at the bottom left corner.

[0042] The binary mask region of the first photovoltaic panel is read, and a contour detection algorithm is used to detect all contours within the mask region. The contour with the largest area is selected, and its minimum bounding rectangle is calculated as the preliminary contour of the first photovoltaic panel, which is the first region. In this way, the first region is automatically determined based on the second image, improving both the speed and accuracy of acquiring the first region.

[0043] S203: Determine the first plane and the first axis based on the first region and the first preset equation.

[0044] Determining the first plane and the first axis based on the first region and the first preset equation includes the following steps:

[0045] Determine the first coordinate group of the first data point group based on the first region;

[0046] The first plane is determined based on the first coordinate set and the first preset equation;

[0047] The first axis is determined based on the normal vector of the first plane.

[0048] In this embodiment of the application, a first data point group is determined in the preliminary contour. The first data point group includes multiple first data points, and the coordinates of the multiple first data points form a first coordinate group.

[0049] In this embodiment, a keypoint detection algorithm is used to determine a first coordinate group of a first data point group in a first region. This first data point group includes multiple target endpoints, which include first-type, second-type, third-type, and fourth-type target endpoints. Endpoints of battery cells located at the first boundary are designated as first-type target endpoints, endpoints of battery cells located at the second boundary are designated as second-type target endpoints, endpoints of battery cells located at the third boundary are designated as third-type target endpoints, and endpoints of battery cells located at the fourth boundary are designated as fourth-type target endpoints. In this application, the keypoint detection algorithm is used to detect target endpoints in the first region, and the steps include:

[0050] 1. Determine the position of each solar cell in the photovoltaic panel relative to the initial outline, then identify the positions of the first, second, third, and fourth boundaries. Moving from the first boundary towards the third boundary, select the first column of cells closest to the first boundary as the target endpoint for the first cell group of the photovoltaic panel. Moving from the third boundary towards the first boundary, select the first column of cells closest to the third boundary as the target endpoint for the third cell group. Moving from the second boundary towards the fourth boundary, select the first row of cells closest to the second boundary as the target endpoint for the second cell group. Moving from the fourth boundary towards the second boundary, select the first row of cells closest to the fourth boundary as the target endpoint for the fourth cell group.

[0051] 2. Compare the positions of the endpoints of each cell in the first cell group with the first boundary, and detect the endpoints of the cells in the first cell group that fall on the first boundary, as the first type of target endpoints. Compare the positions of the endpoints of each cell in the second cell group with the second boundary, and detect the endpoints of the cells in the second cell group that fall on the second boundary, as the second type of target endpoints. Compare the positions of the endpoints of each cell in the third cell group with the third boundary, and detect the endpoints of the cells in the third cell group that fall on the third boundary, as the third type of target endpoints. Compare the positions of the endpoints of each cell in the fourth cell group with the fourth boundary, and detect the endpoints of the cells in the third cell group that fall on the third boundary, as the fourth type of target endpoints.

[0052] 3. Load the pre-trained keypoint detection model, read the region of interest sub-image, convert the region of interest sub-image into the model input format, use the keypoint detection model to infer the missing or blurred target endpoints among the first type of target endpoints, the second type of target endpoints, the third type of target endpoints and the fourth type of target endpoints, predict the target endpoints, and draw the predicted target endpoints on the region of interest sub-image.

[0053] Next, a first preset equation is set, typically a plane equation as the fitting function. For example, the first preset equation can be set as z = ax + by + c. Then, the coordinates of multiple target endpoints are substituted into the first preset equation, and an error function (usually the sum of squared distances from points to the fitting plane) is constructed to measure the goodness of fit. Next, the parameters a, b, and c of the fitting function are solved by minimizing this error function. The obtained parameters a, b, and c are substituted into the form of the fitting function to obtain the final first plane. Then, the normal vector of the first plane is used as the first axis; for example, the normal vector of z = ax + by + c is (a, b, -1). Thus, based on the first region, the first coordinate set of the first data point group is determined, accurately obtaining the spatial position information of the first data point group. Using the first coordinate set and the first preset equation to determine the first plane ensures the accuracy and reliability of the first plane.

[0054] Optionally, when the target component is the first photovoltaic panel, a coordinate system is established starting from the origin, with the horizontal axis as the X-axis and the vertical axis as the Y-axis. The coordinates (x, y) of the endpoints of the first type of target are then obtained. i y i ) 1 The coordinates (x) of the endpoints of the second type of target i y i ) 2 The coordinates (x) of the endpoints of the third type of target i y i ) 3 and the coordinates (x) of the endpoints of the fourth type of target i y i ) 4 Where i = 1, ..., n. In this application, a key point detection algorithm is used to detect the target endpoints of the photovoltaic panel image. The key point detection algorithm detects the key points of the horizontal and vertical edges on the photovoltaic panel. The key point detection algorithm has a good processing effect on the invisible key points caused by occlusion, high reflectivity, etc., and can supplement the accuracy of the instance segmentation algorithm. It ensures that the system can correctly and accurately find the edge of the photovoltaic panel when the instance segmentation algorithm fails to accurately segment the edge.

[0055] The coordinates (x) of the endpoints of the first type of target i y i ) 1 (where i = 1, ..., n) preprocessing and regression analysis are performed to obtain the equation of the first straight line; the coordinates (x, y) of the endpoints of the second type of target are then processed. i y i ) 2 (Where i = 1, ..., n) Preprocessing and regression analysis are performed to obtain the equation of the second straight line. The coordinates (x, y) of the endpoints of the third type of target are then... i y i ) 3 (Where i = 1, ..., n) Preprocessing and regression analysis are performed to obtain the equation of the third straight line. The coordinates (x, y) of the endpoints of the fourth type of target are then... i y i ) 4 (where i = 1, ..., n) Preprocessing and regression analysis are performed to obtain the equation of the fourth straight line.

[0056] In this application, a support vector machine (SVM) is used for regression analysis to find a straight line that makes all target endpoints as close to this line as possible, while allowing for a certain degree of error. The SVM uses an insensitive loss function ε to measure the prediction error. Its definition is:

[0057] maxL ε(y,f(x))=(0,|yf(x)|-ε)

[0058] In the formula: y is the true value; f(x) is the predicted value; ε is a user-defined parameter, representing the allowable error range.

[0059] The objective function is:

[0060]

[0061] The constraints are:

[0062]

[0063] In the formula: ω is the weight vector, ω T b is the transpose of ω; b is the bias term; δ i and C is a slack variable;

[0064] This is a regularization parameter that controls the balance between model complexity and error.

[0065] Solving using the Lagrange duality problem:

[0066]

[0067] The constraints are:

[0068]

[0069] In the formula: α i α i *、α j and α j * represents the Lagrange multiplier; i = 1, 2, ..., n; j = 1, 2, ..., n.

[0070] After obtaining the Lagrange multipliers, the weight vector ω is calculated using the following formula:

[0071]

[0072] The bias term b is calculated using the support vectors, for support vector x. s have:

[0073] b = y s -ω T x s -ε or b = y s -ω T x s +ε

[0074] In the formula: y s Represents support vector x sThe corresponding real labels are s = 1, 2, ..., n.

[0075] Use all support vectors x s Calculate the average value of b:

[0076]

[0077] In the formula: S is the set of support vectors, N s It represents the number of support vectors.

[0078] The prediction model of a support vector machine can be represented as:

[0079] f(x)=ω T x+b

[0080] The equation of the line is:

[0081] y = ω1x1 + ... + ω d x d +b

[0082] In the formula: x = (x1, x2, ..., x...) d Let ω be the input feature vector, where ω = (ω1, ω2, ..., ω3). d ) is the weight vector, and b is the bias term.

[0083] Taking the first type of target endpoint as an example, assuming the coordinates of the first type of target endpoint are:

[0084]

[0085] After training with a support vector machine, we obtain:

[0086]

[0087] The equation of the line is:

[0088] y = 2x1 + 0x2 + 0 = 2x1

[0089] The closed region formed by the equations of the first, second, third, and fourth lines is the first plane of the photovoltaic panel, and the normal vector of the first plane is taken as the first axis.

[0090] S204: Determine the origin, second axis, and third axis based on the first plane and the first region, with the first axis, second axis, and third axis intersecting each other.

[0091] The origin, second axis, and third axis are determined based on the first plane and the first region. The first axis, second axis, and third axis intersect each other pairwise, including the following steps:

[0092] A second set of coordinates is determined based on the first region, which contains multiple first target points on the first and second sides.

[0093] The first straight line and the first residual are determined based on multiple second coordinate sets, second preset equations, and the least squares method.

[0094] The second line and the second residual are determined based on multiple second coordinate sets, a third preset equation, and the least squares method. The first line and the second line intersect.

[0095] The origin is determined based on the first and second straight lines;

[0096] The second and third axes are determined based on the first plane and the three-dimensional image.

[0097] In this embodiment of the application, optionally, the first side is a horizontal side and the second side is a vertical side.

[0098] When the target component is the first photovoltaic panel, a key point detection algorithm is used to detect key points in the first region to obtain the coordinates of multiple first target points on the horizontal and vertical sides.

[0099] In this embodiment of the application, a second preset equation y = m1x + n1 is set to obtain multiple first target points on the horizontal side, and the coordinates (X, Y, Z) of the multiple first target points are set. i Y i Let i = 1, 2, ..., n, and substitute these values ​​into the second pre-set equation to calculate X. i The average value is X p Calculate Y i The average value of Y is obtained p Based on the coordinates and X of multiple first target points p and Y p Calculate the slope m1, based on the slope m1 and X p and Y p Calculate the intercept n1. Finally, substitute the slope m1 and the intercept n1 into the second preset equation to obtain the equation of the first line. Then, calculate the equation of each data point (X). i Y i The first residual is obtained by finding the perpendicular distance from the first straight line y = m1x + n1.

[0100] For example, set a third preset equation y = m²x + n², obtain multiple first target points on the vertical side, and set the coordinates (X, Y, N) of the multiple first target points. i Y i Let i = 1, ..., n, and substitute these values ​​into the second pre-set equation to calculate X. i The average value is X p Calculate Y i The average value of Y is obtained pBased on the coordinates and X of multiple first target points p and Y p Calculate the slope m2, based on the slope m2 and X. p and Y p Calculate the intercept n2. Finally, substitute the slope m2 and the intercept n2 into the second preset equation to obtain the equation of the second line. Then, calculate the equation of each data point (X). i Y i The perpendicular distance from the second line y = m²x + n² to the second line is used to obtain the second residual.

[0101] The first straight line and the second straight line intersect to form the first intersection point, and the two-dimensional coordinates of the first intersection point are the same as the two-dimensional coordinates of the origin.

[0102] Based on the first region, a second set of coordinates is determined for multiple first target points on the first and second sides, including:

[0103] The target component includes a first photovoltaic panel or a feature component;

[0104] When the target area contains a first photovoltaic panel, the first corner point, second corner point, third corner point, and fourth corner point are determined based on the first area;

[0105] Multiple first target points are determined based on the first corner point, the second corner point, the third corner point, and the fourth corner point;

[0106] When there are features within the target area, a key point detection algorithm is used to process the first area to obtain multiple first target points.

[0107] The first region also includes a first corner point, a second corner point, a third corner point, and a fourth corner point. The four corner points of the first region can be obtained using a key point detection algorithm. The intersection of the first line equation and the second line equation is the first corner point, the intersection of the second line equation and the third line equation is the second corner point, the intersection of the third line equation and the fourth line equation is the third corner point, and the intersection of the fourth line equation and the first line equation is the fourth corner point.

[0108] In this embodiment of the application, when installing photovoltaic panels on each set of mounting frames, if it is the first time installing photovoltaic panels and there are no photovoltaic panels on the mounting frame at this time, a feature is placed at a fixed position on the edge of the mounting frame. The feature includes a feature block, which is a strictly planar object with straight lines in the horizontal and vertical directions printed on it.

[0109] In this embodiment of the application, when installing photovoltaic panels on each set of mounting frames, if some photovoltaic panels have already been installed on the set of mounting frames, the 3D camera will capture the leftmost or rightmost photovoltaic panel and identify the photovoltaic panel in the lower left or lower right corner.

[0110] Thus, when the target area contains a first photovoltaic panel, the first corner point, second corner point, third corner point, and fourth corner point are determined based on the first area. Multiple first target points are then determined based on the first corner point, second corner point, third corner point, and fourth corner point. When the target area contains feature components, the first area is processed using a key point detection algorithm to obtain multiple first target points. Multiple first target points are obtained according to different types of target components, which improves the accuracy and efficiency of obtaining multiple first target points and also expands the applicability of the method.

[0111] Determining the second and third axes based on the first plane and the three-dimensional image includes the following steps:

[0112] The first endpoint and the second endpoint are determined based on the first straight line;

[0113] The third and fourth endpoints are determined based on the second straight line;

[0114] Determine the third straight line in the first direction based on the first endpoint and the second endpoint;

[0115] The fourth straight line in the second direction is determined based on the third and fourth endpoints, and the first direction intersects the second direction;

[0116] The second and third axes are determined based on the first residual, the second residual, the third line, and the fourth line.

[0117] In this embodiment, two endpoints are identified at both ends of the first straight line: the first endpoint and the second endpoint. Optionally, when the first straight line is a straight line on the horizontal side, x takes the minimum value and substitutes it into y = m1x + n1 to obtain the first endpoint; when x takes the maximum value, substitutes it into y = m1x + n1 to obtain the second endpoint. Among multiple data points in the three-dimensional image, the three-dimensional coordinates of the first endpoint are searched based on the two-dimensional coordinates of the first endpoint to obtain the three-dimensional coordinates of the first endpoint. Among multiple data points in the three-dimensional image, the three-dimensional coordinates of the second endpoint are searched based on the two-dimensional coordinates of the second endpoint to obtain the three-dimensional coordinates of the second endpoint. Based on the three-dimensional coordinates of the first endpoint and the three-dimensional coordinates of the second endpoint, a third straight line on the horizontal side is obtained, and the third straight line is used as the x-axis.

[0118] In this embodiment, two endpoints are found at both ends of the second straight line: the third endpoint and the fourth endpoint. Optionally, when the second straight line is a straight line on the vertical side, y takes the minimum value, and it is substituted into y = m1x + n1 to obtain the third endpoint. When y takes the maximum value, it is substituted into y = m1x + n1 to obtain the fourth endpoint. Among multiple data points in the three-dimensional image, the three-dimensional coordinates of the third endpoint are searched according to the two-dimensional coordinates of the third endpoint to obtain the three-dimensional coordinates of the third endpoint. Among multiple data points in the three-dimensional image, the three-dimensional coordinates of the fourth endpoint are searched according to the two-dimensional coordinates of the fourth endpoint to obtain the three-dimensional coordinates of the fourth endpoint. The fourth straight line on the vertical side is obtained based on the three-dimensional coordinates of the third endpoint and the three-dimensional coordinates of the fourth endpoint, and the fourth straight line is used as the y-axis.

[0119] When the first residual is less than the second residual, the third line is taken as the x-axis. The first direction vector of the third line is obtained according to the equation of the third line. The cross product of the normal vector of the first plane and the first direction vector is calculated to obtain the first vector. The first vector is taken as the final y-axis.

[0120] When the first residual is greater than or equal to the second residual, the fourth line is used as the y-axis. The second direction vector of the fourth line is obtained based on its equation. The cross product of the normal vector of the first plane and the second direction vector is calculated to obtain the second vector, which is then used as the final x-axis. A second coordinate set is determined based on the first region. The first line and the first residual are determined based on multiple second coordinate sets, a second preset equation, and the least squares method. The second line and the second residual are determined based on multiple second coordinate sets, a third preset equation, and the least squares method. The origin is determined based on the first and second lines. The second and third axes are determined based on the first plane and the 3D image, improving the accuracy and efficiency of obtaining the second and third axes. A third line in the first direction is determined based on the first and second endpoints. A fourth line in the second direction is determined based on the third and fourth endpoints. The second and third axes are determined based on the first residual, the second residual, the third line, and the fourth line, further improving the accuracy and efficiency of obtaining the second and third axes. Please refer to [reference needed]. Figure 3 , Figure 3 This is a schematic diagram of a mounting frame supporting a photovoltaic panel, provided as an embodiment of this application. This diagram is for illustrative purposes only and should not be considered a limitation of this application. The diagram includes a first photovoltaic panel 301, a first side 302, a support column 303, a mounting frame 304, a second axis 305, a third axis 306, and a first axis 307. The mounting frame 304 includes multiple support columns 303, and multiple first photovoltaic panels 301 can be mounted on the mounting frame 304. Each first photovoltaic panel 301 includes a first side 302. The second axis 305, the third axis 306, and the first axis 307 intersect each other. Optionally, the first axis 307 is the z-axis, the second axis 305 is the x-axis, and the third axis 306 is the y-axis.

[0121] S205: Determine the rotation matrix based on the origin, the first axis, the second axis, and the third axis.

[0122] The rotation matrix is ​​determined based on the origin, the first axis, the second axis, and the third axis, including:

[0123] Find the three-dimensional image based on the two-dimensional coordinates of the origin, and obtain the three-dimensional coordinates of the origin;

[0124] The first coordinates of the origin are determined based on the three-dimensional coordinates of the origin and the first plane.

[0125] The rotation matrix is ​​determined based on the first coordinate, the first axis, the second axis, and the third axis.

[0126] In this embodiment of the application, among multiple data points in a three-dimensional image, the three-dimensional coordinates of the origin are obtained by searching among them based on the two-dimensional coordinates of the origin. The three-dimensional coordinates of the origin are then projected onto a first plane to obtain the first coordinates of the origin.

[0127] Using the origin as a translation vector, which describes the linear distance the photovoltaic panel to be installed moves from its original position to its target position, the X-axis, Y-axis, and Z-axis vectors are arranged in columns to form a first matrix. This first matrix serves as the rotation matrix, describing the rotation of the photovoltaic panel about its axes in three-dimensional space. This rotation matrix is ​​output to the robotic arm, which, based on the translation vector and rotation matrix, calculates the corner coordinates and rotation angle of the photovoltaic panel in its coordinate system and places it at the target position within the target area. Thus, determining the rotation matrix based on the first coordinate, first axis, second axis, and third axis improves the accuracy and efficiency of obtaining the rotation matrix.

[0128] S206: Install the photovoltaic panels to be installed to the target location in the target area according to the origin and rotation matrix.

[0129] In this embodiment, the photovoltaic panel installation device further includes a robotic arm. The processor of the photovoltaic panel installation device controls the robotic arm to install the photovoltaic panels to be installed at the target position in the target area. The robotic arm may include an upper robotic arm, a lower robotic arm, a first rotating shaft, and a second rotating shaft. The first rotating shaft connects the upper robotic arm and the installation body of the photovoltaic panel installation device, and is used to rotate the upper robotic arm to move it relative to the installation body. The second rotating shaft connects the upper robotic arm and the lower robotic arm, and is used to rotate the lower robotic arm to rotate it relative to the upper robotic arm. The path formed by the first rotating shaft rotating the upper robotic arm and / or the second rotating shaft rotating the lower robotic arm can be referred to as the movement path.

[0130] In this embodiment, the robotic arm may further include a gripper connected to the upper robotic arm. This gripper is used to grasp the photovoltaic panel to be installed and can also be used to place the photovoltaic panel. For example, the gripper grasps the photovoltaic panel and maintains the grasp until the photovoltaic panel reaches its target area. The gripper may include multiple suction cups to hold the photovoltaic panel in place using suction force. Alternatively, the gripper may include multiple magnets to attract the photovoltaic panel, etc. This application does not limit the type of gripper.

[0131] In this embodiment, the robotic arm may further include a movable frame connected to the upper robotic arm and the gripper. This movable frame is used to move the gripper to extend the position of the upper robotic arm, allowing the gripper to move to a more distant position. The movable frame can be implemented using a servo motor.

[0132] In this embodiment, the robotic arm may further include a turntable connected to a gripper. The turntable can be used to rotate the gripper by an angle, for example, by 30 degrees, so that the gripper can grasp the photovoltaic panel to be installed without moving the upper and lower robotic arms.

[0133] The above structure is just an example. The robotic arm may also include other devices, and the processor of the paving equipment may also control other components in the paving equipment (such as cameras) to implement the corresponding methods.

[0134] Specifically, when the target component is a feature component, the translation vector and rotation matrix are input into the control system of the robotic arm. The control system typically receives and parses this data. Based on the translation vector, the new position of the photovoltaic panel to be installed in the coordinate system of the robotic arm is calculated. Then, combined with the original size and shape of the photovoltaic panel, the coordinates of its four corner points (or more) on the mounting frame are calculated. The rotation matrix is ​​applied to the calculated coordinates of the four corner points to obtain the new coordinates of the photovoltaic panel after rotation. Specifically, the coordinates of each corner point are multiplied by the rotation matrix to obtain the coordinates after rotation. Since the rotation matrix itself contains information about the rotation angle, the rotation angles around the X, Y, and Z axes are then extracted by parsing the rotation matrix. These rotation angles are used to guide the rotational movement of the robotic arm.

[0135] For example, let's consider the target component as a feature component. Assume the photovoltaic panel to be installed is a rectangle with four corner points P1, P2, P3, and P4. Follow these steps to calculate and move the photovoltaic panel.

[0136] 1. Apply translation vector. Use translation vector T to calculate the new position of the photovoltaic panel after translation. For each corner point Pi = [xi, yi, zi], its translated coordinates Pi' can be calculated by the following equation: Pi' = Pi + T, that is: [xi', yi', zi'] = [xi, yi, zi] + [Tx, Ty, Tz].

[0137] 2. Obtain the rotation matrix R.

[0138] 3. Apply the rotation matrix R to calculate the new coordinates of the photovoltaic panel to be installed after rotation. For each translated corner point Pi', its rotated coordinates Pi” can be calculated by the following equation: Pi”=R*Pi', that is: [xi”,yi”,zi”]=R*[xi',yi',zi'].

[0139] 4. The robotic arm places the photovoltaic panel to be installed at the target location. The robotic arm includes a gripping device. Based on the calculated rotational corner coordinates Pi", the robotic arm uses the appropriate gripping device to grasp and place the photovoltaic panel at the target location (or the center position and orientation of the photovoltaic panel to be installed determined based on these coordinates). During the movement, the robotic arm may need to avoid obstacles or make fine adjustments to ensure that the photovoltaic panel can be accurately placed.

[0140] To give another example, let's consider the target component as a feature component. Suppose the photovoltaic panel to be installed is a square with a side length of 1 meter, and the coordinates of its four corner points in its original position are: P1 = [0, 0, 0], P2 = [1, 0, 0], P3 = [1, 1, 0], P4 = [0, 1, 0]. Then, calculate and move the photovoltaic panel to be installed according to the following steps:

[0141] 1. Applying the translation vector, assuming the translation vector is T = [2, 3, 1], the coordinates of the translated corner point are:

[0142] P1'=[0,0,0]+[2,3,1]=[2,3,1],

[0143] P2'=[1,0,0]+[2,3,1]=[3,3,1],

[0144] P3'=[1,1,0]+[2,3,1]=[3,4,1],

[0145] P4'=[0,1,0]+[2,3,1]=[2,4,1].

[0146] 2. Obtain the rotation matrix. Assume the rotation matrix is: R = [[0, -1, 0], [1, 0, 0], [0, 0, 1]], which represents a 90-degree rotation around the Z-axis.

[0147] 3. Apply the rotation matrix. For each translated corner point Pi', calculate its rotated coordinates Pi”: P1”=R*[2, 3, 1]=[0*2+(-1)*3+0*1,1*2+0*3+0*1,0*2+0*3+1*1]=[-3, 2, 1]. Similarly, we can calculate the rotated coordinates of other corner points: P2”=R*[3, 3, 1]=[-3, 3, 1],P3”=R*[3, 4, 1]=[-4, 3, 1],P4”=R*[2, 4, 1]=[-4, 2, 1].

[0148] 4. The robotic arm places the photovoltaic panels to be installed at the target position. The robotic arm includes a gripping device. The robotic arm uses the appropriate gripping device to grip and place the photovoltaic panels at the target position based on the calculated rotational corner coordinates Pi”.

[0149] In this way, the first plane and the first axis are determined based on the first region and the first preset equation, the origin, the second axis and the third axis are determined based on the first plane and the first region, and the rotation matrix is ​​determined based on the origin, the first axis, the second axis and the third axis. The photovoltaic panels to be installed are then installed at the target position in the target region according to the origin and the rotation matrix. This eliminates the need for manual handling and improves the efficiency and accuracy of photovoltaic panel installation.

[0150] Please see Figure 4 , Figure 4 This is a schematic diagram of a photovoltaic panel installation device provided in an embodiment of this application. The photovoltaic panel installation device can be a server, or a device or component within a server, or a computer program, etc. Alternatively, the photovoltaic panel installation device can be a photovoltaic panel installation robot, or a device or component within that robot, or a computer program, etc. This application also provides a computer-readable storage medium storing a computer program. The computer program includes program instructions, which, when executed by a processor, cause the processor to perform some or all of the steps of any of the photovoltaic panel installation methods described in the above method embodiments, such as... Figure 4 As shown, the photovoltaic panel installation device includes a processor 401, a memory 402, a communication interface 404, and at least one program 403. The at least one program 403 is stored in the memory 402 and configured to be executed by the processor 401. The program 403 includes instructions for performing the following steps:

[0151] Acquire the first image of the target area, which contains the target object;

[0152] A first region of the target component is determined based on the first image;

[0153] The first plane and the first axis are determined based on the first region and the first preset equation;

[0154] The origin, second axis, and third axis are determined based on the first plane and the first region, and the first axis, second axis, and third axis intersect each other in pairs;

[0155] The rotation matrix is ​​determined based on the origin, the first axis, the second axis, and the third axis;

[0156] Based on the origin and rotation matrix, the photovoltaic panels to be installed are placed at the target location in the target area.

[0157] In one possible example, program 403 is specifically used to perform the following instructions:

[0158] The first image includes a three-dimensional image and a second image;

[0159] The location of the target component is determined based on the second image;

[0160] Obtain the first mask image of the target component based on its location;

[0161] Detect the contours in the first mask image and determine the contour with the largest area in the first mask image as the first region.

[0162] In one possible example, program 403 is specifically used to perform the following instructions:

[0163] Determine the first coordinate group of the first data point group based on the first region;

[0164] The first plane is determined based on the first coordinate set and the first preset equation;

[0165] The first axis is determined based on the normal vector of the first plane.

[0166] In one possible example, program 403 is specifically used to perform the following instructions:

[0167] A second set of coordinates is determined based on the first region, which contains multiple first target points on the first and second sides.

[0168] The first straight line and the first residual are determined based on multiple second coordinate sets, second preset equations, and the least squares method.

[0169] The second line and the second residual are determined based on multiple second coordinate sets, a third preset equation, and the least squares method. The first line and the second line intersect.

[0170] The origin is determined based on the first and second straight lines;

[0171] The second and third axes are determined based on the first plane and the three-dimensional image.

[0172] In one possible example, program 403 is specifically used to perform the following instructions:

[0173] The target component includes a first photovoltaic panel or a feature component;

[0174] When the target area contains a first photovoltaic panel, the first corner point, second corner point, third corner point, and fourth corner point are determined based on the first area;

[0175] Multiple first target points are determined based on the first corner point, the second corner point, the third corner point, and the fourth corner point;

[0176] When there are features within the target area, a key point detection algorithm is used to process the first area to obtain multiple first target points.

[0177] In one possible example, program 403 is specifically used to perform the following instructions:

[0178] The first endpoint and the second endpoint are determined based on the first straight line;

[0179] The third and fourth endpoints are determined based on the second straight line;

[0180] Determine the third straight line in the first direction based on the first endpoint and the second endpoint;

[0181] The fourth straight line in the second direction is determined based on the third and fourth endpoints, and the first direction intersects the second direction;

[0182] The second and third axes are determined based on the first residual, the second residual, the third line, and the fourth line.

[0183] In one possible example, program 403 is specifically used to perform the following instructions:

[0184] Find the three-dimensional image based on the two-dimensional coordinates of the origin, and obtain the three-dimensional coordinates of the origin;

[0185] The first coordinates of the origin are determined based on the three-dimensional coordinates of the origin and the first plane.

[0186] The rotation matrix is ​​determined based on the first coordinate, the first axis, the second axis, and the third axis.

[0187] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0188] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0189] In the embodiments disclosed in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0190] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0191] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0192] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product.

[0193] Those skilled in the art will understand that, for ease of explanation, Figure 4 Only one memory 402 and processor 401 are shown in the illustration. In a real terminal or server, multiple processors 401 and memory 402 may exist. The memory 402 may also be referred to as a storage medium or storage device, etc., and this embodiment does not impose any limitations on this.

[0194] It should be understood that in this application, processor 401 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 401 may also employ a general-purpose microprocessor, graphics processing unit (GPU), or one or more integrated circuits to execute relevant programs to achieve the functions required by the embodiments of this application.

[0195] Processor 401 can also be an integrated circuit chip with signal processing capabilities. In implementation, each step of this application can be completed through integrated logic circuits in the hardware of processor 401 or through software instructions. The processor 401 described above can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or as execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 402. Processor 401 reads information from memory 402 and, in conjunction with its hardware, completes the functions required by the units included in the methods, apparatus, and storage media of the embodiments of this application.

[0196] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). The memory 402 can also be a Compact Disc Read-Only Memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having an instruction or data structure form and accessible by a computer, but is not limited thereto. The memory 402 can exist independently and be connected to the processor 401 via a bus. Alternatively, the memory 402 can be integrated with the processor 401. The memory 402 can store programs, and when the program stored in the memory 402 is executed by the processor 401, the processor 401 performs the various steps of the determination method in the above embodiments of this application.

[0197] It should be noted that when the processor 401 is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory 402 (memory module) is integrated into the processor 401. It should be noted that the memory 402 described herein is intended to include, but is not limited to, these and any other suitable types of memory 402.

[0198] This application also provides a robot, which includes a photovoltaic panel installation device. In this application, the robot may further include a camera or lens for acquiring images, and a robotic arm. The camera or lens for acquiring images is movably mounted on the robotic arm of the robot that grasps the photovoltaic modules, for real-time image capture of the target area. The photovoltaic panel installation device can be installed on the robot or on a server used to operate the robot. Depending on the installation location of the photovoltaic panel installation device, the robot's operation mode for installing the photovoltaic panels to be installed is divided into two modes: active operation and passive operation. When the photovoltaic panel installation device is installed on the robot, the robot's operation mode for installing the photovoltaic panels is active operation. When the photovoltaic panel installation device is installed on a server used to operate the robot, the robot's operation mode for installing the photovoltaic panels is passive operation.

[0199] The robot's working steps in active operation mode are as follows: acquire a first image of the target area, which contains a target component; determine a first region of the target component based on the first image; determine a first plane and a first axis based on the first region and a first preset equation; determine the origin, a second axis, and a third axis based on the first plane and the first region, with the first axis, second axis, and third axis intersecting each other; determine a rotation matrix based on the origin, the first axis, the second axis, and the third axis; and install the photovoltaic panel to be installed to the target position in the target area according to the origin and the rotation matrix.

[0200] The robot's working steps in passive operation mode are as follows: First, acquire a first image of the target area, which contains the target component. Transmit the first image to a server via wired or wireless connection. The server processes the image to obtain the first region of the target component. Based on the first region and a first preset equation, determine a first plane and a first axis. Based on the first plane and the first region, determine the origin, a second axis, and a third axis, ensuring that the first, second, and third axes intersect each other. Based on the origin, the first axis, the second axis, and the third axis, determine a rotation matrix. Generate motion commands based on the rotation matrix to control the robot in laying photovoltaic panels.

[0201] In this application, a first plane and a first axis are determined based on a first region and a first preset equation. An origin, a second axis, and a third axis are determined based on the first plane and the first region. A rotation matrix is ​​determined based on the origin, the first axis, the second axis, and the third axis. The photovoltaic panels to be installed are then installed at the target position in the target region according to the origin and the rotation matrix. This eliminates the need for manual handling and improves the efficiency and accuracy of photovoltaic panel installation.

[0202] In this application, the robot may also include a 3D camera, which can extract a first region of the first photovoltaic panel by combining the 3D image and the photovoltaic panel installation device, and calculate the pose of the first photovoltaic panel. The main steps include: First, obtaining the coordinates of the photovoltaic panel in 3D space based on the 3D image of the first photovoltaic panel. Next, establishing a correspondence between 3D and 2D based on the corner points, line equations, and coordinates of the photovoltaic panel in 3D space. To ensure the correct correspondence between 3D and 2D points, a feature matching algorithm can be used to automatically establish the correspondence. Obtaining the camera's intrinsic parameter matrix K, including focal length and principal point. Then, solving for the pose of the first photovoltaic panel from the corresponding 3D and 2D points, the pose of the first photovoltaic panel includes the rotation matrix and translation vector of the first photovoltaic panel relative to the camera. For example, a perspective triangulation algorithm combined with a random sampling consensus algorithm can be used. The main steps are to obtain the coordinates (A, B, C, D) of four 3D points in the world coordinate system, the coordinates (a, b, c, d) of the corresponding 2D corner points in the image, and the camera intrinsic parameter matrix K. The four angles (α, β, γ, ι) are calculated using the law of cosines, and the lengths of PA, PB, PC, and PD are solved using the Wu elimination method. The coordinates of A, B, C, and D in the camera coordinate system are then calculated based on these lengths. The pose (rotation matrix R and translation vector t) is solved through a transformation from the world coordinate system to the camera coordinate system.

[0203] The robot adjusts the pose of the photovoltaic panel to be installed based on the pose of the first photovoltaic panel, so that the vertical and horizontal edges of the photovoltaic panel to be installed correspond to the first photovoltaic panel, thereby successfully installing it at the target position in the target area, improving the accuracy and efficiency of the robot in installing the photovoltaic panel to be installed.

[0204] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0205] In implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in processor 401 or by instructions in software form. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by the hardware processor, or by a combination of hardware and software modules in processor 401. The software modules can be located in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 402. Processor 401 reads information from memory 402 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, these will not be described in detail here.

[0206] Those skilled in the art will recognize that the various illustrative logical blocks (ILBs) and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0207] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer-programmed program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a processor, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic) or wireless (e.g., infrared, wireless, microwave, etc.) means, or from one website, computer, server, or data center to a mobile phone processor via a wired means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0208] In the description of the embodiments of the present invention, it should be noted that if the orientation or positional relationship of the terms "center", "upper", "lower", "left", "right", "horizontal", "inner", "outer" and other indicators are based on the orientation or positional relationship of the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0209] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0210] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for installing photovoltaic panels, characterized in that, include: Acquire a first image of a target region, wherein the target region contains a target component; A first region of the target component is determined based on the first image; The first plane and the first axis are determined based on the first region and the first preset equation; The origin, second axis, and third axis are determined based on the first plane and the first region, and the first axis, second axis, and third axis intersect each other in pairs; The rotation matrix is ​​determined based on the origin, the first axis, the second axis, and the third axis; The photovoltaic panels to be installed are placed at the target positions in the target area based on the origin and the rotation matrix.

2. The photovoltaic panel installation method according to claim 1, characterized in that, Determining the first region of the target component based on the first image includes: The first image includes a three-dimensional image and a second image; The position of the target component is determined based on the second image; Obtain the first mask image of the target component based on its position; The contours in the first mask image are detected, and the contour with the largest area in the first mask image is determined as the first region.

3. The photovoltaic panel installation method according to claim 2, characterized in that, The step of determining the first plane and the first axis based on the first region and the first preset equation includes: Based on the first region, determine the first coordinate group of the first data point group; The first plane is determined based on the first coordinate set and the first preset equation; The first axis is determined based on the normal vector of the first plane.

4. The photovoltaic panel installation method according to claim 3, characterized in that, The step of determining the origin, the second axis, and the third axis based on the first plane and the first region, wherein the first axis, the second axis, and the third axis intersect each other pairwise, includes: Based on the first region, a second set of coordinates is determined for multiple first target points on the first and second sides; The first straight line and the first residual are determined based on multiple second coordinate sets, second preset equations, and least squares method; The second straight line and the second residual are determined based on multiple sets of the second coordinate system, the third preset equation, and the least squares method. The first straight line and the second straight line intersect. The origin is determined based on the first straight line and the second straight line; The second axis and the third axis are determined based on the first plane and the three-dimensional image.

5. The photovoltaic panel installation method according to claim 4, characterized in that, The second coordinate group for determining multiple first target points on the first and second sides based on the first region includes: The target component includes a first photovoltaic panel or a feature component; When the first photovoltaic panel is present in the target area, the first corner point, the second corner point, the third corner point, and the fourth corner point are determined based on the first area; Multiple first target points are determined based on the first corner point, the second corner point, the third corner point, and the fourth corner point; When the target area contains the feature, the first area is processed using a key point detection algorithm to obtain multiple first target points.

6. The photovoltaic panel installation method according to claim 4, characterized in that, Determining the second axis and the third axis based on the first plane and the three-dimensional image includes: Determine the first endpoint and the second endpoint based on the first straight line; The third and fourth endpoints are determined based on the second straight line; A third straight line in the first direction is determined based on the first endpoint and the second endpoint; A fourth straight line in the second direction is determined based on the third endpoint and the fourth endpoint, wherein the first direction intersects the second direction; The second axis and the third axis are determined based on the first residual, the second residual, the third line, and the fourth line.

7. The photovoltaic panel installation method according to claim 6, characterized in that, Determining the rotation matrix based on the origin, the first axis, the second axis, and the third axis includes: The three-dimensional coordinates of the origin are obtained by searching the three-dimensional image based on the two-dimensional coordinates of the origin. The first coordinates of the origin are determined based on the three-dimensional coordinates of the origin and the first plane; The rotation matrix is ​​determined based on the first coordinate, the first axis, the second axis, and the third axis.

8. A photovoltaic panel installation device, characterized in that, The method includes a processor and a memory, the memory being used to store computer-readable instructions, and the processor being used to invoke the instructions stored in the memory to perform the method of any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium stores a computer program, which includes program instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

10. A robot, characterized in that, The robot includes the photovoltaic panel installation device of claim 8, used for installing the photovoltaic panel.

Images