Photovoltaic module edge position extraction method and device, storage medium and robot

Abstract

The invention discloses a photovoltaic module edge position extraction method and device, a storage medium and a robot, and the method comprises the steps: obtaining an image of a target region, and enabling a photovoltaic module to exist in the target region; a preliminary contour of a photovoltaic module is determined based on the image, the photovoltaic module comprising a plurality of cells, each cell comprising four endpoints. And determining a target endpoint of the plurality of battery pieces in the initial contour based on the image. And determining a final contour of the photovoltaic module according to the target end point. According to the photovoltaic module edge position extraction method provided by the invention, the photovoltaic module in the target area is identified, and the initial contour of the photovoltaic module is determined according to the identified image of the photovoltaic module. And detecting the end point of the battery piece near the initial contour and taking the end point as a target end point for determining the final contour of the photovoltaic module, and calculating the final contour of the photovoltaic module according to the coordinate of the target end point, thereby obtaining the accurate boundary position of the photovoltaic module, and improving the accuracy and working efficiency of paving the photovoltaic module.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a photovoltaic module edge position extraction method and device, a storage medium and a robot. BACKGROUND

[0002] A photovoltaic module paving robot needs to identify the installed photovoltaic panels, and calculate the pose of the next photovoltaic panel to be installed according to their poses. A key step here is to calculate the horizontal edge, vertical edge, and corner point of the reference photovoltaic panel (the leftmost or rightmost photovoltaic panel that has been paved), and calculate the pose of the photovoltaic panel according to the 2D corner point and straight line equation, combined with the 3D image.

[0003] In actual applications, due to complex factors such as camera pose, light (high reflectivity, shadow, etc.), it is difficult to accurately and reliably determine the horizontal edge and vertical edge of the photovoltaic panel. SUMMARY

[0004] The present application provides a photovoltaic module edge position extraction method and device, a storage medium and a robot, which can accurately and reliably determine the horizontal edge and vertical edge of the photovoltaic module.

[0005] In the first aspect, the present application provides a photovoltaic module edge position extraction method, including obtaining an image of a target region, the target region having a photovoltaic module therein. Determining a preliminary contour of the photovoltaic module based on the image, the photovoltaic module including a plurality of cell pieces, each cell piece including four end points. Determining a target end point of the plurality of cell pieces at the preliminary contour based on the image. Determining a final contour of the photovoltaic module according to the target end point.

[0006] In some feasible examples, determining the preliminary contour of the photovoltaic module based on the image includes determining a position of the photovoltaic module according to the image of the target region. Obtaining a mask image of the photovoltaic module according to the position of the photovoltaic module. Detecting a contour in the mask image, and determining the contour with the largest area in the mask image as the preliminary contour of the photovoltaic module.

[0007] In some feasible examples, the preliminary contour includes a first boundary, a second boundary, a third boundary, and a fourth boundary, the first boundary being perpendicular to the second boundary and the fourth boundary, the third boundary being perpendicular to the second boundary and the fourth boundary, determining the target end point of the plurality of cell pieces at the preliminary contour based on the image includes determining positions of the end points of the plurality of cell pieces, the first boundary, the second boundary, the third boundary, and the fourth boundary according to the image. Determining the target end point of the plurality of cell pieces at the first boundary, the second boundary, the third boundary, or the fourth boundary based on the positions of the end points of the plurality of cell pieces, the first boundary, the second boundary, the third boundary, and the fourth boundary.

[0008] In some possible examples, the target endpoints include first-type target endpoints, second-type target endpoints, third-type target endpoints, and fourth-type target endpoints. An endpoint of the cell located at the first boundary is detected as a first-type target endpoint. An endpoint of the cell located at the second boundary is detected as a second-type target endpoint. An endpoint of the cell located at the third boundary is detected as a third-type target endpoint. An endpoint of the cell located at the fourth boundary is detected as a fourth-type target endpoint.

[0009] In some possible examples, a plurality of cell arrays are arranged in a region formed by the first boundary, the second boundary, the third boundary, and the fourth boundary.

[0010] In some possible examples, determining the final contour of the photovoltaic module according to the target endpoints includes obtaining coordinates of the first-type target endpoints, the second-type target endpoints, the third-type target endpoints, and the fourth-type target endpoints. A first straight line equation is obtained according to the coordinates of the first-type target endpoints. A second straight line equation is obtained according to the coordinates of the second-type target endpoints. A third straight line equation is obtained according to the coordinates of the third-type target endpoints. A fourth straight line equation is obtained according to the coordinates of the fourth-type target endpoints. A contour of a closed region formed by the first straight line equation, the second straight line equation, the third straight line equation, and the fourth straight line equation is the final contour of the photovoltaic module.

[0011] In some possible examples, the final contour includes a first corner point, a second corner point, a third corner point, and a fourth corner point. The first corner point is determined according to the first straight line equation and the second straight line equation, the second corner point is determined according to the second straight line equation and the third straight line equation, the third corner point is determined according to the third straight line equation and the fourth straight line equation, and the fourth corner point is determined according to the fourth straight line equation and the first straight line equation.

[0012] In a second aspect, the present application provides a photovoltaic module edge position extraction device, including a processor and a memory, the memory is used to store computer readable instructions, and the processor is used to call the instructions stored in the memory to execute the method in the first aspect.

[0013] In a third aspect, the present application provides a storage medium, the storage medium stores a computer program, the computer program includes program instructions, and the program instructions make the processor execute the method in the first aspect when executed by the processor.

[0014] In a fourth aspect, the present application provides a robot, the robot includes the photovoltaic module edge position extraction device in the second aspect, and is used to lay the photovoltaic module.

[0015] The photovoltaic module edge position extraction method provided by the application extracts the edge position of the photovoltaic module by recognizing the photovoltaic module in the target area, determining the preliminary contour of the photovoltaic module according to the image of the recognized photovoltaic module, detecting the end point of the cell located near the preliminary contour as the target end point for establishing the final contour of the photovoltaic module, and calculating the final contour of the photovoltaic module according to the coordinates of the target end point, so as to obtain the accurate boundary position of the photovoltaic module and improve the accuracy and work efficiency of the photovoltaic module laying. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 The flow chart of the photovoltaic module edge position extraction method provided by the application is shown in the figure.

[0018] Figure 2 The mask image of the photovoltaic module described in the application is shown in the figure.

[0019] Figure 3 The preliminary contour schematic diagram of the photovoltaic module described in the application is shown in the figure.

[0020] Figure 4 The preliminary contour detail schematic diagram of the photovoltaic module described in the application is shown in the figure.

[0021] Figure 5 The final contour schematic diagram of the photovoltaic module described in the application is shown in the figure.

[0022] Figure 6 The schematic diagram of the photovoltaic module edge position extraction device described in the application is shown in the figure.

[0023] LIST OF ELEMENTS

[0024] 1-photovoltaic module, 2-background, 3-first boundary, 4-second boundary, 5-third boundary, 6-fourth boundary, 7-cell, end point-8, 41-rectangular area, 42-visible end point, 43-invisible end point, 44-straight line equation, 9-first corner point, 10-second corner point, 11-third corner point, 12-fourth corner point. DETAILED DESCRIPTION

[0025] The terms "first" and "second" and the like in the description, claims and drawings of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprises", "comprising", "includes", "including" and the like are used synonymously to denote a non-exclusive inclusion (that is, an inclusion not serving as a limitation) of the elements noted, and variants thereof, in the described implementations, to the extent not described otherwise herein in this Detail Description are but some embodiments of the application.

[0026] "at least one" means one or more, "multiple" means two or more, "at least two" means two or three or more, and "and / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or the like means any combination of these items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c".

[0027] The technical solutions of the present application will be described clearly and completely in combination with the drawings of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0028] Please refer to Figure 1 The flowchart of the method for extracting the edge position of the photovoltaic module provided in the present application is as follows:

[0029] S01, obtaining the image of the target area, the target area has a photovoltaic module.

[0030] The target area is an area where photovoltaic modules are to be installed. Before installing the next photovoltaic module, the target area already has a plurality of installed photovoltaic modules. To extract the edge position of the installed photovoltaic module, an image of the target area is first obtained. The image of the target area is an RGB (Red Green Blue) image. In the RGB image, each pixel is represented by three values, corresponding to the three colors of red, green, and blue. For example, a pixel can be represented as (R, G, B), where R, G, and B are integers between 0 and 255. The image of the target area can be obtained by taking a picture with a camera. The RGB image of the target area is converted to a grayscale image, and after applying filtering to reduce noise, an image segmentation algorithm is used to segment the image of each photovoltaic module in the target area. A separate binary mask is generated for each photovoltaic module, serving as a unique identifier for each photovoltaic module. The photovoltaic module located in the lower left corner is selected as the reference photovoltaic module.

[0031] S02, determining a preliminary contour of the photovoltaic module based on the image, the photovoltaic module comprising a plurality of cell pieces, each cell piece comprising four end points.

[0032] In this application, the YOLO (You Only Look Once) instance segmentation algorithm is used to segment the image of the target area to determine the preliminary contour of the photovoltaic module. The main steps are as follows:

[0033] (1) Label the boundary box and corresponding binary mask of each photovoltaic module in the target area. The boundary box of the photovoltaic module includes its width and height. Please refer to Figure 2 The binary mask of the photovoltaic module is a binary image, where each pixel has a value of 0 or 255. In image processing: 0 (black): represents the background 2. 255 (white): represents the photovoltaic module 1. The photovoltaic module is separated from the background by the binary mask.

[0034] (2) Segment the RGB image of the target area according to the labeled boundary box and corresponding binary mask of each photovoltaic module in the target area to obtain the binary mask image of each photovoltaic module.

[0035] (3) Identify the center point of each photovoltaic module. Assume that the coordinate system of the image has the lower left corner as the origin, the X-axis to the right, and the Y-axis upward. Find the center point with the smallest Y-coordinate, i.e., the photovoltaic module located at the lowest position. If there are multiple photovoltaic modules with the same smallest Y-coordinate, select the one with the smallest X-coordinate, i.e., the leftmost photovoltaic module. The photovoltaic module obtained in this way is the photovoltaic module located in the lower left corner.

[0036] (4) reading the binary mask region of the photovoltaic module, detecting all contours in the mask region using a contour detection algorithm, selecting the largest contour and calculating the minimum circumscribed rectangle of the contour as the preliminary contour of the photovoltaic module.

[0037] Please refer to Figure 3 , the preliminary contour of the photovoltaic module includes a first boundary 3, a second boundary 4, a third boundary 5 and a fourth boundary 6, the first boundary 3 is perpendicular to the second boundary 4 and the fourth boundary 6, the third boundary 5 is perpendicular to the second boundary 4 and the fourth boundary 6, and a plurality of cell pieces 7 are arranged in the area formed by the first boundary 3, the second boundary 4, the third boundary 5 and the fourth boundary 6. Due to the reasons of light change, object shielding or complex background, there is a certain range of error in the edge of the preliminary contour of the photovoltaic module generated by the segmentation algorithm, resulting in that the preliminary contour of the photovoltaic module obtained is not complete, and the coordinates of the visible end points of the cell pieces located in the edge range of the preliminary contour need to be detected by the key point detection algorithm, and the coordinates of the invisible end points of the cell pieces are predicted, and then the straight line fitting is carried out based on the complete cell piece end point coordinates to obtain the straight line equation of the first boundary 3, the second boundary 4, the third boundary 5 and the fourth boundary 6. The contour of the closed area composed of the straight line equations of the first boundary 3, the second boundary 4, the third boundary 5 and the fourth boundary 6 is the final contour of the photovoltaic module.

[0038] Please refer to Figure 4 Taking the first boundary as an example, the first boundary generated by the segmentation algorithm can be regarded as a rectangular region 41 with a certain error, and the end points of the cell pieces close to the first boundary are located in the rectangular region 41, and there are some invisible end points 43 due to the reasons of light change, object shielding or complex background. The visible end points 42 in the rectangular region of the first boundary are detected by the key point detection algorithm, and the positions of the invisible end points 43 are predicted to supplement the end points of the cell pieces in the rectangular region 41 of the first boundary. After supplementing the end points of the cell pieces in the rectangular region 41 of the first boundary, the straight line equation 44 of the first boundary is calculated based on the complete cell piece end points. In this way, the straight line equations of the second boundary, the third boundary and the fourth boundary are obtained, and finally the final contour of the photovoltaic module is obtained.

[0039] S03, determining that the end points in the plurality of cell pieces are target end points in the preliminary contour based on the image.

[0040] As shown in Figure 3 , each cell piece 7 has four end points 8, and the end points of the cell pieces located in the rectangular regions of the first boundary, the second boundary, the third boundary and the fourth boundary are taken as the target end points for determining the final contour of the photovoltaic module.

[0041] First, according to the width and height of the preliminary contour of the photovoltaic module, a ROI (Region of Interest, ROI) sub-image containing the photovoltaic module is cut from the RGB image of the target region for the detection of the target endpoint. Using the ROI sub-image to detect the target endpoint for determining the final contour of the photovoltaic module for further analysis can significantly reduce the amount of calculation.

[0042] Then the key point detection algorithm is used to determine the target endpoint in the ROI sub-image. In this application, the YOLO key point detection algorithm is used to detect the target endpoint of the sub-image, and the steps include:

[0043] (1) Determine the position of each cell piece and the preliminary contour, and then identify the positions of the first boundary, the second boundary, the third boundary and the fourth boundary. From the first boundary to the third boundary, the first column of cell pieces close to the first boundary is selected as the first cell piece group to extract the target endpoint. From the third boundary to the first boundary, the first column of cell pieces close to the third boundary is selected as the third cell piece group to extract the target endpoint. From the second boundary to the fourth boundary, the first row of cell pieces close to the second boundary is selected as the second cell piece group to extract the target endpoint. From the fourth boundary to the second boundary, the first row of cell pieces close to the fourth boundary is selected as the fourth cell piece group to extract the target endpoint.

[0044] (2) Compare the positions of the endpoints of each cell piece in the first cell piece group with the first boundary to detect the endpoints of the cell pieces in the first cell piece group falling on the first boundary as the first type of target endpoint. Compare the positions of the endpoints of each cell piece in the second cell piece group with the second boundary to detect the endpoints of the cell pieces in the second cell piece group falling on the second boundary as the second type of target endpoint. Compare the positions of the endpoints of each cell piece in the third cell piece group with the third boundary to detect the endpoints of the cell pieces in the third cell piece group falling on the third boundary as the third type of target endpoint. Compare the positions of the endpoints of each cell piece in the fourth cell piece group with the fourth boundary to detect the endpoints of the cell pieces in the fourth cell piece group falling on the fourth boundary as the fourth type of target endpoint.

[0045] (3) Load the pre-trained YOLO key point detection model, read the ROI sub-image, convert the ROI sub-image into the model input format, use the YOLO key point detection model to infer the missing or blurred target endpoints in the first type of target endpoint, the second type of target endpoint, the third type of target endpoint and the fourth type of target endpoint, predict the target endpoints, and draw the predicted target endpoints on the ROI sub-image.

[0046] The YOLO key point detection algorithm is used to detect the target end points of the preliminary outline on the photovoltaic module, and the invisible target end points caused by occlusion, high reflection bright spots and the like are predicted to provide complete target end points, thereby supplementing the accuracy of the instance segmentation algorithm and ensuring that the system can correctly and accurately find the final outline of the photovoltaic module in the case where the instance segmentation algorithm does not accurately segment the edges.

[0047] In the present application, in consideration of the case where the invisibility of the target end points is due to insufficient light, on the basis of using the YOLO key point detection algorithm, when the RGB image of the target region is obtained by the camera, a light supplementing device such as a flash is used to obtain a sufficient-light RGB image, thereby avoiding the invisibility of the target end points due to insufficient light.

[0048] S04, determining the final outline of the photovoltaic module according to the target end points.

[0049] After the target end points are detected by the key point detection model and the invisible target end points in the region of interest sub-image are supplemented, the straight line equations of the first boundary, the second boundary, the third boundary and the fourth boundary of the photovoltaic module are obtained by numerical fitting according to the coordinates of the complete first type target end points, the second type target end points, the third type target end points and the fourth type target end points, and then the final outline of the photovoltaic module is obtained, and the main steps are as follows:

[0050] (1) Please refer to Figure 5 , a point is selected as the origin, and the coordinates of the point are (0, 0). Starting from the origin, the horizontal axis is taken as the X axis and the vertical axis is taken as the Y axis to establish a coordinate system. The coordinates (x i ,y i ) 1 , i = 1,..., n of the first type target end points, the coordinates (x i ,y i ) 2 , i = 1,..., n of the second type target end points, the coordinates (x i ,y i ) 3 , i = 1,..., n of the third type target end points and the coordinates (x i ,y i ) 4, i = 1,..., n. In this application, the YOLO key point detection algorithm is used to detect the target end points of the image of the photovoltaic module. The YOLO key point detection algorithm detects the key points of the horizontal edges and vertical edges on the photovoltaic panel. The key point detection algorithm has good processing effect on the invisible key points caused by occlusion, high light spots, etc. and can supplement the accuracy of the instance segmentation algorithm. In the case where the instance segmentation algorithm does not accurately segment the edges, the system can correctly and accurately find the edges of the photovoltaic panel.

[0051] (2) The coordinates (x i ,y i ) 1 , i = 1,..., n are preprocessed and regression analysis is performed to obtain the first straight line equation; the coordinates (x i ,y i ) 2 , i = 1,..., n are preprocessed and regression analysis is performed to obtain the second straight line equation. The coordinates (x i ,y i ) 3 , i = 1,..., n are preprocessed and regression analysis is performed to obtain the third straight line equation. The coordinates (x i ,y i ) 4 , i = 1,..., n are preprocessed and regression analysis is performed to obtain the fourth straight line equation.

[0052] In this application, support vector machine (Support Vector Regression, SVR) is used for regression analysis to find a straight line so that all target end points can be as close to this straight line as possible while allowing a certain error. Support vector machine uses insensitive loss function ε to measure the prediction error. Its definition is:

[0053] maxL ε (y, f(x)) = (0, |y - f(x)| - ε)

[0054] In the formula: y is the true value; f(x) is the predicted value; ε is a self-defined parameter representing the allowed error range.

[0055] The objective function is:

[0056]

[0057] The constraint condition is:

[0058]

[0059] where ω is the weight vector; ω T is the transpose of ω; b is the bias term; δ i and are slack variables; C is the regularization parameter, controlling the balance between the complexity of the model and the error.

[0060] Solve the Lagrange dual problem:

[0061]

[0062] The constraints are:

[0063]

[0064] where y i is the sample label, y i ∈(-1, +1); α i , α j , is the Lagrange multiplier; x i , x j are the horizontal coordinates of the target endpoints; K(x i -x j ) is the kernel function, used to measure the similarity of x i and x j .

[0065] After obtaining the Lagrange multiplier, the weight vector ω is calculated, and the calculation formula is as follows:

[0066]

[0067] The bias term b is calculated by the support vector, and for the support vector x s :

[0068] b = y s - ω T x s - ε or b = y s - ω T x s + ε

[0069] where y s represents the true label corresponding to the support vector x s .

[0070] The average value of all support vectors x s is used to calculate b:

[0071]

[0072] where S is the set of support vectors, N s is the number of support vectors.

[0073] The prediction model of the support vector machine can be expressed as:

[0074] f(x)=ω T x+b

[0075] The linear equation is:

[0076] y=ω1x1+ω2x2+......+ω d x d +b

[0077] In the formula, x=(x1, x2, …, x d ) is an input feature vector, ω=(ω1, ω2, … ω d ) is a weight vector, and b is a bias term.

[0078] Taking the first type of target endpoint as an example, it is assumed that the coordinates of the first type of target endpoint are:

[0079]

[0080] After training by the support vector machine, the following is obtained:

[0081] b=0

[0082] The linear equation is:

[0083] y=2x1+0x2+0=2x1

[0084] (3) The contour of the closed region composed of the first linear equation, the second linear equation, the third linear equation and the fourth linear equation is the final contour of the photovoltaic module.

[0085] The final contour further includes four corner points, namely a first corner point 9, a second corner point 10, a third corner point 11 and a fourth corner point 12. The intersection of the first linear equation and the second linear equation is the first corner point 9, the intersection of the second linear equation and the third linear equation is the second corner point 10, the intersection of the third linear equation and the fourth linear equation is the third corner point 11, and the intersection of the fourth linear equation and the first linear equation is the fourth corner point 12.

[0086] In actual operation, the target endpoint of the photovoltaic module edge extracted by the method can achieve a residual error of 0.07 pixels when performing linear fitting. In a conventional processor, such as an RTX 3090 GPU, the single prediction time is less than 1s, which has good real-time performance.

[0087] In the present application, the final contour of the photovoltaic module extracted in combination with the 3D image and the photovoltaic module edge position extraction device can be used to calculate the pose of the photovoltaic module, and the main steps include:

[0088] (1) Obtain the coordinates of the photovoltaic module in 3D space according to the 3D image of the photovoltaic module.

[0089] (2) Establish a 3D-2D correspondence relationship according to the corner points, straight line equations of the photovoltaic module and the coordinates of the photovoltaic module in 3D space. To ensure the correct correspondence between 3D points and 2D points, a feature matching algorithm can be used to automatically establish the correspondence.

[0090] (3) Obtain the intrinsic matrix K of the camera, including the focal length and the principal point.

[0091] (4) Solve the pose of the photovoltaic module from the 3D-2D corresponding points, which includes the rotation matrix and the translation vector of the photovoltaic module relative to the camera. For example, the Perspective-Three-Point (P3P) algorithm combined with the Random Sample Consensus (RANSAC) algorithm can be used to solve the pose of the photovoltaic module from the 3D-2D corresponding points. The main steps are to obtain the coordinates of four 3D points in the world coordinate system (A, B, C, D). The 2D corner points corresponding to these four points in the image are (a, b, c, d), and the camera intrinsic matrix (K) is obtained. Calculate the four angles (α, β, γ, ι) by the cosine law, and solve the lengths of PA, PB, PC, and PD using Wu's elimination method. Calculate the coordinates of A, B, C, and D in the camera coordinate system according to the lengths. Solve the pose (rotation matrix R and translation vector t) by transforming the world coordinate system to the camera coordinate system.

[0092] Please refer to Figure 6 , Figure 6 for a schematic diagram of a photovoltaic module edge position extraction device provided by the present application. The photovoltaic module edge position extraction device can be a server, or a device or component within the server, or a computer program, etc. The photovoltaic module edge position extraction device can also be a photovoltaic module paving robot, or a device or component within the robot, or a computer program, etc. As shown in Figure 6 , the photovoltaic module edge position extraction device includes a processor 401, a communication interface 402, and a memory 403, which can be connected to each other through a bus 404, or connected through other means.

[0093] The processor 401 includes one or more processors, for example, one or more central processing units (CPUs). In the case of a single CPU, the CPU can be a single core processor or a multi-core processor. In the embodiments of the present application, the processor 401 is configured to control the photovoltaic module edge position extraction device to implement the following steps: Figure 1

[0094] The memory 403 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read only memory (EPROM), or a compact disc read-only memory (CD-ROM). The memory 403 is configured to store relevant instructions and data.

[0095] The communication interface 402 is configured to implement communication with other devices. For example, the communication interface of the server is configured to implement communication with the photovoltaic module paving robot, and the communication interface of the photovoltaic module paving robot is configured to implement communication with the server. In the present application, the memory 403 stores computer readable instructions 405, and the processor 401 is configured to invoke the instructions stored in the memory 403. When the photovoltaic module edge position extraction device is a server, the above instructions are configured to perform the following steps:

[0096] (1) Obtain an image of a target region, and the target region has a photovoltaic module.

[0097] (2) Determine a preliminary contour of the photovoltaic module based on the image, and the photovoltaic module includes a plurality of cell pieces, and each cell piece includes four end points.

[0098] (3) Determine target end points of the preliminary contour at which the end points of the plurality of cell pieces are located based on the image.

[0099] (4) Determine a final contour of the photovoltaic module according to the target end points.

[0100] The present application provides a computer readable storage medium, which stores a computer program. The computer program includes program instructions, and the program instructions, when executed by the processor, cause the processor to perform part or all of the steps of any one of the photovoltaic module edge position extraction methods described in the above method embodiments.

[0101] The present application also provides a robot for paving a photovoltaic module, and the robot includes Figure 6 ​The photovoltaic module edge position extraction device is shown. In this application, the robot can also include a camera or a camera for acquiring images, a mechanical arm, a camera for acquiring images, and a camera for acquiring images. The robot grasps the photovoltaic module, and the camera for acquiring images is movably arranged on the mechanical arm of the robot for capturing images of the target area in real time. The photovoltaic module edge position extraction device can be arranged on the robot or on the server for operating the robot. According to the installation position of the photovoltaic module edge position extraction device, the working mode of the robot for laying the photovoltaic module is divided into two modes of active operation and passive operation. When the photovoltaic module edge position extraction device is arranged on the robot, the working mode of the robot for laying the photovoltaic module is active operation. When the photovoltaic module edge position extraction device is arranged on the server for operating the robot, the working mode of the robot for laying the photovoltaic module is passive operation.

[0102] The working steps of the robot in the active operation mode are: acquiring an image of a target area, the target area having a photovoltaic module; determining a preliminary contour of the photovoltaic module based on the image, the photovoltaic module including a plurality of cell pieces, each cell piece including four end points; determining target end points of the end points of the plurality of cell pieces in the preliminary contour based on the image; determining a final contour of the photovoltaic module according to the target end points; and adjusting the grasping of the photovoltaic module to be installed and laying according to the final contour of the photovoltaic module.

[0103] The working steps of the robot in the passive operation mode are: acquiring an image of a target area, the target area having a photovoltaic module; transmitting the image of the target area to the server through wired or wireless transmission, and processing the image by the server to obtain a preliminary contour of the photovoltaic module; determining target end points of the end points of the plurality of cell pieces in the preliminary contour based on the image; determining a final contour of the photovoltaic module according to the target end points; and generating a motion instruction according to the final contour of the photovoltaic module, and controlling the robot to lay the photovoltaic module.

[0104] In this application, the robot including the photovoltaic module edge position extraction device is used, the YOLO instance segmentation algorithm is used to segment the image to obtain the preliminary contour of the photovoltaic module for the subsequent target end point detection. The YOLO key point detection algorithm is used to detect the target end points of the preliminary contour on the photovoltaic module, and the invisible target end points caused by occlusion, high reflection bright spots and the like are predicted to provide complete target end points, which supplements the precision of the instance segmentation algorithm and ensures that the system can correctly and accurately find the final contour of the photovoltaic module in the case that the instance segmentation algorithm does not accurately segment the edge. The support vector machine regression algorithm is used for linear fitting of the target end points of the photovoltaic module, and smaller residual error is obtained in the case of small sample, which has higher precision and improves the installation precision of the robot for laying the photovoltaic module.

[0105] In the present application, the robot can also include a 3D camera, which can combine the 3D image and the final contour of the photovoltaic module extracted by the photovoltaic module edge position extraction device to calculate the pose of the photovoltaic module. The main steps include:

[0106] (1) Obtain the coordinates of the photovoltaic module in the 3D space according to the 3D image of the photovoltaic module.

[0107] (2) Establish a 3D-2D correspondence according to the corner points, straight line equations of the photovoltaic module and the coordinates of the photovoltaic module in the 3D space. To ensure the correct correspondence between the 3D points and the 2D points, a feature matching algorithm can be used to automatically establish the correspondence.

[0108] (3) Obtain the intrinsic matrix K of the camera, including the focal length and the principal point.

[0109] (4) Solve the pose of the photovoltaic module from the 3D-2D corresponding points, which includes the rotation matrix and the translation vector of the photovoltaic module relative to the camera. For example, the Perspective-Three-Point (P3P) algorithm combined with the Random Sample Consensus (RANSAC) algorithm can be used. The main steps are to obtain the coordinates of four 3D points in the world coordinate system (A, B, C, D). The 2D corner points corresponding to the four points in the image have coordinates (a, b, c, d), and the camera intrinsic matrix K. Calculate the four angles (α, β, γ, ι) by the law of cosines, and use the Wu elimination method to solve the lengths of PA, PB, PC, and PD. According to the length, calculate the coordinates of A, B, C, and D in the camera coordinate system. Solve the pose (rotation matrix R and translation vector t) by transforming the world coordinate system to the camera coordinate system.

[0110] The robot adjusts the pose of the photovoltaic module to be installed according to the obtained pose of the photovoltaic module, so that the vertical and horizontal edges of the photovoltaic module to be installed correspond to the boundaries of the final contour of the installed photovoltaic module, thereby smoothly paving the target area, improving the accuracy and work efficiency of the robot in paving the photovoltaic module.

[0111] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0112] In the above embodiments, the description of each embodiment is focused on, and the part not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0113] In several embodiments disclosed in the present application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are only schematic. The division of the units described above is only a logical function division. In actual implementation, another division mode can be adopted. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical or other forms.

[0114] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0115] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software program module.

[0116] The integrated unit, if realized in the form of a software program module and sold or used as an independent product, can be stored in a computer readable memory. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: a U disk, a RAM, a ROM, a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0117] The embodiments of the present application are described in detail above, and the principles and implementation modes of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed. In view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for extracting the edge position of a photovoltaic module, characterized in that: include: Acquiring an image of a target area, wherein the target area has a photovoltaic module; determining a preliminary outline of the photovoltaic assembly based on the image, wherein the photovoltaic assembly includes a plurality of cells, and each cell includes four endpoints; determining, based on the image, that endpoints of the plurality of battery slices are target endpoints of the preliminary outline; A final profile of the photovoltaic assembly is determined according to the target endpoint.

2. The method according to claim 1, characterized in that The determining a preliminary outline of the photovoltaic assembly based on the image includes: determining a position of the photovoltaic assembly based on the image of the target area; Acquiring a mask image of the photovoltaic assembly according to the position of the photovoltaic assembly; The contours in the mask image are detected, and the contour with the largest area in the mask image is determined as the preliminary contour of the photovoltaic module.

3. The method according to claim 1, characterized in that The preliminary outline includes a first boundary, a second boundary, a third boundary, and a fourth boundary, the first boundary being perpendicular to the second boundary and the fourth boundary, and the third boundary being perpendicular to the second boundary and the fourth boundary, and determining, based on the image, that endpoints of the plurality of battery slices are at target endpoints of the preliminary outline includes: determining positions of endpoints of a plurality of battery cells, the first boundary, the second boundary, the third boundary, and the fourth boundary according to the image; A target endpoint where the endpoints of the plurality of battery slices are located at the first boundary, the second boundary, the third boundary, or the fourth boundary is determined based on the positions of the endpoints of the plurality of battery slices, the first boundary, the second boundary, the third boundary, and the fourth boundary.

4. The method according to claim 3, characterized in that The target endpoints include a first type of target endpoint, a second type of target endpoint, a third type of target endpoint and a fourth type of target endpoint; detecting endpoints of the battery cell located at the first boundary as first-type target endpoints; detecting endpoints of the battery slice located at the second boundary as second-type target endpoints; detecting endpoints of the battery cell located at the third boundary as third-type target endpoints; The endpoints of the battery cell located at the fourth boundary are detected as fourth-type target endpoints.

5. The method according to claim 4, characterized in that The plurality of battery cell arrays are arranged in a region formed by the first boundary, the second boundary, the third boundary, and the fourth boundary.

6. The method according to claim 4, characterized in that Determining a final profile of the photovoltaic assembly according to the target endpoint includes: Obtaining the coordinates of the first-category target endpoint, the second-category target endpoint, the third-category target endpoint, and the fourth-category target endpoint; Obtain the first straight line equation according to the coordinates of the first type target endpoint; Obtain the second straight line equation according to the coordinates of the second target endpoint; The equation of the third straight line is obtained according to the coordinates of the endpoints of the third type of target; The fourth straight line equation is obtained according to the coordinates of the fourth target endpoint; The outline of the closed area formed by the first straight line equation, the second straight line equation, the third straight line equation and the fourth straight line equation is the final outline of the photovoltaic assembly.

7. The method according to claim 6, characterized in that The final outline includes a first corner point, a second corner point, a third corner point and a fourth corner point; The first corner point is determined according to the first straight line equation and the second straight line equation, the second corner point is determined according to the second straight line equation and the third straight line equation, the third corner point is determined according to the third straight line equation and the fourth straight line equation, and the fourth corner point is determined according to the fourth straight line equation and the first straight line equation.

8. A photovoltaic module edge position extraction device, characterized in that: The system comprises a processor and a memory, wherein the memory is used to store computer-readable instructions, and the processor is used to call the instructions stored in the memory to execute the method according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 7.

10. A robot, characterized in that: The robot includes the photovoltaic module edge position extraction device according to claim 8, and is used for paving photovoltaic modules.

Images