Photovoltaic module edge position extraction method and device, storage medium and robot

Abstract

The invention discloses a photovoltaic module edge position extraction method and device, a storage medium and a robot, and the method comprises the steps: obtaining an image of a target region, and enabling a photovoltaic module to exist in the target region; and determining the area of the photovoltaic module according to the image. The initial contour of the photovoltaic module is determined according to the area where the photovoltaic module is located, the initial contour is provided with four end points, and a plurality of edge points are arranged between every two adjacent end points. And determining edge points on the initial contour according to the region. And determining the final contour of the photovoltaic module according to the coordinates of the edge points. Therefore, the system can accurately and accurately find the area where the photovoltaic module is located, and the accuracy and the working efficiency of paving the photovoltaic module are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a photovoltaic module edge position extraction method and device, a storage medium and a robot. BACKGROUND

[0002] The photovoltaic module paving robot needs to identify the pose of the installed photovoltaic panel during work, and further calculate the pose of the next photovoltaic panel to be installed. The key step to calculate the pose of the reference photovoltaic panel (the leftmost or rightmost photovoltaic panel that has been paved) is to calculate the horizontal edge, vertical edge, and corner point of the panel.

[0003] In actual application, due to the influence of complex factors such as camera pose, light (high reflection, shadow, etc.), a simple instance segmentation algorithm cannot reliably obtain the accurate edge of the photovoltaic panel. SUMMARY

[0004] The present application provides a photovoltaic module edge position extraction method and device, a storage medium and a robot, which can accurately calculate the edge straight line equation of the photovoltaic module.

[0005] In a first aspect, the present application provides a photovoltaic module edge position extraction method, comprising:

[0006] Obtaining an image of a target region, the target region having a photovoltaic module therein;

[0007] Determining the region of the photovoltaic module according to the image;

[0008] Determining the preliminary contour of the photovoltaic module according to the region, the preliminary contour having four end points, and a plurality of edge points arranged between each adjacent two end points;

[0009] Determining the edge points on the preliminary contour according to the region;

[0010] Determining the final contour of the photovoltaic module according to the coordinates of the edge points.

[0011] In some feasible examples, determining the preliminary contour of the photovoltaic module according to the region comprises: the region has a plurality of vertices, the vertices form a plurality of contours, and the contours in the region are detected. The detected contours are polygonally approximated to obtain the preliminary contour of the photovoltaic module.

[0012] In some feasible examples, the four end points of the preliminary contour are a first end point, a second end point, a third end point, and a fourth end point, and determining the edge points on the preliminary contour according to the region comprises:

[0013] Determining the positions of the first end point, the second end point, the third end point, and the fourth end point according to the region;

[0014] extracting a vertex between the first endpoint and the second endpoint, the second endpoint and the third endpoint, the third endpoint and the fourth endpoint, and the fourth endpoint and the first endpoint as an edge point.

[0015] In some feasible examples, the edge points are divided into first-type edge points, second-type edge points, third-type edge points, and fourth-type edge points, the coordinates of the first-type edge points are determined by the first endpoint and the second endpoint; the coordinates of the second-type edge points are determined by the second endpoint and the third endpoint; the coordinates of the third-type edge points are determined by the third endpoint and the fourth endpoint; and the coordinates of the fourth-type edge points are determined by the fourth endpoint and the first endpoint.

[0016] In some feasible examples, the edge points are divided into first-type edge points, second-type edge points, third-type edge points, and fourth-type edge points, and the final contour of the reference photovoltaic module is determined according to the coordinates of the edge points, including:

[0017] The first-type edge points, the second-type edge points, the third-type edge points, and the fourth-type edge points are respectively preprocessed to obtain first-type final points, second-type final points, third-type final points, and fourth-type final points.

[0018] A first straight line equation is obtained according to the coordinates of the first-type final points, a second straight line equation is obtained according to the coordinates of the second-type final points, a third straight line equation is obtained according to the coordinates of the third-type final points, and a fourth straight line equation is obtained according to the coordinates of the fourth-type final points.

[0019] The contour of the closed region composed of the first straight line equation, the second straight line equation, the third straight line equation, and the fourth straight line equation is the final contour of the photovoltaic module.

[0020] In some feasible examples, the first-type edge points, the second-type edge points, the third-type edge points, and the fourth-type edge points are preprocessed to obtain the first-type final points, the second-type final points, the third-type final points, and the fourth-type final points, including:

[0021] A specified threshold is determined, a first pre-straight line is obtained according to the first-type edge points, a second pre-straight line is obtained according to the second-type edge points, a third pre-straight line is obtained according to the third-type edge points, and a fourth pre-straight line is obtained according to the fourth-type edge points.

[0022] Edge points in the first-type edge points with a distance greater than the specified threshold to the first pre-straight line are removed to obtain the first-type final points.

[0023] Edge points in the second-type edge points with a distance greater than the specified threshold to the second pre-straight line are removed to obtain the second-type final points.

[0024] Edge points in the third-type edge points with a distance greater than the specified threshold to the third pre-straight line are removed to obtain the third-type final points.

[0025] The fourth type of edge points are pruned to obtain fourth type of final points.

[0026] In some possible examples, the final contour includes a first corner point, a second corner point, a third corner point and a fourth corner point.

[0027] The first corner point is determined according to the first line equation and the second line equation, the second corner point is determined according to the second line equation and the third line equation, the third corner point is determined according to the third line equation and the fourth line equation, and the fourth corner point is determined according to the fourth line equation and the first line equation.

[0028] In a second aspect, the present application provides a photovoltaic module edge position extraction device, comprising a processor and a memory, the memory is used to store computer readable instructions, and the processor is used to call the instructions stored in the memory to execute the method of the first aspect.

[0029] In a third aspect, the present application provides a storage medium, the storage medium stores a computer program, the computer program includes program instructions, and the program instructions make the processor execute the method of the first aspect when executed by the processor.

[0030] In a fourth aspect, the present application provides a robot, the robot includes the photovoltaic module edge position extraction device of the second aspect, and is used to pave the photovoltaic module.

[0031] The photovoltaic module edge position extraction method provided by the present application acquires an image of a target region with a photovoltaic module, determines a region of the photovoltaic module according to the image, determines a preliminary contour of the photovoltaic module according to the region, the preliminary contour has four end points, and a plurality of edge points are arranged between each adjacent two end points. The edge points on the preliminary contour are determined according to the region. The final contour of the reference photovoltaic module is determined according to the coordinates of the edge points, so that the edge line equation of the photovoltaic module is obtained accurately, and the accuracy and work efficiency of paving the photovoltaic module are improved. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0033] Figure 1 The flow chart of the photovoltaic module edge position extraction method provided by the present application is shown in the figure.

[0034] Figure 2 The image of the region of the photovoltaic module described in the present application is shown in the figure.

[0035] Figure 3 Preliminary profile of a photovoltaic module according to the present application;

[0036] Figure 4 First linear fit of a photovoltaic module according to the present application;

[0037] Figure 5 Second linear fit of a photovoltaic module according to the present application;

[0038] Figure 6 Final profile of a photovoltaic module according to the present application;

[0039] Figure 7 Schematic view of an edge position extraction device for a photovoltaic module according to the present application.

[0040] Reference Signs

[0041] 1 - region, 2 - vertex, 3 - preliminary profile, 4 - first end point, 5 - second end point, 6 - third end point, 7 - fourth end point, 8 - edge point, 9 - first pre-straight line, 10 - second pre-straight line, 11 - third pre-straight line, 12 - fourth pre-straight line, 13 - first straight line equation, 14 - second straight line equation, 15 - third straight line equation, 16 - fourth straight line equation, 17 - final profile, 18 - first corner point, 19 - second corner point, 20 - third corner point, 21 - fourth corner point, 401 - processor, 402 - communication interface, 403 - memory, 404 - bus, 405 - computer readable instructions. DETAILED DESCRIPTION

[0042] The terms "first" and "second" and the like in the description, claims and drawings of the application merely mean different objects without necessarily implying a particular order. Furthermore, the terms "comprising" and "including" and any of their derivatives, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that comprises a list of steps or units is not limited to the listed steps or units, but can optionally further include other steps or units not listed, or can optionally further include other steps or units inherent to such processes, methods, products, or apparatus.

[0043] "At least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three and more than three, and "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c".

[0044] The following will clearly and completely describe the technical solution of this application in conjunction with the drawings of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0045] See Figure 1 , which is a flow chart of the photovoltaic module edge position extraction method provided in this application:

[0046] S01, acquiring an image of a target area, where there are photovoltaic modules.

[0047] The target area is the area where photovoltaic modules are to be installed. Before the next photovoltaic module is installed, the target area already contains multiple installed photovoltaic modules. To extract the edge positions of the installed photovoltaic modules, an image of the target area is first acquired. The image of the target area is an RGB (Red, Green, Blue) image. In an RGB image, each pixel is represented by three values, corresponding to the colors red, green, and blue. For example, a pixel can be represented as (R, G, B), where R, G, and B are integers between 0 and 255. The image of the target area can be obtained by taking a photo with a camera. The RGB image of the target area is converted to a grayscale image and filtered to reduce noise. Then, an image segmentation algorithm is used to segment the image of each photovoltaic module in the target area. A separate binary mask is generated for each photovoltaic module, which serves as a unique identifier for each photovoltaic module. The photovoltaic module in the lower left corner is selected as the reference photovoltaic module.

[0048] S02: Determine the area where the photovoltaic module is located based on the image.

[0049] In this application, the YOLO (You Only Look Once, YOLO) instance segmentation algorithm is used to segment the image of the target area. The main steps are as follows:

[0050] (1) Label the bounding box and corresponding binary mask of each photovoltaic component in the target region. The bounding box of the photovoltaic component includes its width and height. The binary mask of the photovoltaic component is a binary image, where the value of each pixel is 0 or 255. In image processing: 0 (black): represents the background. 255 (white): represents the photovoltaic component. Through the binary mask, the photovoltaic component is separated from the background.

[0051] (2) Segment the RGB image of the target region according to the bounding box and corresponding binary mask of each photovoltaic component in the target region labeled, to obtain the binary mask image of each photovoltaic component.

[0052] (3) Identify the center point of each photovoltaic component. Assume that the coordinate system of the image has the lower left corner as the origin, the X-axis to the right, and the Y-axis upward. Find the center point with the smallest Y coordinate, i.e. the photovoltaic component located at the bottom. If there are multiple photovoltaic components with the same smallest Y coordinate, select the one with the smallest X coordinate, i.e. the leftmost photovoltaic component. The photovoltaic component obtained in this way is the one located at the lower left corner.

[0053] (4) Read the binary mask region of the photovoltaic component, and use the matting algorithm to perform matting in the mask region to obtain the refined region of the photovoltaic component. In this application, PP-Matting (Paddle Paddle-Matting) matting algorithm is used for matting. PP-Matting uses two main branches: semantic context branch (Semantic Context Branch, SCB) and high-resolution detail branch (High-Resolution Detail Branch, HRDB). SCB is used to extract the high-level semantic information of the picture, while HRDB is used to accurately extract the details of the transition area under the guidance of semantic information. This double processing mechanism makes the matting result not only maintain high accuracy, but also retain fine details.

[0054] S03, determine the preliminary contour of the photovoltaic component according to the region, the preliminary contour has four end points, and a plurality of edge points are arranged between each adjacent two end points.

[0055] Please refer to Figure 2The region where the photovoltaic module is located 1 is obtained by matting from the target region, and the region where the photovoltaic module is located 1 has a plurality of vertices 2, which form a plurality of contours. In this application, the PP-Matting (Paddle Paddle-Matting) matting algorithm is used for matting. The mask image of the target region is preprocessed, the pre-trained PP-Matting model is loaded, the preprocessed image is input into the model for inference, and the output of the model is post-processed to obtain a clearer output result. The obtained model output separates the foreground from the original image and synthesizes it with a new background. The PP-Matting matting algorithm is used to extract the accurate region where the photovoltaic module is located 1. Compared with the instance segmentation algorithm, it has higher accuracy and can supplement the accuracy of the instance segmentation algorithm. In the case where the instance segmentation algorithm does not accurately segment the region where the photovoltaic module is located 1, it ensures that the system can correctly and accurately find the region where the photovoltaic module is located 1.

[0056] The preliminary contour 3 of the photovoltaic module is extracted from the region where the photovoltaic module is located 1, and a plurality of vertices are distributed on the preliminary contour 3 of the photovoltaic module. In this application, the polygon approximation method is used to determine the preliminary contour from the boundary points of the original shape of the region where the photovoltaic module is located 1. By adjusting the vertex position of the polygon, the perimeter is gradually reduced, and when the perimeter no longer significantly reduces or reaches a preset number of iterations, the algorithm is terminated. Please refer to Figure 3 The preliminary contour 3 of the photovoltaic module extracted from the region where the photovoltaic module is located 1 is used to further obtain a more accurate contour of the photovoltaic module. Four end points on the preliminary contour and edge points between each two adjacent end points need to be determined. The edge points are divided into four categories, four straight line equations are calculated according to the coordinates of the four categories of edge points, the distance from each edge point in each category of edge points to the corresponding straight line is calculated, the points with a distance greater than a specified threshold are removed, and then a second straight line fitting is performed to obtain four straight line equations. The outer contour of the closed region composed of the four straight line equations is the final contour of the photovoltaic module.

[0057] S04, according to the region where the photovoltaic module is located 1, the edge points on the preliminary contour are determined.

[0058] Please refer to Figure 3, the end points on the preliminary contour 3 include a first end point 4, a second end point 5, a third end point 6 and a fourth end point 7. There are edge points 8 distributed between the first end point 4 and the second end point 5, the second end point 5 and the third end point 6, the third end point 6 and the fourth end point 7, and the fourth end point 7 and the first end point 4. In the present application, the first end point 4, the second end point 5, the third end point 6 and the fourth end point 7 on the preliminary contour are determined by a convex hull algorithm. First, the center point of the preliminary contour is determined according to the region, the distance of each vertex on the preliminary contour to the center point is calculated, and the four vertices with the largest distance are selected as the first end point 4, the second end point 5, the third end point 6 and the fourth end point 7. After determining the positions of the first end point 4, the second end point 5, the third end point 6 and the fourth end point 7, the edge points distributed between the first end point 4 and the second end point 5 are taken as the first type of edge points, the edge points distributed between the second end point 5 and the third end point 6 are taken as the second type of edge points, the edge points distributed between the third end point 6 and the fourth end point 7 are taken as the third type of edge points, and the edge points distributed between the fourth end point 7 and the first end point 4 are taken as the fourth type of edge points. A pre-trained point detection model is loaded, an image of the preliminary contour is read, the image of the preliminary contour is converted into a model input format, and a point detection algorithm is used to extract the coordinates of the first type of edge points, the second type of edge points, the third type of edge points and the fourth type of edge points.

[0059] S05, determining the final contour of the photovoltaic module according to the coordinates of the edge points.

[0060] To obtain an accurate final contour of the photovoltaic module, four pre-straight line equations are calculated according to the coordinates of the first type of edge points, the second type of edge points, the third type of edge points and the fourth type of edge points, the distance of each edge point in each type of edge points to the corresponding pre-straight line is calculated, the points with a distance greater than a specified threshold are removed, and then a second straight line fitting is performed to obtain four straight line equations. The outer contour of the closed area composed of the four straight line equations is the final contour of the photovoltaic module. The main steps are as follows:

[0061] (1) Please refer to Figure 4 , a first pre-straight line 9 is obtained according to the first type of edge points, a second pre-straight line 10 is obtained according to the second type of edge points, a third pre-straight line 11 is obtained according to the third type of edge points, and a fourth pre-straight line 12 is obtained according to the fourth type of edge points.

[0062] (2) Determine the specified threshold. In the present application, the specified threshold is determined according to the standard deviation method. Taking the first type of edge points as an example, the distance of all points in the first type of edge points to the first pre-straight line is calculated, the average and the standard deviation of all distances in the first type of edge points are calculated, and the specified threshold is calculated as follows:

[0063] Q = μ + kσ

[0064] Where μ is the mean of all distances, σ is the standard deviation of all distances, and k is a multiple of the standard deviation. When k = 2, approximately 95% of the data points are excluded, which is suitable for most cases. When k = 3, 99.7% of the data points are excluded, which is suitable for more stringent elimination of outliers.

[0065] (3) Eliminate edge points whose distance to the first pre-straight line is greater than a specified threshold from the first category edge points to obtain the first category final points;

[0066] (4) Eliminate the edge points of the second category whose distance to the second pre-straight line is greater than a specified threshold, and obtain the final points of the second category;

[0067] (5) Eliminate the edge points of the third category whose distance to the third pre-straight line is greater than the specified threshold, and obtain the final points of the third category;

[0068] (6) Eliminate the edge points of the fourth category whose distance to the fourth pre-straight line is greater than the specified threshold, and obtain the final points of the fourth category.

[0069] (7) See Figure 5 , according to the coordinates of the final point of the first type, the first straight line equation 13 is obtained, according to the coordinates of the final point of the second type, the second straight line equation 14 is obtained, according to the coordinates of the final point of the third type, the third straight line equation 15 is obtained, and according to the coordinates of the final point of the fourth type, the fourth straight line equation 16 is obtained;

[0070] (8) The outline of the closed area formed by the first straight line equation 13, the second straight line equation 14, the third straight line equation 15 and the fourth straight line equation 16 is the final outline 17 of the photovoltaic module.

[0071] In this application, the least squares method is used to perform straight line fitting on the first type edge points, the second type edge points, the third type edge points and the fourth type edge points to obtain the first pre-straight line, the second pre-straight line, the third pre-straight line and the fourth pre-straight line. Taking the first type edge points as an example, assuming that the coordinates of the first type edge points are (x i ,y i ), to obtain the first linear equation y = ax + b, first calculate ∑x i ,∑y i 、 ∑x i y i , then calculate:

[0072]

[0073] After obtaining the first predicted straight line y=ax+b, the distance from each point in the first type of edge points to the first predicted straight line is calculated, and points with distances greater than a specified threshold are eliminated, and the remaining points are the first final points.

[0074] The coordinates of the first final point of the first type (x i ,y i ) 1 ,i=1,.....,n preprocessing, calculate ∑X i ,∑Y i 、 ∑X i Y i , then calculate:

[0075]

[0076] The least square method is used to perform linear fitting on the first final point again to obtain the first linear equation Y=AX+B.

[0077] See Figure 6 The final contour also includes four corner points, namely the first corner point 18, the second corner point 19, the third corner point 20, and the fourth corner point 21. The intersection of the first straight line equation 13 and the second straight line equation 14 is the second corner point 19, the intersection of the second straight line equation 14 and the third straight line equation 15 is the third corner point 20, the intersection of the third straight line equation 15 and the fourth straight line equation 16 is the fourth corner point 21, and the intersection of the fourth straight line equation 16 and the first straight line equation 13 is the first corner point 18.

[0078] In this application, the pose of the photovoltaic module can be calculated by combining the 3D image and the final contour of the photovoltaic module extracted by the photovoltaic module edge position extraction device. The main steps include:

[0079] (1) Obtain the coordinates of the photovoltaic module in 3D space based on the 3D image of the photovoltaic module.

[0080] (2) Establish a 3D-2D correspondence based on the corner points, line equations, and coordinates of the PV module in 3D space. To ensure the correct correspondence between 3D points and 2D points, a feature matching algorithm can be used to automatically establish the correspondence.

[0081] (3) Obtain the camera’s intrinsic parameter matrix K, including focal length and principal point.

[0082] (4) solving the pose of the photovoltaic module from the 3D-2D corresponding points, the pose of the photovoltaic module including a rotation matrix and a translation vector of the photovoltaic module relative to the camera. For example, the pose of the photovoltaic module can be solved from the 3D-2D corresponding points using a P3P perspective-three-point (P3P) algorithm combined with a random sample consensus (RANSAC) algorithm. The main steps are to obtain the coordinates (A, B, C, D) of four 3D points in the world coordinate system. The 2D corner points corresponding to the four points in the image have coordinates (a, b, c, d), and the camera intrinsic matrix (K) is obtained. The four angles (α, β, γ, ι) are calculated by the cosine law, and the lengths of PA, PB, PC, and PD are solved using Wu's elimination method. The coordinates of A, B, C, and D in the camera coordinate system are calculated according to the lengths. The pose (rotation matrix R and translation vector t) is solved by transforming the world coordinate system to the camera coordinate system.

[0083] Please refer to Figure 7 , Figure 7 A schematic diagram of a photovoltaic module edge position extraction device is provided in the present application. The photovoltaic module edge position extraction device can be a server, or can be a device or component within the server, or a computer program, etc. The photovoltaic module edge position extraction device can also be a photovoltaic module paving robot, or a device or component within the robot, or a computer program, etc. As shown in Figure 7 , the photovoltaic module edge position extraction device includes a processor 401, a communication interface 402, and a memory 403, which can be connected to each other through a bus 404, or can be connected through other means.

[0084] The processor 401 includes one or more processors, such as one or more central processing units (CPUs). In the case of a CPU, the CPU can be a single-core CPU or a multi-core CPU. In the present application, the processor 401 is used to control the photovoltaic module edge position extraction device to implement the embodiments shown in Figure 1 .

[0085] The memory 403 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read only memory (EPROM), or a compact disc read-only memory (CD-ROM), which is used to store relevant instructions and data.

[0086] The communication interface 402 is used to realize communication with other devices. For example, the communication interface of the server is used to realize communication with the photovoltaic module paving robot, and the communication interface of the photovoltaic module paving robot is used to realize communication with the server. In the present application, the memory 403 stores computer readable instructions 405, and the processor 401 is used to call the instructions stored in the memory 403. When the photovoltaic module edge position extraction device is a server, the above instructions are used to execute the following steps:

[0087] (1) obtaining an image of a target region, the target region having a photovoltaic module therein;

[0088] (2) determining a region of the photovoltaic module according to the image;

[0089] (3) determining a preliminary contour of the photovoltaic module according to the region, the preliminary contour having four end points, and a plurality of edge points arranged between each adjacent two end points;

[0090] (4) determining an edge point on the preliminary contour according to the region;

[0091] (5) determining a final contour of a reference photovoltaic module according to the coordinates of the edge point.

[0092] The present application provides a computer readable storage medium, which stores a computer program, the computer program includes program instructions, the program instructions, when executed by the processor, cause the processor to execute part or all steps of any one of the photovoltaic module edge position extraction methods as described in the above method embodiments.

[0093] The present application also provides a robot for paving a photovoltaic module, the robot comprising Figure 7The edge position extraction device of the photovoltaic module is shown. In this application, the robot can also include a camera or a camera for image acquisition, a mechanical arm, a camera or a camera for image acquisition, which is arranged on the mechanical arm of the robot for grabbing the photovoltaic module, for capturing the image of the target area in real time. The edge position extraction device of the photovoltaic module can be arranged on the robot, and can also be arranged on the server for operating the robot. According to the installation position of the edge position extraction device of the photovoltaic module, the working mode of the robot for laying the photovoltaic module is divided into two modes of active operation and passive operation. When the edge position extraction device of the photovoltaic module is arranged on the robot, the working mode of the robot for laying the photovoltaic module is active operation. When the edge position extraction device of the photovoltaic module is arranged on the server for operating the robot, the working mode of the robot for laying the photovoltaic module is passive operation.

[0094] The working steps of the robot in the active operation mode are: acquiring the image of the target area, and the target area has the photovoltaic module. The position area of the photovoltaic module is determined according to the image. The preliminary contour of the photovoltaic module is determined according to the position area, and the preliminary contour has four end points, and a plurality of edge points are arranged between each adjacent two end points. The edge points on the preliminary contour are determined according to the position area. The final contour of the reference photovoltaic module is determined according to the coordinates of the edge points. The photovoltaic module to be installed is adjusted and laid according to the final contour of the photovoltaic module.

[0095] The working steps of the robot in the passive operation mode are: acquiring the image of the target area, and the target area has the photovoltaic module. The image of the target area is transmitted to the server through wired or wireless transmission, and the server processes the image to determine the position area of the photovoltaic module. The preliminary contour of the photovoltaic module is determined according to the position area, and the preliminary contour has four end points, and a plurality of edge points are arranged between each adjacent two end points. The edge points on the preliminary contour are determined according to the position area. The final contour of the reference photovoltaic module is determined according to the coordinates of the edge points. The action instruction is generated according to the final contour of the photovoltaic module, and the robot is controlled to lay the photovoltaic module.

[0096] In the present application, a robot including a photovoltaic module edge position extraction device is used, a YOLO instance segmentation algorithm is used to segment the image, a PP-Matting (Paddle Paddle-Matting) matting algorithm is used for matting, a mask image of the target region is preprocessed, a pre-trained PP-Matting model is loaded, the preprocessed image is input into the model for inference, and the output of the model is post-processed to obtain a clearer output result. The obtained model output separates the foreground from the original image and synthesizes it with a new background to obtain the region where the photovoltaic module is located. The PP-Matting matting algorithm extracts the accurate region where the photovoltaic module is located, has higher accuracy than the instance segmentation algorithm, can supplement the accuracy of the instance segmentation algorithm, and ensures that the system can correctly and accurately find the region where the photovoltaic module is located in the case that the instance segmentation algorithm does not accurately segment the region where the photovoltaic module is located.

[0097] The region where the photovoltaic module is located is approximated by a polygon to obtain a first endpoint 4, a second endpoint 5, a third endpoint 6, and a fourth endpoint 7 on the preliminary contour of the photovoltaic module, so as to determine the preliminary contour of the photovoltaic module. According to the first endpoint 4, the second endpoint 5, the third endpoint 6, and the fourth endpoint 7, the edge points between each two adjacent endpoints are divided into first-type edge points, second-type edge points, third-type edge points, and fourth-type edge points, the coordinates of the first-type edge points, the second-type edge points, the third-type edge points, and the fourth-type edge points are extracted by using a point detection algorithm, and the coordinates of the first-type edge points, the second-type edge points, the third-type edge points, and the fourth-type edge points are subjected to twice linear fitting by using a least square method to obtain a final contour of the photovoltaic module, thereby improving the accuracy of linear fitting.

[0098] In the present application, the robot can also include a 3D camera, which can combine the 3D image and the final contour of the photovoltaic module extracted by the photovoltaic module edge position extraction device to calculate the pose of the photovoltaic module. The main steps include:

[0099] (1) Obtain the coordinates of the photovoltaic module in the 3D space according to the 3D image of the photovoltaic module.

[0100] (2) Establish a 3D-2D correspondence relationship according to the corner points, linear equations of the photovoltaic module, and the coordinates of the photovoltaic module in the 3D space. To ensure that the correspondence relationship between the 3D points and the 2D points is correct, a feature matching algorithm can be used to automatically establish the correspondence relationship.

[0101] (3) Obtain the intrinsic matrix K of the camera, including the focal length and the principal point.

[0102] (4) solving the pose of the photovoltaic module from the 3D-2D correspondences, the pose of the photovoltaic module includes a rotation matrix and a translation vector of the photovoltaic module relative to the camera. For example, a Perspective-Three-Point (P3P) algorithm combined with a Random Sample Consensus (RANSAC) algorithm can be used. The main steps are to obtain the coordinates of four 3D points in the world coordinate system (A, B, C, D). The 2D corner points corresponding to the four points in the image are (a, b, c, d), and the camera intrinsic matrix K is obtained. The four angles (α, β, γ, ι) are calculated by the cosine law, and the lengths of PA, PB, PC, and PD are solved by the Wu elimination method. The coordinates of A, B, C, and D in the camera coordinate system are calculated according to the lengths. The pose (rotation matrix R and translation vector t) is solved by the transformation from the world coordinate system to the camera coordinate system.

[0103] The robot adjusts the pose of the photovoltaic module to be installed according to the obtained pose of the photovoltaic module, so that the vertical edge and the horizontal edge of the photovoltaic module to be installed correspond to the boundary of the final contour of the installed photovoltaic module, thereby smoothly paving the target area, improving the accuracy and work efficiency of the robot in paving the photovoltaic module. In actual operation, the residual error of the photovoltaic panel edge point extracted by the method can reach 0.05 pixels when performing linear fitting.

[0104] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the application is not limited by the action sequence described, because according to the application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the application.

[0105] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0106] In several embodiments disclosed in the present application, it should be understood that the disclosed device can be implemented by other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical or other forms.

[0107] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0108] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software program module.

[0109] The integrated unit, if realized in the form of a software program module and sold or used as an independent product, can be stored in a computer readable memory. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: a U disk, a RAM, a ROM, a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0110] The embodiments of the present application are described in detail above, and the specific examples are applied to the principles and implementation modes of the present application. The above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for extracting the edge position of a photovoltaic module, characterized in that: include: Acquiring an image of a target area, wherein the target area has a photovoltaic module; determining a location of the photovoltaic module according to the image; Determining a preliminary outline of the photovoltaic assembly according to the area, wherein the preliminary outline has four endpoints, and a plurality of edge points are arranged between each two adjacent endpoints; Determine the edge points on the preliminary outline according to the area; A final outline of the photovoltaic module is determined according to the coordinates of the edge points.

2. The method according to claim 1, characterized in that The determining of a preliminary outline of the photovoltaic assembly according to the region includes: There are multiple vertices in the area, and the vertices form multiple contours. The contours in the area are detected; polygonal approximation is performed on the detected contours to obtain a preliminary contour of the photovoltaic component.

3. The method according to claim 1, characterized in that The four endpoints of the preliminary outline are a first endpoint, a second endpoint, a third endpoint, and a fourth endpoint, and determining the edge point on the preliminary outline according to the area includes: Determine the positions of the first endpoint, the second endpoint, the third endpoint, and the fourth endpoint according to the area; The vertex between the first endpoint and the second endpoint, the vertex between the second endpoint and the third endpoint, the vertex between the third endpoint and the fourth endpoint, and the vertex between the fourth endpoint and the first endpoint are extracted as edge points.

4. The method according to claim 3, characterized in that The edge points are divided into first-class edge points, second-class edge points, third-class edge points and fourth-class edge points. The coordinates of the first-class edge points are determined by the first endpoint and the second endpoint; the coordinates of the second-class edge points are determined by the second endpoint and the third endpoint; the coordinates of the third-class edge points are determined by the third endpoint and the fourth endpoint; and the coordinates of the fourth-class edge points are determined by the fourth endpoint and the first endpoint.

5. The method according to claim 3, characterized in that The edge points are divided into first-class edge points, second-class edge points, third-class edge points, and fourth-class edge points, and the final outline of the photovoltaic module is determined according to the coordinates of the edge points, including: Preprocessing the first type edge points, the second type edge points, the third type edge points, and the fourth type edge points respectively to obtain first type final points, second type final points, third type final points, and fourth type final points; Obtain a first straight line equation based on the coordinates of the first type of final point, obtain a second straight line equation based on the coordinates of the second type of final point, obtain a third straight line equation based on the coordinates of the third type of final point, and obtain a fourth straight line equation based on the coordinates of the fourth type of final point; The outline of the closed area formed by the first straight line equation, the second straight line equation, the third straight line equation and the fourth straight line equation is the final outline of the photovoltaic assembly.

6. The method according to claim 5, characterized in that The preprocessing of the first type edge points, the second type edge points, the third type edge points and the fourth type edge points to obtain the first type final points, the second type final points, the third type final points and the fourth type final points comprises: Determine a specified threshold, obtain a first pre-straight line based on the first type of edge points, obtain a second pre-straight line based on the second type of edge points, obtain a third pre-straight line based on the third type of edge points, and obtain a fourth pre-straight line based on the fourth type of edge points; Eliminate edge points of the first category whose distance to the first pre-straight line is greater than the specified threshold from the first category edge points to obtain first category final points; Eliminate edge points of the second category whose distance to the second pre-straight line is greater than the specified threshold from the second category edge points to obtain the second category final points; Eliminate edge points of the third category whose distance to the third pre-straight line is greater than the specified threshold from the edge points of the third category, to obtain final points of the third category; Eliminate edge points of the fourth category whose distance to the fourth pre-straight line is greater than the specified threshold from the fourth category edge points to obtain fourth category final points.

7. The method according to claim 5, characterized in that The final outline includes a first corner point, a second corner point, a third corner point and a fourth corner point; The first corner point is determined according to the first straight line equation and the second straight line equation, the second corner point is determined according to the second straight line equation and the third straight line equation, the third corner point is determined according to the third straight line equation and the fourth straight line equation, and the fourth corner point is determined according to the fourth straight line equation and the first straight line equation.

8. A photovoltaic module edge position extraction device, characterized in that: The system comprises a processor and a memory, wherein the memory is used to store computer-readable instructions, and the processor is used to call the instructions stored in the memory to execute the method according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 7.

10. A robot, characterized in that: The robot includes the photovoltaic module edge position extraction device according to claim 8, and is used for paving photovoltaic modules.

Images