Photovoltaic panel edge detection method and related device

Abstract

The invention discloses a photovoltaic panel edge detection method and a related device, and relates to the field of artificial intelligence, a laser radar and an image acquisition device are installed on a photovoltaic cleaning robot at the same time, in the operation process of the photovoltaic cleaning robot, point cloud data acquired by the laser radar are acquired, point cloud boundary points in the point cloud data are extracted, and the point cloud boundary points are extracted; the method comprises the following steps: acquiring a point cloud boundary point under a radar coordinate system, acquiring a photovoltaic panel image acquired by image acquisition equipment, detecting line features in the photovoltaic panel image, and converting the point cloud boundary point under the radar coordinate system into an image coordinate system to realize linear fitting of the point cloud boundary point by taking the line features as reference under the image coordinate system so as to obtain photovoltaic panel edge information; compared with photovoltaic panel edge detection based on a single camera or a single laser radar, the accuracy of photovoltaic panel edge detection is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a photovoltaic panel edge detection method and related devices. Background Art

[0002] The photovoltaic cleaning robot works on unattended photovoltaic panels. In order to ensure that the photovoltaic cleaning robot can perform cleaning operations safely, it is necessary to ensure that the photovoltaic cleaning robot can detect the edge information of the photovoltaic panel in real time, so that the photovoltaic cleaning robot can adjust its operating posture according to the detected edge information of the photovoltaic panel.

[0003] Existing photovoltaic cleaning robots are generally equipped with cameras or lidars, and perform photovoltaic panel edge detection based on information collected by the cameras or lidars. This has low accuracy and makes it difficult to ensure that the photovoltaic cleaning robots can safely perform cleaning operations. Summary of the invention

[0004] In view of the above problems, the present application provides a photovoltaic panel edge detection method and related devices, which improve the accuracy of photovoltaic panel edge detection. The specific scheme is as follows:

[0005] The first aspect of the present application provides a photovoltaic panel edge detection method, comprising:

[0006] During the operation of the photovoltaic cleaning robot, point cloud data collected by a laser radar on the photovoltaic cleaning robot is obtained, and point cloud boundary points in the point cloud data are extracted;

[0007] Acquire a photovoltaic panel image acquired by an image acquisition device on the photovoltaic cleaning robot, and detect line features in the photovoltaic panel image;

[0008] According to the pre-calibrated external parameters of the laser radar and the image acquisition device, the point cloud boundary points in the radar coordinate system are converted to the image coordinate system;

[0009] In the image coordinate system, the line features are used as references to perform straight line fitting on the boundary points of the point cloud to obtain the edge information of the photovoltaic panel.

[0010] In a possible implementation, the performing straight line fitting on the boundary points of the point cloud with reference to the line features in the image coordinate system to obtain the edge information of the photovoltaic panel includes:

[0011] For each line segment in the line feature in the image coordinate system, extracting target point cloud boundary points in a preset range on both sides of the line segment from the point cloud boundary points;

[0012] A straight line fitting is performed on the boundary points of the target point cloud to obtain a fitting line segment corresponding to the line segment.

[0013] In a possible implementation, after obtaining the photovoltaic panel edge information, the photovoltaic panel edge detection method further includes:

[0014] Acquire the coordinates of a plurality of first corner points in the image coordinate system and the coordinates of a second corner point corresponding to each of the first corner points, wherein the first corner points are corner points in the line features, and the second corner points are corner points in the photovoltaic panel edge information;

[0015] Substituting the coordinates of the plurality of first corner points in the image coordinate system and the coordinates of the second corner point corresponding to each of the first corner points into a registration error function to calculate a registration error;

[0016] If the registration error is greater than a threshold, calibrating the extrinsic parameters;

[0017] If the registration error is not greater than the threshold, the extrinsic parameters are not calibrated.

[0018] In a possible implementation, calibrating the extrinsic parameter includes:

[0019] Taking minimization of the registration error function as the optimization goal, the external parameter parameters are updated, and the step of obtaining the point cloud data collected by the laser radar on the photovoltaic cleaning robot is returned to be executed. The external parameter parameters include the rotation matrix and the translation vector between the radar coordinate system and the image coordinate system.

[0020] In a possible implementation, after obtaining the photovoltaic panel edge information, the photovoltaic panel edge detection method further includes:

[0021] Determining the angle difference and height difference between adjacent photovoltaic panels according to the photovoltaic panel edge information corresponding to the adjacent photovoltaic panels;

[0022] Based on the angle difference and height difference between adjacent photovoltaic panels, the running posture of the photovoltaic cleaning robot is adjusted.

[0023] In a possible implementation, after obtaining the photovoltaic panel edge information, the photovoltaic panel edge detection method further includes:

[0024] The operation area of ​​the photovoltaic cleaning robot is determined according to the photovoltaic panel edge information.

[0025] A second aspect of the present application provides a controller, comprising at least one processor and a memory connected to the processor, wherein:

[0026] The memory is used to store computer programs;

[0027] The processor is used to execute the computer program so that the controller can implement the photovoltaic panel edge detection method of the first aspect or any implementation manner of the first aspect.

[0028] A third aspect of the present application provides a photovoltaic cleaning robot, comprising: a controller, a laser radar, an image acquisition device, a motion component, and a cleaning component;

[0029] The controller is respectively connected to the laser radar, the image acquisition device, the motion component and the cleaning component for communication;

[0030] The controller is used to obtain point cloud data collected by the laser radar during the operation of the photovoltaic cleaning robot, extract point cloud boundary points in the point cloud data, obtain the photovoltaic panel image collected by the image acquisition device, detect line features in the photovoltaic panel image, and convert the point cloud boundary points in the radar coordinate system to the image coordinate system based on the pre-calibrated external parameters of the laser radar and the image acquisition device. In the image coordinate system, the point cloud boundary points are fitted with straight lines with reference to the line features to obtain the edge information of the photovoltaic panel.

[0031] In a possible implementation, the laser radar and the image acquisition device are installed at relatively vertical positions on the photovoltaic cleaning robot.

[0032] A fourth aspect of the present application provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on a controller, the controller implements the photovoltaic panel edge detection method of the first aspect or any implementation of the first aspect.

[0033] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by a controller, the controller can implement the photovoltaic panel edge detection method of the above-mentioned first aspect or any implementation of the first aspect.

[0034] By means of the above technical scheme, the photovoltaic panel edge detection method and related devices provided by the present application simultaneously install a laser radar and an image acquisition device on a photovoltaic cleaning robot. During the operation of the photovoltaic cleaning robot, the point cloud data collected by the laser radar is obtained, the point cloud boundary points in the point cloud data are extracted, and the photovoltaic panel image collected by the image acquisition device is obtained, and the line features in the photovoltaic panel image are detected. By converting the point cloud boundary points in the radar coordinate system to the image coordinate system, the point cloud boundary points are fitted with straight lines with reference to the line features in the image coordinate system to obtain the edge information of the photovoltaic panel. This solves the problem that the image collected by the camera cannot accurately reflect the edge of the photovoltaic panel under strong light or flare generated by the photovoltaic panel, resulting in the false detection of the edge information of the photovoltaic panel based on a single camera or even the inability to detect the edge information of the photovoltaic panel. It also solves the problem that the point cloud data of a single laser radar cannot accurately fit the edge information of an irregular photovoltaic array, thereby improving the accuracy of photovoltaic panel edge detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0036] Figure 1 A schematic diagram of a photovoltaic panel edge detection method provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of straight line fitting of point cloud boundary points provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of the angle difference and height difference between adjacent photovoltaic panels provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of the controller structure provided in an embodiment of the present application;

[0040] Figure 5 Schematic diagram of edge information extraction of the laser radar and camera in the photovoltaic cleaning robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In the following description, in order to ensure the accuracy of the citation and the fluency of reading, the key technical terms, abbreviations or abbreviations involved in the text are summarized and explained as follows:

[0042] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0043] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0044] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0045] The sensors commonly used in existing photovoltaic edge detection schemes are cameras or lidars, but the detection method of a single sensor has certain disadvantages. For the detection method of a single camera, in the case of strong light or flare generated by the photovoltaic panel, the photovoltaic panel image taken by the camera cannot accurately reflect the edge information of the photovoltaic panel. Processing the photovoltaic panel image may cause false detection of edge information or even fail to detect edge information, seriously affecting the normal cleaning operation of the robot. For the detection method of a single lidar, the point cloud information of the single lidar may not be able to accurately fit the edge point information in the case of an irregular photovoltaic array, resulting in errors in the calculation of the geometric dimensions of the photovoltaic panel. This inaccuracy will affect the accuracy of the calculation of the angle difference and height difference between the photovoltaic panels, which may cause the robot's posture control adjustment to fail.

[0046] In order to solve the above problems, the embodiment of the present application provides a photovoltaic panel edge detection method, which is applied to the controller of a photovoltaic cleaning robot. The photovoltaic cleaning robot is also equipped with a laser radar and an image acquisition device. The image acquisition device can be a camera, a camera, or other device that can capture images of photovoltaic panels. The photovoltaic panel edge detection method of the embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0047] Reference Figure 1 , Figure 1 A schematic diagram of a photovoltaic panel edge detection method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, a data processing method provided in an embodiment of the present application may include steps 101 to 104, and these steps are described in detail below.

[0048] 101: During the operation of the photovoltaic cleaning robot, point cloud data collected by a laser radar on the photovoltaic cleaning robot is obtained, and point cloud boundary points in the point cloud data are extracted;

[0049] Some points in the point cloud data are boundary points of photovoltaic panels, while some points are not boundary points of photovoltaic panels. In order to perform edge detection of photovoltaic panels, it is necessary to extract the point cloud boundary points in the point cloud data.

[0050] In one possible implementation, points in the same ring (channel or channel group where each laser beam is emitted when the laser radar is scanning) in the point cloud data that meet the intensity (reflection intensity when the laser pulse returns) variation within a certain range are determined as point cloud boundary points. It should be noted that the intensity variation intensity here is defined as the range difference between the previous and next points in each frame of point cloud data, and then the points whose intensity variation intensity is greater than the threshold and meet the point cloud coordinate range are retained as boundary points, where the range of the previous point represents the distance from the laser radar to the previous point, and the range of the next point represents the distance from the laser radar to the next point.

[0051] 102: Acquire a photovoltaic panel image acquired by an image acquisition device on the photovoltaic cleaning robot, and detect line features in the photovoltaic panel image;

[0052] Line features in photovoltaic panel images are detected using line segment detection algorithms.

[0053] For example, the LSD (A Line Segment Detector) algorithm is used to detect line features in photovoltaic panel images. The LSD algorithm is an algorithm for extracting line segments in local images. It can select an adaptive mode, does not require manual parameter adjustment, has a fast processing speed, and can obtain sub-pixel precision detection results in a linear time.

[0054] Exemplarily, the Hough line detection algorithm is used to detect line features in the photovoltaic panel image. The Hough line detection algorithm has strong anti-interference ability and is insensitive to noise in the image.

[0055] The above two line segment detection algorithms are merely examples, and the present embodiment is not limited thereto.

[0056] Photovoltaic panel images can provide rich texture and color information. By performing line segment detection on photovoltaic panel images, more accurate line features representing the geometric information of photovoltaic panels can be obtained. Line features in photovoltaic panel images include multiple line segments corresponding to the edges of photovoltaic panels and the intersection of adjacent line segments, i.e., the first corner point.

[0057] 103: According to the pre-calibrated external parameters of the laser radar and the image acquisition device, the point cloud boundary points in the radar coordinate system are converted to the image coordinate system;

[0058] The external parameters of the laser radar and image acquisition equipment include: the rotation matrix R and the translation vector T between the radar coordinate system and the image coordinate system.

[0059] In the initial state, the open source algorithm Lidar-camera-calibration can be used to jointly calibrate the rotation matrix R and translation vector T of the lidar and image acquisition device as the initial extrinsic parameters. Specifically, the lidar is initially rotated and translated within a certain range, and the yaw angle, pitch angle, and roll angle are traversed with a certain step size. Then, based on the successful matching ratio of the radar point cloud and the line feature, a set of rotation matrices R and translation vectors T with the highest successful matching ratio are selected as the initial extrinsic parameters.

[0060] The external parameter parameters of the laser radar and the image acquisition device can also be calibrated external parameter parameters.

[0061] by Represents the boundary points of the point cloud in the radar coordinate system, Take the point cloud boundary points in the image coordinate system as an example, .

[0062] The point cloud boundary points in the radar coordinate system are converted to the image coordinate system, that is, the point cloud boundary points are projected into the photovoltaic panel image.

[0063] 104: In the image coordinate system, using the line features as a reference, a straight line is fitted to the boundary points of the point cloud to obtain the edge information of the photovoltaic panel.

[0064] Generally, the boundary points of the point cloud float on one or both sides of each line segment in the line feature. In order to reduce the impact of noise in the point cloud data on the final photovoltaic panel edge information, the line features are used as reference in the image coordinate system to perform straight line fitting on the boundary points of the point cloud to obtain the photovoltaic panel edge information.

[0065] In one possible implementation, for each line segment in the line feature in the image coordinate system, the target point cloud boundary points in a preset range on both sides of the line segment are extracted from the point cloud boundary points, that is, the noise in the point cloud boundary points is eliminated, and then the target point cloud boundary points are fitted with a straight line to obtain the fitted line segment corresponding to the line segment in the line feature.

[0066] For example, the RANSAC (RANdom SAmple Consensus) algorithm is used to perform straight line fitting on the boundary points of the target point cloud. The fitting result is as follows: Figure 2As shown in the figure, the intersection of adjacent fitting line segments is the corner point of the photovoltaic panel. In order to distinguish it from the first corner point in the above line feature, Figure 2 The corner point shown is defined below as the second corner point.

[0067] This embodiment uses line features as prior information to ensure that the edge shape of the photovoltaic panel in the photovoltaic panel edge information obtained after linear fitting of the point cloud boundary points is consistent with the edge shape of the photovoltaic panel in the photovoltaic panel image, which solves the problem that the point cloud data of a single laser radar cannot accurately fit the edge information of an irregular photovoltaic array. At the same time, this embodiment uses the point cloud boundary point fitting result of the laser radar as the final photovoltaic panel edge information. Since the point cloud data can represent the accurate distance and 3D spatial information of the photovoltaic panel, strong light or flare generated by the photovoltaic panel does not affect the accuracy of the point cloud data, and also solves the problem that the image collected by the camera cannot accurately reflect the edge of the photovoltaic panel under strong light or flare generated by the photovoltaic panel, resulting in the false detection of the photovoltaic panel edge information based on a single camera or even the inability to detect the photovoltaic panel edge information. In other words, this embodiment realizes the complementarity of the two sensor data through dual detection of the laser radar and the image acquisition device, and realizes that the photovoltaic panel edge information can be accurately detected in various extreme cases, ensuring the reliability and stability of the photovoltaic edge detection results.

[0068] After the photovoltaic cleaning robot obtains the edge information of the photovoltaic panel, the operation area of ​​the photovoltaic cleaning robot can be determined according to the edge information of the photovoltaic panel, and the safe operation area of ​​the photovoltaic cleaning robot can be planned in real time.

[0069] like Figure 3 As shown, the photovoltaic cleaning robot can also determine the angle difference and height difference between adjacent photovoltaic panels based on the photovoltaic panel edge information corresponding to the adjacent photovoltaic panels. Specifically, the center of mass and plane normal vectors of the adjacent photovoltaic panels are calculated according to the coordinates of the second corner points in the photovoltaic panel edge information of the adjacent photovoltaic panels. The angle difference of the adjacent photovoltaic panels is calculated based on the calculation of the plane normal vectors of the adjacent photovoltaic panels, and the height difference of the adjacent photovoltaic panels is calculated based on the center of mass of the adjacent photovoltaic panels. Then, based on the angle difference and height difference between the adjacent photovoltaic panels, the running posture of the photovoltaic cleaning robot is adjusted to ensure the safe operation of the photovoltaic cleaning robot.

[0070] It should be noted that the photovoltaic panel edge detection method in the above embodiment involves coordinate system conversion, so the accuracy of the external parameter parameters of the laser radar and the image acquisition device directly affects the accuracy of the photovoltaic panel edge detection. In order to verify whether the external parameter parameters of the laser radar and the image acquisition device are accurate, after obtaining the photovoltaic panel edge information, it is also possible to verify whether the external parameter parameters are accurate based on the error between the coordinates of the corner points in the line features of the image coordinate system and the coordinates of the corner points in the photovoltaic panel edge information.

[0071] In a possible implementation, the method for verifying whether the external parameters of the laser radar and the image acquisition device are accurate includes the following steps 201-204:

[0072] 201: Obtaining coordinates of multiple first corner points and coordinates of second corner points corresponding to each first corner point in an image coordinate system, where the first corner points are corner points in line features and the second corner points are corner points in photovoltaic panel edge information;

[0073] It should be noted that the coordinates of the first corner point and the coordinates of the second corner point are both three-dimensional coordinates.

[0074] 202: Substituting the coordinates of the plurality of first corner points in the image coordinate system and the coordinates of the second corner point corresponding to each first corner point into the registration error function to calculate the registration error;

[0075] Exemplarily, the registration error function is:

[0076]

[0077] Exemplarily, the registration error function is:

[0078]

[0079] Above represents the registration error function, Represents the number of the first corner points. It should be noted that the number of the first corner points is consistent with the number of the second corner points. R represents the rotation matrix in the extrinsic parameter, and T represents the translation vector in the extrinsic parameter. represents the coordinates of the i-th first corner point, Represents the coordinates of the second corner point.

[0080] 203: If the registration error is greater than the threshold, calibrate the external parameters;

[0081] Exemplarily, the external parameter is updated with minimization of the registration error function as the optimization goal, and the process returns to execute step 101 in the above embodiment until the registration error is no greater than a threshold, and the updating of the external parameter is stopped.

[0082] 204: If the registration error is not greater than the threshold, the external parameter is not calibrated.

[0083] It should be noted that since the photovoltaic cleaning robot runs at a relatively slow speed, the laser radar and image acquisition equipment collect data very quickly. When the external parameter parameters need to be calibrated, the laser radar and image acquisition equipment will update the external parameter parameters every time they collect data. Therefore, the external parameter parameters will be updated very quickly after multiple iterations, which will not affect the safe operation of the photovoltaic cleaning robot.

[0084] A photovoltaic panel edge detection method provided by an embodiment of the present application is introduced above, and a device for executing the photovoltaic panel edge detection method will be introduced below.

[0085] An embodiment of the present application provides a photovoltaic panel edge detection device, comprising:

[0086] A boundary point extraction unit is used to obtain point cloud data collected by a laser radar on the photovoltaic cleaning robot during the operation of the photovoltaic cleaning robot, and extract point cloud boundary points in the point cloud data;

[0087] A line feature monitoring unit, used to obtain a photovoltaic panel image acquired by an image acquisition device on the photovoltaic cleaning robot, and detect line features in the photovoltaic panel image;

[0088] A conversion unit, used for converting the point cloud boundary points in the radar coordinate system to the image coordinate system according to the pre-calibrated external parameters of the laser radar and the image acquisition device;

[0089] The straight line fitting unit is used to perform straight line fitting on the boundary points of the point cloud in the image coordinate system with reference to the line features to obtain the edge information of the photovoltaic panel.

[0090] In a possible implementation, the straight line fitting unit is used to extract target point cloud boundary points within a preset range on both sides of the line segment from the point cloud boundary points for each line segment in the line feature in the image coordinate system; perform straight line fitting on the target point cloud boundary points to obtain a fitted line segment corresponding to the line segment.

[0091] In a possible implementation, the photovoltaic panel edge detection device further includes:

[0092] A corner point acquisition unit, used to acquire the coordinates of a plurality of first corner points in the image coordinate system and the coordinates of a second corner point corresponding to each of the first corner points, wherein the first corner points are corner points in the line features, and the second corner points are corner points in the photovoltaic panel edge information;

[0093] An error calculation unit is used to substitute the coordinates of multiple first corner points in the image coordinate system and the coordinates of the second corner point corresponding to each first corner point into a registration error function to calculate a registration error; if the registration error is not greater than a threshold, the extrinsic parameter is not calibrated; if the registration error is greater than a threshold, a parameter calibration unit is triggered.

[0094] The parameter calibration unit is used to calibrate the external parameter.

[0095] In a possible implementation, the parameter calibration unit is specifically used to update the extrinsic parameters with minimization of the registration error function as an optimization goal, and trigger the boundary point extraction unit, wherein the extrinsic parameters include a rotation matrix and a translation vector between the radar coordinate system and the image coordinate system.

[0096] In a possible implementation, the photovoltaic panel edge detection device further includes:

[0097] The posture adjustment unit is used to determine the angle difference and height difference between adjacent photovoltaic panels according to the photovoltaic panel edge information corresponding to the adjacent photovoltaic panels; based on the angle difference and height difference between the adjacent photovoltaic panels, adjust the operating posture of the photovoltaic cleaning robot.

[0098] In a possible implementation, the photovoltaic panel edge detection device further includes:

[0099] An operating area determination unit is used to determine the operating area of ​​the photovoltaic cleaning robot according to the photovoltaic panel edge information.

[0100] The present application also provides a controller in an embodiment. Figure 4 As shown, it shows a structural schematic diagram of a controller suitable for implementing the embodiments of the present application. Figure 4 The controller shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0101] like Figure 4 As shown, the controller may include a processing device (e.g., a central processing unit, a graphics processor, etc., an AI processor) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 to a random access memory (RAM) 403. When the controller is powered on, various programs and data required for the operation of the controller are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0102] Typically, the following devices may be connected to the I / O interface 405: input devices 406 including, for example, a touch screen, a touch pad, a laser radar, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 408 including, for example, a memory card, a hard disk, etc.; and communication devices 409. The communication device 409 may allow the controller to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4A controller having various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0103] A photovoltaic cleaning robot is also provided in an embodiment of the present application, including a controller, a laser radar, an image acquisition device, a motion component and a cleaning component.

[0104] The image acquisition device may be a camera, a webcam or other device capable of acquiring images of photovoltaic panels.

[0105] The motion component may be a mobile chassis, which is disposed at the bottom of the photovoltaic cleaning robot and is used to drive the photovoltaic cleaning robot to move under the control of the controller.

[0106] The cleaning component may include a robot arm and a roller brush disposed at the end of the robot arm, and is used to clean the photovoltaic panel under the control of the controller.

[0107] The controller is respectively connected to the laser radar, the image acquisition device, the motion component and the cleaning component for communication.

[0108] The controller is used to obtain point cloud data collected by the laser radar during the operation of the photovoltaic cleaning robot, extract point cloud boundary points in the point cloud data, obtain the photovoltaic panel image collected by the image acquisition device, detect line features in the photovoltaic panel image, and convert the point cloud boundary points in the radar coordinate system to the image coordinate system based on the pre-calibrated external parameters of the laser radar and the image acquisition device. In the image coordinate system, the point cloud boundary points are fitted with straight lines with reference to the line features to obtain the edge information of the photovoltaic panel.

[0109] The embodiment of the present application does not limit the relative position between the laser radar and the image acquisition device. The external parameters of the laser radar and the image acquisition device are calibrated according to the relative position between the laser radar and the image acquisition device.

[0110] In a possible implementation, the laser radar and the image acquisition device are installed at a relatively vertical position on the photovoltaic cleaning robot, and the photovoltaic panels are detected from a bird's-eye view. The external parameter between the laser radar and the image acquisition device is only one dimension of the Z axis, and a relatively accurate external parameter can be obtained through the external parameter calibration algorithm. Take the image acquisition device as a camera as an example, Figure 5 As shown in the figure, the photovoltaic panel image is captured by a camera. Since the image has rich texture and color information, a line segment detection algorithm can be used to perform line segment detection on the photovoltaic panel image. Thus, the border information of the photovoltaic panel can be detected. Figure 5The image line features in the laser radar point cloud data are then converted from the radar coordinate system to the image coordinate system. The line features recognized by the camera are then used as a reference to further fit the edge information of the photovoltaic panel to the point cloud boundary points, that is, Figure 5 The point cloud line features in the image can obtain more accurate photovoltaic panel edge information.

[0111] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on a controller, the controller implements any photovoltaic panel edge detection method provided in the embodiment of the present application.

[0112] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by a controller, the controller can implement any photovoltaic panel edge detection method provided in the embodiment of the present application.

[0113] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0114] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0115] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0116] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A photovoltaic panel edge detection method, characterized in that: include: During the operation of the photovoltaic cleaning robot, point cloud data collected by a laser radar on the photovoltaic cleaning robot is obtained, and point cloud boundary points in the point cloud data are extracted; Acquire a photovoltaic panel image acquired by an image acquisition device on the photovoltaic cleaning robot, and detect line features in the photovoltaic panel image; According to the pre-calibrated external parameters of the laser radar and the image acquisition device, the point cloud boundary points in the radar coordinate system are converted to the image coordinate system; In the image coordinate system, the line features are used as references to perform straight line fitting on the boundary points of the point cloud to obtain the edge information of the photovoltaic panel.

2. The photovoltaic panel edge detection method according to claim 1, characterized in that: The step of performing straight line fitting on the boundary points of the point cloud in the image coordinate system with reference to the line features to obtain the edge information of the photovoltaic panel includes: For each line segment in the line feature in the image coordinate system, extracting target point cloud boundary points in a preset range on both sides of the line segment from the point cloud boundary points; A straight line fitting is performed on the boundary points of the target point cloud to obtain a fitting line segment corresponding to the line segment.

3. The photovoltaic panel edge detection method according to claim 1, characterized in that: After obtaining the photovoltaic panel edge information, the photovoltaic panel edge detection method further includes: Acquire the coordinates of a plurality of first corner points in the image coordinate system and the coordinates of a second corner point corresponding to each of the first corner points, wherein the first corner points are corner points in the line features, and the second corner points are corner points in the photovoltaic panel edge information; Substituting the coordinates of the plurality of first corner points in the image coordinate system and the coordinates of the second corner point corresponding to each of the first corner points into a registration error function to calculate a registration error; If the registration error is greater than a threshold, calibrating the extrinsic parameters; If the registration error is not greater than the threshold, the extrinsic parameters are not calibrated.

4. The photovoltaic panel edge detection method according to claim 3, characterized in that: The calibrating the external parameters includes: Taking minimization of the registration error function as the optimization goal, the external parameter parameters are updated, and the step of obtaining the point cloud data collected by the laser radar on the photovoltaic cleaning robot is returned to be executed. The external parameter parameters include the rotation matrix and the translation vector between the radar coordinate system and the image coordinate system.

5. The photovoltaic panel edge detection method according to claim 1, characterized in that: After obtaining the photovoltaic panel edge information, the photovoltaic panel edge detection method further includes: Determining the angle difference and height difference between adjacent photovoltaic panels according to the photovoltaic panel edge information corresponding to the adjacent photovoltaic panels; Based on the angle difference and height difference between adjacent photovoltaic panels, the running posture of the photovoltaic cleaning robot is adjusted.

6. The photovoltaic panel edge detection method according to claim 1, characterized in that: After obtaining the photovoltaic panel edge information, the photovoltaic panel edge detection method further includes: The operation area of ​​the photovoltaic cleaning robot is determined according to the photovoltaic panel edge information.

7. A controller, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the controller can implement the photovoltaic panel edge detection method according to any one of claims 1 to 6.

8. A photovoltaic cleaning robot, characterized in that: include: Controller, LiDAR, image acquisition equipment, motion components, and cleaning components; The controller is respectively connected to the laser radar, the image acquisition device, the motion component and the cleaning component for communication; The controller is used to obtain point cloud data collected by the laser radar during the operation of the photovoltaic cleaning robot, extract point cloud boundary points in the point cloud data, obtain the photovoltaic panel image collected by the image acquisition device, detect line features in the photovoltaic panel image, and convert the point cloud boundary points in the radar coordinate system to the image coordinate system based on the pre-calibrated external parameters of the laser radar and the image acquisition device. In the image coordinate system, the point cloud boundary points are fitted with straight lines with reference to the line features to obtain the edge information of the photovoltaic panel.

9. The photovoltaic cleaning robot according to claim 8, characterized in that: The installation positions of the laser radar and the image acquisition device on the photovoltaic cleaning robot are relatively vertical.

10. A computer program product, characterized in that The method comprises computer-readable instructions, and when the computer-readable instructions are executed on a controller, the controller is enabled to implement the photovoltaic panel edge detection method according to any one of claims 1 to 6.

11. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by the controller, the controller can implement the photovoltaic panel edge detection method as described in any one of claims 1 to 6.

Images