A photovoltaic robot localization and navigation method and device based on SLAM technology

Abstract

This invention provides a method and apparatus for positioning and navigation of a photovoltaic robot based on SLAM technology. The method includes: acquiring an environmental point cloud map of the photovoltaic robot using a rotatable lidar mounted on the robot; acquiring a scene image of the photovoltaic robot using a vision camera mounted on the robot; fusing the environmental point cloud map and the scene image at the same time to create an omnidirectional 3D photovoltaic scene map using SLAM mapping; and, based on the omnidirectional 3D photovoltaic scene map and in conjunction with obstacle avoidance sensors mounted on the photovoltaic robot, performing positioning identification and navigation control on the photovoltaic robot to guide it to its destination. The photovoltaic robot positioning and navigation method and apparatus based on SLAM technology provided by this invention can effectively and automatically avoid obstacles and accurately identify targets while achieving on-site navigation through multiple sensors, thus assisting in the operation.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic robot positioning and navigation, and in particular to a photovoltaic robot positioning and navigation method and apparatus based on SLAM technology. Background Technology

[0002] As the photovoltaic cleaning robot moves along the photovoltaic modules, it uses the rapid rotation of its brushes to complete the cleaning function. In addition to ensuring smooth movement along the modules, it also needs to have a certain ability to identify obstacles.

[0003] SLAM (Simultaneous Localization and Mapping) technology can locate the position and orientation of a photovoltaic robot by repeatedly observing map features (such as the frame of the photovoltaic module, pillars, etc.) during its movement. Then, it can incrementally build a map based on its own position, thereby achieving the purpose of simultaneous localization and map building, which can help the photovoltaic robot to realize navigation and positioning functions.

[0004] While SLAM technology can achieve basic navigation and positioning functions, current photovoltaic robots relying solely on SLAM technology cannot effectively identify and navigate around special obstacles such as potholes, puddles, and snowdrifts. Furthermore, when photovoltaic robots are handling objects or cleaning photovoltaic modules, recognition accuracy issues often prevent them from achieving the desired results. Summary of the Invention

[0005] The purpose of this invention is to provide a photovoltaic robot positioning and navigation method and device based on SLAM technology, enabling the photovoltaic robot to achieve all-terrain navigation without human intervention at the photovoltaic site.

[0006] The technical solution provided by this invention is as follows:

[0007] In some embodiments, the photovoltaic robot localization and navigation method based on SLAM technology provided by the present invention includes:

[0008] The environmental point cloud map of the photovoltaic robot is obtained by a rotatable lidar mounted on the photovoltaic robot;

[0009] Scene images of the photovoltaic robot are acquired using a vision camera installed on the photovoltaic robot;

[0010] The environmental point cloud map and the scene image at the same moment are fused together, and SLAM mapping is used to obtain an omnidirectional three-dimensional photovoltaic field map;

[0011] Based on the omnidirectional three-dimensional photovoltaic site map, and combined with the obstacle avoidance sensors installed on the photovoltaic robot, the photovoltaic robot is positioned, identified, and navigated to reach its destination.

[0012] In some implementations, the fusion of the environmental point cloud map and the scene image at the same time, and the use of SLAM mapping to obtain an omnidirectional three-dimensional photovoltaic field map, specifically includes:

[0013] The environmental point cloud image is subjected to noise reduction filtering to obtain the effective point cloud in the environmental point cloud image;

[0014] Based on the effective point cloud, SLAM is used to build a 3D environment point cloud map;

[0015] The scene image corresponding to the environmental point cloud map at the same time is input into the trained target recognition model, which identifies and outputs each target object in the scene image; the target objects identified in the scene image are matched into the 3D environmental point cloud map to generate an omnidirectional three-dimensional photovoltaic field map.

[0016] In some implementations, the step of matching each target object identified in the scene image to the 3D environmental point cloud map to generate an omnidirectional three-dimensional photovoltaic site map specifically includes:

[0017] Obtain the coordinates (1) of each target object identified in the scene image in the world coordinate system;

[0018] The segmentation algorithm identifies each target object in the 3D environment point cloud map and calculates the coordinates of each target object in the world coordinate system.

[0019] Based on coordinate 1, select the target object that is closest to coordinate 1 from the 3D environment point cloud map as the matching object;

[0020] Based on coordinate 1 and coordinate 2, obtain the coordinate difference between each target object and the corresponding matching object;

[0021] If the coordinate difference is within the set difference range, then the matched object is determined to be the corresponding target object in the scene image;

[0022] The matching objects in the 3D environmental point cloud map are labeled as the corresponding target objects to generate the omnidirectional three-dimensional photovoltaic field map.

[0023] In some embodiments, the photovoltaic robot is further equipped with a robotic arm for transporting the first target photovoltaic module; after acquiring the omnidirectional three-dimensional photovoltaic site image, it also includes:

[0024] The local point cloud map of the current scene is detected by the lidar installed on the photovoltaic robot;

[0025] The local point cloud map of the current scene is matched with the omnidirectional 3D photovoltaic site map to locate the current position of the photovoltaic robot; and the position of the first target photovoltaic module to be transported and the target position to be transported are determined in the omnidirectional 3D photovoltaic site map.

[0026] Based on the current position of the photovoltaic robot, the position of the first target photovoltaic module, and the target position to be transported, a first movement path of the photovoltaic robot before transport and a second movement path during transport are planned.

[0027] According to the first movement path, the photovoltaic robot is driven to move to the position of the first target photovoltaic module;

[0028] Obtain the current pose of the robotic arm of the photovoltaic robot;

[0029] Based on the current pose of the photovoltaic robot's robotic arm and the position of the first target photovoltaic module, the photovoltaic robot is controlled to use the robotic arm to grasp the first target photovoltaic module.

[0030] After grasping the first target photovoltaic module, the photovoltaic robot is controlled to transport the first target photovoltaic module to the target location according to the second movement path.

[0031] In some embodiments, the photovoltaic robot is further equipped with a cleaning brush for cleaning a second target photovoltaic module; the destination of the photovoltaic robot is the location of the second target photovoltaic module; and after controlling the photovoltaic robot to reach the location of the second target photovoltaic module, the following steps are also included:

[0032] The second target photovoltaic module is cleaned using the cleaning brush mounted on the photovoltaic robot;

[0033] During the cleaning of the second target photovoltaic module, the distance between the brush handle of the cleaning brush and the surface of the second target photovoltaic module is obtained in real time by the lidar or ultrasonic sensor installed on the photovoltaic robot, so as to control the pressure applied by the cleaning brush to the second target photovoltaic module.

[0034] In some embodiments, based on the same technical concept, a photovoltaic robot positioning and navigation device based on SLAM technology is also provided, comprising:

[0035] An environmental point cloud acquisition module is used to acquire an environmental point cloud map of the photovoltaic robot using a rotatable lidar mounted on the photovoltaic robot.

[0036] A scene image acquisition module is used to acquire scene images of the photovoltaic robot through a vision camera installed on the photovoltaic robot;

[0037] The map building module is used to fuse the environmental point cloud map and the scene image at the same time, and use SLAM to build a map to obtain an omnidirectional three-dimensional photovoltaic field map;

[0038] The positioning and navigation module is used to detect the local point cloud map of the current scene based on the omnidirectional three-dimensional photovoltaic site map, using a lidar installed on the photovoltaic robot. Combined with the obstacle avoidance sensor installed on the photovoltaic robot, the module performs positioning identification and navigation control on the photovoltaic robot, and controls the photovoltaic robot to reach the destination.

[0039] In some implementations, the map building module includes:

[0040] The noise reduction and filtering submodule is used to perform noise reduction and filtering on the environmental point cloud map to obtain the effective point cloud in the environmental point cloud map.

[0041] The map building submodule is used to build a 3D environment point cloud map based on the effective point cloud using SLAM.

[0042] The image recognition submodule is used to input the scene image into the trained target recognition model and identify and output each target object in the scene image.

[0043] The matching and fusion submodule is used to match each target object identified in the scene image to the 3D environment point cloud map to generate an omnidirectional three-dimensional photovoltaic field map.

[0044] In some implementations, the matching and fusion submodule includes:

[0045] The coordinate acquisition submodule is used to acquire the coordinates of each target object identified in the scene image in the world coordinate system.

[0046] The segmentation and recognition submodule is used to identify each target object in the 3D environment point cloud map through a segmentation algorithm, and to calculate the coordinates of each target object in the 3D environment point cloud map in the world coordinate system through the coordinate acquisition submodule.

[0047] The matching selection submodule is used to select the target object that is closest to the coordinate 1 from the 3D environment point cloud map, and use it as the matching object;

[0048] The difference calculation submodule is used to calculate the coordinate difference of the target object based on the coordinate 1 and the coordinate 2;

[0049] The matching determination submodule is used to determine that the matching object is the corresponding target object in the scene image if the coordinate difference is within a set difference range.

[0050] The annotation submodule is used to annotate the matching objects in the 3D environmental point cloud map as the corresponding target objects, and generate an omnidirectional three-dimensional photovoltaic field map.

[0051] In some embodiments, the photovoltaic robot is equipped with a robotic arm for transporting the first target photovoltaic module; wherein:

[0052] The environmental point cloud acquisition module is also used to detect a local point cloud map of the current scene using the lidar installed on the photovoltaic robot;

[0053] The positioning and navigation module specifically includes:

[0054] The positioning and determination submodule is used to match the local point cloud map of the current scene with the omnidirectional three-dimensional photovoltaic field map to locate the current position of the photovoltaic robot; and to determine the position of the first target photovoltaic module to be transported and the target position to be transported in the omnidirectional three-dimensional photovoltaic field map.

[0055] The path planning submodule is used to plan a first movement path of the photovoltaic robot before transporting the photovoltaic robot, and a second movement path during transporting the photovoltaic robot, based on the current position of the photovoltaic robot, the position of the first target photovoltaic module, and the target position to be transported.

[0056] The mobile control submodule is used to drive the photovoltaic robot to the position of the first target photovoltaic module according to the first mobile path;

[0057] The pose acquisition submodule is used to acquire the current pose of the robotic arm of the photovoltaic robot;

[0058] The grasping control submodule is used to control the photovoltaic robot to grasp the first target photovoltaic module by means of the robotic arm, based on the current pose of the robotic arm of the photovoltaic robot and the position of the first target photovoltaic module.

[0059] The mobile control submodule is further configured to, after grasping the first target photovoltaic module, control the photovoltaic robot to transport the first target photovoltaic module to the target location according to the second movement path.

[0060] In some embodiments, the photovoltaic robot is further equipped with a cleaning brush for cleaning a second target photovoltaic module; the positioning and navigation device also includes:

[0061] A cleaning control module is used to clean the second target photovoltaic module by means of a cleaning brush installed on the photovoltaic robot after the photovoltaic robot arrives at the destination where the second photovoltaic module is located;

[0062] Furthermore, during the cleaning of the second target photovoltaic module, the distance between the brush handle of the cleaning brush and the surface of the second target photovoltaic module is obtained in real time by the lidar or ultrasonic sensor installed on the photovoltaic robot, so as to control the pressure applied by the cleaning brush to the second target photovoltaic module.

[0063] The photovoltaic robot localization and navigation method and apparatus based on SLAM technology provided by this invention have at least the following beneficial effects:

[0064] 1. The photovoltaic robot localization and navigation method based on SLAM technology provided in this application integrates deep learning algorithms of LiDAR and visual camera to perform SLAM mapping and automatic obstacle avoidance, and realizes on-site navigation through multiple sensors.

[0065] 2. The omnidirectional three-dimensional photovoltaic field map established by the scheme of this application can realize the identification of target photovoltaic modules, target locations, pits, water puddles, snow piles, etc., which can be used for inspection work and also provide auxiliary handling and cleaning functions for other field work.

[0066] 2. The photovoltaic robot positioning and navigation method based on SLAM technology provided by this invention adopts a mobile chassis equipped with navigation and obstacle avoidance sensors such as lidar to achieve all-terrain navigation in photovoltaic sites. It can also be combined with maps and high-precision GPS or Beidou positioning to achieve specific positioning and navigation functions with an accuracy of centimeter level. Attached Figure Description

[0067] The preferred embodiments will be described below in a clear and easy-to-understand manner, with reference to the accompanying drawings, to further explain the above-mentioned characteristics, technical features, advantages, and implementation methods of a photovoltaic robot positioning and navigation method and device based on SLAM technology.

[0068] Figure 1 This is a flowchart illustrating an embodiment of a photovoltaic robot positioning and navigation method based on SLAM technology according to the present invention.

[0069] Figure 2 This is a schematic diagram of the world coordinate system in another embodiment of the photovoltaic robot positioning and navigation method based on SLAM technology of the present invention;

[0070] Figure 3 This is a schematic diagram of a photovoltaic robot equipped with a robotic arm, in another embodiment of a photovoltaic robot positioning and navigation method based on SLAM technology according to the present invention.

[0071] Figure 4 This is a schematic diagram of a photovoltaic robot equipped with a cleaning brush in another embodiment of a photovoltaic robot positioning and navigation method based on SLAM technology according to the present invention;

[0072] Figure 5 This is a block diagram of a module of an embodiment of a photovoltaic robot positioning and navigation device based on SLAM technology according to the present invention;

[0073] Figure 6 This is a block diagram of another embodiment of a photovoltaic robot positioning and navigation device based on SLAM technology according to the present invention.

[0074] Figure label:

[0075] a--Photovoltaic module; b--LiDAR; c--Vision camera; d--Mobile chassis; e--Origin of world coordinate system; f--Ultrasonic sensor; g--Robotic arm; h--Cleaning brush. Detailed Implementation

[0076] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0077] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or sets.

[0078] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0079] Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0080] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative effort.

[0081] In one embodiment, the photovoltaic robot localization and navigation method based on SLAM technology provided by the present invention refers to... Figure 1 The steps include:

[0082] S100 acquires environmental point cloud maps of the photovoltaic robot using a rotatable lidar mounted on the photovoltaic robot;

[0083] Specifically, the photovoltaic robot is equipped with a 360° rotating lidar for sampling its surroundings. This includes 3D coordinates (X, Y, Z), color, classification value, intensity value, time, and more. Lidar utilizes the principle of laser ranging, recording the 3D coordinates, reflectivity, and texture information of numerous dense points on the surface of the object being measured. This allows for the rapid reconstruction of a 3D model of the target object, as well as various point cloud data such as lines, surfaces, and volumes.

[0084] S200 acquires scene images of the photovoltaic robot through a vision camera installed on the photovoltaic robot;

[0085] The S300 fuses environmental point cloud images and scene images at the same time, uses SLAM mapping to obtain an omnidirectional 3D photovoltaic field image;

[0086] The S400, based on an omnidirectional 3D photovoltaic field map and combined with obstacle avoidance sensors installed on the photovoltaic robot, performs positioning recognition and navigation control on the photovoltaic robot, and controls the photovoltaic robot to reach its destination.

[0087] Specifically, point cloud data acquired by LiDAR and scene images of the photovoltaic robot acquired by a visual camera are fused together using SLAM (simultaneous localization and mapping), also known as real-time localization and mapping, to obtain an omnidirectional 3D photovoltaic site map. Then, obstacle avoidance sensors and other tools are used to locate, identify, and navigate the photovoltaic robot.

[0088] Specifically, step S400 includes:

[0089] S410 acquires a local point cloud map of the current scene using a lidar mounted on a photovoltaic robot;

[0090] S420 matches the local point cloud map of the current scene with the omnidirectional 3D photovoltaic field map to locate the current position of the photovoltaic robot;

[0091] S440, determines the destination based on the destination instructions received by the photovoltaic robot;

[0092] S450 determines its movement trajectory based on its current location and destination, combined with an omnidirectional three-dimensional photovoltaic field map;

[0093] S460 controls the movement of the photovoltaic robot based on the movement trajectory, and uses obstacle avoidance sensors to avoid obstacles during the movement.

[0094] Preferably, the step S420 above is followed by the following step:

[0095] S430 acquires a partial scene map of the current scene through the vision camera on the photovoltaic robot and identifies each target object in the partial scene map of the current scene;

[0096] S435, based on the current position of the photovoltaic robot, determine whether there are any identified target objects around the current position of the photovoltaic robot in the omnidirectional three-dimensional photovoltaic field map. If so, it is determined that the current position of the photovoltaic robot is accurately located; otherwise, return to step S410 to reposition.

[0097] In one embodiment, based on the above embodiments, step S300 specifically includes:

[0098] The environmental point cloud image is subjected to noise reduction filtering to obtain the effective point cloud in the environmental point cloud image;

[0099] Based on the effective point cloud, SLAM is used to build a 3D environment point cloud map.

[0100] The scene image corresponding to the environmental point cloud map at the same time is input into the trained target recognition model to identify and output each target object in the scene image;

[0101] Each target object identified in the scene image is matched into a 3D environmental point cloud map to generate an omnidirectional 3D photovoltaic field map.

[0102] Specifically, due to the influence of scanning equipment, the surrounding environment, and the characteristics of the scanned target itself, some noise is unavoidable in the acquired point cloud data. Therefore, denoising processing is required for the extracted point cloud. In this embodiment, effective point cloud is obtained by denoising and filtering the environmental point cloud. Based on the effective point cloud, SLAM technology is used to generate a 3D environmental point cloud map. Then, deep learning is used to identify and label various objects in the scene images acquired by the visual camera, such as photovoltaic modules, supports, hills, pits, puddles, etc. A model file is obtained by training a deep learning algorithm for machine vision and used for target detection. Combining the point cloud map data from the LiDAR and visual recognition, an omnidirectional 3D photovoltaic site map with labeled target names is jointly established.

[0103] In one embodiment, based on the above embodiments, each target object identified in the scene image is matched to a 3D environmental point cloud map to generate an omnidirectional three-dimensional photovoltaic site map; specifically including:

[0104] Obtain the coordinates of each target object identified in the scene image in the world coordinate system.

[0105] The segmentation algorithm identifies each target object in the 3D environment point cloud map and calculates the coordinates of each target object in the world coordinate system.

[0106] Based on coordinate 1, select the target object that is closest to coordinate 1 from the 3D environment point cloud map as the matching object;

[0107] Based on coordinates 1 and 2, obtain the coordinate difference between each target object and its corresponding matching object;

[0108] If the coordinate difference is within the set range, the matched object is determined to be the corresponding target object in the scene image;

[0109] The matching objects in the 3D environmental point cloud map are labeled as the corresponding target objects to generate an omnidirectional 3D photovoltaic field map.

[0110] Specifically, this embodiment employs the RANSAC (Random Sample Consensus) algorithm for 3D object point cloud segmentation based on LiDAR. This algorithm identifies target objects in the 3D environment point cloud map and calculates their coordinates in the world coordinate system. Based on the acquired coordinate data, the photovoltaic modules in the 3D environment point cloud map are matched and labeled with the photovoltaic modules in the scene image, generating an omnidirectional 3D photovoltaic field map. Specifically, when identifying photovoltaic modules, segmentation is performed using the top and bottom borders and left and right border lengths of the photovoltaic module as features. The remaining point cloud data is used as outliers to obtain the module's features and position, thereby obtaining the center position of the target module. The average value of the point cloud data in the depth direction (z) of the target module is calculated and defined as the target's depth value. Simultaneously, a segmentation algorithm is used with a vision camera mounted on the photovoltaic robot to obtain the x and y coordinates of the target module's center position. By combining the relative positions of LiDAR and a vision camera, the relative coordinates between the two sensors can be derived. By combining the LiDAR point cloud and the vision camera data, the x and y values ​​of the component are compared. If the x and y values ​​of the LiDAR point cloud and the vision camera are within a preset range of relative coordinate differences, the matched object is determined to be the corresponding photovoltaic component in the scene image. Thus, the overall x, y, and z coordinates of the current component can be obtained. Furthermore, the matched object in the 3D environment point cloud map is labeled as the corresponding photovoltaic component, generating an omnidirectional 3D photovoltaic field image.

[0111] Specifically, the relative coordinates of the LiDAR and vision camera based on the world coordinate system (mobile chassis) are fixed, referencing... Figure 2For example, a photovoltaic robot's mobile chassis d is equipped with a LiDAR d and a vision camera c. In the diagram, point e serves as the origin of the world coordinate system of the mobile chassis d. The center of LiDAR d is at world coordinate position x1, y1 relative to the mobile chassis d, and the center of vision camera c is at world coordinate position x2, y2 relative to the mobile chassis. The differences between them in the x and y directions in the world coordinate system are x1-x2 = m and y1-y2 = n. A point cloud is created using LiDAR d, and then a segmentation algorithm is used to identify the target component a. The coordinates of the center of target component a in the LiDAR coordinate system are obtained from the point cloud: X1, Y1, Z1. Simultaneously, the coordinates of the center of the target component in the coordinate system of vision camera c are calculated using the vision camera segmentation algorithm: X2, Y2. When X1-X2 = M and Y1-Y2 = N, depending on the site layout, generally, if the difference between M and m is within a set range, and the relative difference between N and n is also within a pre-set range, it can be considered that LiDAR d and vision camera c identify the same component. Meanwhile, the Z1 value obtained by lidar b is the depth value of the same target component.

[0112] In another embodiment, based on the above embodiment, the photovoltaic robot is equipped with a robotic arm for transporting the first target photovoltaic module; after acquiring the omnidirectional three-dimensional photovoltaic site image, it further includes:

[0113] The local point cloud map of the current scene is detected by the lidar installed on the photovoltaic robot;

[0114] The local point cloud map of the current scene is matched with the omnidirectional 3D photovoltaic site map to locate the current position of the photovoltaic robot; and the position of the first target photovoltaic module to be transported and the target position to be transported are determined in the omnidirectional 3D photovoltaic site map.

[0115] Based on the current position of the photovoltaic robot, the position of the first target photovoltaic module, and the target position to be transported, a first movement path of the photovoltaic robot before transport and a second movement path during transport are planned.

[0116] According to the first movement path, the photovoltaic robot is driven to move to the position of the first target photovoltaic module;

[0117] Obtain the current pose of the robotic arm of the photovoltaic robot;

[0118] Based on the current pose of the photovoltaic robot's robotic arm and the position of the first target photovoltaic module, the photovoltaic robot is controlled to use the robotic arm to grasp the first target photovoltaic module.

[0119] After grasping the first target photovoltaic module, the photovoltaic robot is controlled to transport the first target photovoltaic module to the target location according to the second movement path.

[0120] Preferably, before detecting a local point cloud map of the current scene using the lidar mounted on the photovoltaic robot, the method further includes:

[0121] The presence of a first target photovoltaic module in the current scene is identified by a vision camera installed on the photovoltaic robot.

[0122] When the presence of the first target photovoltaic module is detected in the current scene, a local point cloud map of the current scene is obtained and matched with an omnidirectional 3D photovoltaic field map in order to locate the current position of the photovoltaic robot and the position of the first target photovoltaic module.

[0123] Of course, after the photovoltaic robot arrives at the location of the first target photovoltaic module, it can further identify whether the current target object is the first target photovoltaic module to be moved through a vision camera. If so, the robotic arm can then grab and move it.

[0124] For specific examples, please refer to photovoltaic robots. Figure 3 The photovoltaic robot's mobile chassis (d) is equipped with navigation and obstacle avoidance sensors such as LiDAR (b), a vision camera (c), and an ultrasonic sensor (f), enabling all-terrain navigation at photovoltaic sites. Currently, the core algorithm for laser navigation uses a high-precision LiDAR sensor to measure the position of corresponding obstacles, combined with maps and high-precision GPS or BeiDou positioning to achieve specific positioning and navigation functions. In addition, the photovoltaic robot is equipped with a robotic arm (g). After the vision camera (c) identifies the photovoltaic module, the LiDAR (b) confirms the relative position of the photovoltaic module and the robotic arm (g), controlling the robotic arm (g) to grasp the photovoltaic module and transport it to the target location.

[0125] In another embodiment, based on any of the above embodiments, the photovoltaic robot is equipped with a cleaning brush for cleaning a second target photovoltaic module; the destination of the photovoltaic robot is the location of the second target photovoltaic module; and after controlling the photovoltaic robot to reach the location of the second target photovoltaic module, the method further includes:

[0126] The second target photovoltaic module is cleaned using cleaning brushes installed on a photovoltaic robot.

[0127] During the cleaning of the second target photovoltaic module, the distance between the cleaning brush and the surface of the second target photovoltaic module is obtained in real time by a lidar or ultrasonic sensor installed on the photovoltaic robot, so as to control the pressure applied by the cleaning brush to the second target photovoltaic module.

[0128] For specific examples, please refer to photovoltaic robots. Figure 4 The photovoltaic robot's mobile chassis d is equipped with a lidar (b), a vision camera (c), an ultrasonic sensor (f), and a cleaning brush (h). Specifically, after identifying the photovoltaic module to be cleaned, the robot moves to its vicinity and begins cleaning. The lidar confirms the relative position between the photovoltaic module and the brush handle of the cleaning brush (h). The lidar (b) or ultrasonic sensor (f) continuously monitors the distance between the brush handle of the cleaning brush (h) and the surface of the photovoltaic module, thereby controlling the pressure applied by the cleaning brush (h) during cleaning.

[0129] In one embodiment, based on the same technical concept, the present invention also provides a photovoltaic robot positioning and navigation device based on SLAM technology, referencing... Figure 5 ,include:

[0130] The environmental point cloud acquisition module 10 is used to acquire the environmental point cloud map of the photovoltaic robot by means of a rotatable lidar installed on the photovoltaic robot;

[0131] Scene image acquisition module 20 is used to acquire scene images of the photovoltaic robot through a vision camera installed on the photovoltaic robot;

[0132] The map building module 30 is used to fuse the environmental point cloud map and scene image at the same time, and to use SLAM to build the map to obtain an omnidirectional three-dimensional photovoltaic field map;

[0133] The positioning and navigation module 40 is used to detect the local point cloud map of the current scene based on the omnidirectional three-dimensional photovoltaic site map, by using the lidar installed on the photovoltaic robot, and combined with the obstacle avoidance sensor installed on the photovoltaic robot, to perform positioning identification and navigation control of the photovoltaic robot, and control the photovoltaic robot to reach the destination.

[0134] Specifically, in this embodiment, an environmental point cloud acquisition module acquires an environmental point cloud map around the photovoltaic robot, and then combines this with a scene image acquisition module to acquire scene images near the photovoltaic robot. A map building module uses SLAM technology to create a map, generating an omnidirectional 3D photovoltaic site map. After generating the omnidirectional 3D photovoltaic site map, the positioning and navigation module uses a lidar mounted on the photovoltaic robot to detect a local point cloud map of the current scene. Combined with obstacle avoidance sensors mounted on the photovoltaic robot, this enables the photovoltaic robot to perform positioning and navigation functions, such as moving and cleaning photovoltaic modules.

[0135] In one embodiment, based on the above embodiments, referring to Figure 6 The map building module 30 specifically includes:

[0136] The noise reduction and filtering submodule 31 is used to perform noise reduction and filtering on the environmental point cloud map to obtain the effective point cloud in the environmental point cloud map.

[0137] The map building submodule 32 is used to build a 3D environment point cloud map based on the effective point cloud and using SLAM.

[0138] Image recognition submodule 33 is used to input scene images into a trained target recognition model and identify and output target objects in the scene images;

[0139] The matching and fusion submodule 34 is used to match each target object identified in the scene image to the 3D environmental point cloud map to generate an omnidirectional three-dimensional photovoltaic field map.

[0140] Specifically, since the point cloud image acquired by the LiDAR contains a considerable amount of noise, a denoising and filtering submodule is needed to filter out this noise. This is done by denoising and filtering the environmental point cloud to obtain the valid point cloud data. Then, the map building submodule uses SLAM to build a 3D environmental point cloud map based on this valid point cloud. Next, the image recognition submodule identifies various objects in the environmental point cloud image, such as photovoltaic modules, supports, hills, pits, and puddles. Finally, the matching and fusion submodule matches the identified objects in the scene image to the 3D environmental point cloud map, generating an omnidirectional 3D photovoltaic site image.

[0141] In another embodiment, based on the above embodiments, the matching fusion submodule includes:

[0142] The coordinate acquisition submodule is used to obtain the coordinates of each target object identified in the scene image in the world coordinate system.

[0143] The segmentation and recognition submodule is used to identify each target object in the 3D environment point cloud map through the segmentation algorithm, and to calculate the coordinates of each target object in the 3D environment point cloud map in the world coordinate system through the coordinate acquisition submodule.

[0144] The matching selection submodule is used to select the target object that is closest to coordinate 1 from the 3D environment point cloud map based on coordinate 1, and use it as the matching object.

[0145] The difference calculation submodule is used to calculate the coordinate difference of the target object based on coordinate 1 and coordinate 2;

[0146] The matching determination submodule is used to determine that if the coordinate difference is within a set range, the matching object is the corresponding target object in the scene image.

[0147] The annotation submodule is used to annotate matching objects in the 3D environmental point cloud map as corresponding target objects, generating an omnidirectional 3D photovoltaic field map.

[0148] Specifically, the photovoltaic robot uses multiple coordinate systems, such as the world coordinate system, the vision camera coordinate system, and the LiDAR coordinate system. The coordinate acquisition submodule acquires the coordinates of the target photovoltaic module in the world coordinate system. Then, the segmentation and recognition submodule uses algorithms to identify various target objects in the 3D environment point cloud map. Next, the matching and selection submodule selects the target object in the 3D environment point cloud map whose coordinates are closest to those of each photovoltaic module as the matching object. The difference calculation submodule calculates the coordinate difference between each photovoltaic module and the matching object. If the coordinate difference is within the allowable error range, the matching determination submodule determines that the matching object is the photovoltaic module corresponding to the scene image captured by the vision camera. Finally, the annotation submodule annotates each matched object in the 3D environment point cloud map, generating an omnidirectional 3D photovoltaic field map.

[0149] In one embodiment, based on the above embodiments, the photovoltaic robot is equipped with a robotic arm for transporting the first target photovoltaic module; wherein:

[0150] The environmental point cloud acquisition module is also used to detect a local point cloud map of the current scene using the lidar installed on the photovoltaic robot;

[0151] The positioning and navigation module specifically includes:

[0152] The positioning submodule is used to match the local point cloud map of the current scene with the omnidirectional three-dimensional photovoltaic field map to locate the current position of the photovoltaic robot;

[0153] The marker determination submodule is used to determine the location of the first target photovoltaic module to be transported, as well as the target location for transport, in the omnidirectional three-dimensional photovoltaic field map;

[0154] The path planning submodule is used to plan a first movement path of the photovoltaic robot before transporting the photovoltaic robot, and a second movement path during transporting the photovoltaic robot, based on the current position of the photovoltaic robot, the position of the first target photovoltaic module, and the target position to be transported.

[0155] The mobile control submodule is used to drive the photovoltaic robot to the position of the first target photovoltaic module according to the first mobile path;

[0156] The pose acquisition submodule is used to acquire the current pose of the robotic arm of the photovoltaic robot;

[0157] The grasping control submodule is used to control the photovoltaic robot to grasp the first target photovoltaic module by means of the robotic arm, based on the current pose of the robotic arm of the photovoltaic robot and the position of the first target photovoltaic module.

[0158] The mobile control submodule is further configured to, after grasping the first target photovoltaic module, control the photovoltaic robot to transport the first target photovoltaic module to the target location according to the second movement path.

[0159] Specifically, the photovoltaic robot can locate its current position by matching a local map with an omnidirectional 3D photovoltaic site map. Furthermore, the photovoltaic robot is equipped with a robotic arm for transporting objects. After determining the target location of the object to be transported, the path planning submodule can plan the movement path before and during transport, and then drive the photovoltaic robot to transport the target object to the target location.

[0160] Preferably, before the environmental point cloud acquisition module detects the local point cloud map of the current scene, the method further includes: identifying whether there is a first target photovoltaic module to be moved in the current scene through a visual camera; if so, triggering the environmental point cloud acquisition module to detect the local point cloud map of the current scene so that the positioning submodule can locate the current position of the photovoltaic robot.

[0161] Of course, after the photovoltaic robot arrives at the location of the first target photovoltaic module, a vision camera can be used to further identify whether the current target is the real first target photovoltaic module to be moved, in order to avoid moving the wrong module.

[0162] In one embodiment, based on any of the above embodiments, the photovoltaic robot is further equipped with a cleaning brush for cleaning the second target photovoltaic module; the positioning and navigation device also includes:

[0163] A cleaning control module is used to clean the second target photovoltaic module by means of a cleaning brush installed on the photovoltaic robot after the photovoltaic robot arrives at the destination where the second photovoltaic module is located.

[0164] Furthermore, during the cleaning of the second target photovoltaic module, the distance between the brush handle of the cleaning brush and the surface of the second target photovoltaic module is obtained in real time by a lidar or ultrasonic sensor installed on the photovoltaic robot, so as to control the pressure applied by the cleaning brush to the second target photovoltaic module.

[0165] Specifically, the cleaning control module can be used to control the photovoltaic robot to move near the photovoltaic module that needs to be cleaned. Then, the laser radar or ultrasonic sensor installed on the photovoltaic robot can sense the relative distance between the cleaning brush and the surface of the photovoltaic module in real time, so as to control the strength of the cleaning brush and achieve a better cleaning effect.

[0166] These can be implemented using computer-executable program code, thus allowing them to be stored in a storage device for execution by a computing device, or fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, this invention is not limited to any particular hardware and software combination.

[0167] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0168] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0169] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0170] Furthermore, the functional units in the various embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0171] It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A photovoltaic robot localization and navigation method based on SLAM technology, characterized in that, include: The environmental point cloud map of the photovoltaic robot is obtained by a rotatable lidar mounted on the photovoltaic robot; Scene images of the photovoltaic robot are acquired using a vision camera installed on the photovoltaic robot; The environmental point cloud image is subjected to noise reduction filtering to obtain the effective point cloud in the environmental point cloud image; Based on the effective point cloud, SLAM is used to build a 3D environment point cloud map; The scene image corresponding to the environmental point cloud map at the same time is input into the trained target recognition model, and the target objects in the scene image are identified and output, including the target photovoltaic module; Obtain the coordinates (1) of the target photovoltaic module identified in the scene image in the world coordinate system; The target photovoltaic module in the 3D environmental point cloud map is identified by a segmentation algorithm, and the coordinates of the target photovoltaic module in the 3D environmental point cloud map in the world coordinate system are calculated. The process of identifying the target photovoltaic module in the 3D environmental point cloud map using a segmentation algorithm includes: segmenting the point cloud in the 3D environmental point cloud map using the lengths of the top and bottom borders and the left and right borders of the photovoltaic module as features to obtain the center position of the target module, and using the average value of the point cloud in the depth direction z of the target module as the depth value of the target module. Based on coordinate 1, select the target object that is closest to coordinate 1 from the 3D environment point cloud map as the matching object; Based on coordinate 1 and coordinate 2, obtain the coordinate difference between each target object and the corresponding matching object; The relative coordinates of the lidar and the vision camera based on the world coordinate system are fixed values; if the difference between the coordinate difference and the fixed value is within a set range, the matching object is determined to be the corresponding target photovoltaic module in the scene image. The matching objects in the 3D environmental point cloud map are labeled as the corresponding target photovoltaic modules to generate an omnidirectional three-dimensional photovoltaic site map; Based on the omnidirectional three-dimensional photovoltaic site map, and combined with the obstacle avoidance sensors installed on the photovoltaic robot, the photovoltaic robot is positioned, identified, and navigated to reach its destination.

2. The photovoltaic robot localization and navigation method based on SLAM technology according to claim 1, characterized in that, The photovoltaic robot is also equipped with a robotic arm for transporting the first target photovoltaic module; after acquiring the omnidirectional three-dimensional photovoltaic site image, it also includes: The local point cloud map of the current scene is detected by the lidar installed on the photovoltaic robot; The local point cloud map of the current scene is matched with the omnidirectional 3D photovoltaic site map to locate the current position of the photovoltaic robot; and the position of the first target photovoltaic module to be transported and the target position to be transported are determined in the omnidirectional 3D photovoltaic site map. Based on the current position of the photovoltaic robot, the position of the first target photovoltaic module, and the target position to be transported, a first movement path of the photovoltaic robot before transport and a second movement path during transport are planned. According to the first movement path, the photovoltaic robot is driven to move to the position of the first target photovoltaic module; Obtain the current pose of the robotic arm of the photovoltaic robot; Based on the current pose of the photovoltaic robot's robotic arm and the position of the first target photovoltaic module, the photovoltaic robot is controlled to use the robotic arm to grasp the first target photovoltaic module. After grasping the first target photovoltaic module, the photovoltaic robot is controlled to transport the first target photovoltaic module to the target location according to the second movement path.

3. A photovoltaic robot localization and navigation method based on SLAM technology according to claim 1 or 2, characterized in that, The photovoltaic robot is also equipped with a cleaning brush for cleaning the second target photovoltaic module; the destination of the photovoltaic robot is the location of the second target photovoltaic module; and after controlling the photovoltaic robot to reach the location of the second target photovoltaic module, it further includes: The second target photovoltaic module is cleaned using the cleaning brush mounted on the photovoltaic robot; During the cleaning of the second target photovoltaic module, the distance between the brush handle of the cleaning brush and the surface of the second target photovoltaic module is obtained in real time by the lidar or ultrasonic sensor installed on the photovoltaic robot, so as to control the pressure applied by the cleaning brush to the second target photovoltaic module.

4. A photovoltaic robot positioning and navigation device based on SLAM technology, characterized in that, include: An environmental point cloud acquisition module is used to acquire an environmental point cloud map of the photovoltaic robot using a rotatable lidar mounted on the photovoltaic robot. A scene image acquisition module is used to acquire scene images of the photovoltaic robot through a vision camera installed on the photovoltaic robot; The map building module is used to fuse the environmental point cloud map and the scene image at the same time, and use SLAM to build a map to obtain an omnidirectional three-dimensional photovoltaic field map; The map construction module includes: a denoising and filtering submodule, used to perform denoising and filtering on the environmental point cloud map to obtain effective point clouds in the environmental point cloud map; a map construction submodule, used to build a 3D environmental point cloud map based on the effective point clouds using SLAM; an image recognition submodule, used to input the scene image into a trained target recognition model to identify and output each target object in the scene image, wherein each target object includes a target photovoltaic module; and a matching and fusion submodule, used to match the target photovoltaic module identified in the scene image to the 3D environmental point cloud map to generate an omnidirectional three-dimensional photovoltaic field map. The matching and fusion submodule includes: a coordinate acquisition submodule, used to acquire the coordinates 1 of the target photovoltaic module identified in the scene image in the world coordinate system; and a segmentation and recognition submodule, used to identify the target photovoltaic module in the 3D environment point cloud map through a segmentation algorithm, and calculate the coordinates 2 of the target photovoltaic module in the 3D environment point cloud map in the world coordinate system; wherein, identifying the target photovoltaic module in the 3D environment point cloud map through the segmentation algorithm includes: segmenting the point cloud in the 3D environment point cloud map using the top and bottom borders and left and right border lengths of the photovoltaic module as features to obtain the center position of the target module, and using the depth direction z of the target module. The system uses the average value of the point cloud to calculate the depth of the target component; a matching selection submodule is used to select the target photovoltaic component that is closest to coordinate 1 from the 3D environment point cloud map as the matching object; a difference calculation submodule is used to calculate the coordinate difference between the target photovoltaic component and coordinate 2; a matching determination submodule is used to determine that the matching object is the corresponding target photovoltaic component in the scene image if the coordinate difference is within a set range; and a labeling submodule is used to label the matching object in the 3D environment point cloud map as the corresponding target photovoltaic component, generating an omnidirectional three-dimensional photovoltaic field map. The positioning and navigation module is used to detect the local point cloud map of the current scene based on the omnidirectional three-dimensional photovoltaic site map, using a lidar installed on the photovoltaic robot. Combined with the obstacle avoidance sensor installed on the photovoltaic robot, the module performs positioning identification and navigation control on the photovoltaic robot, and controls the photovoltaic robot to reach the destination.

5. A photovoltaic robot positioning and navigation device based on SLAM technology according to claim 4, characterized in that, The photovoltaic robot is equipped with a robotic arm for transporting the first target photovoltaic module; wherein: The environmental point cloud acquisition module is also used to detect a local point cloud map of the current scene using the lidar installed on the photovoltaic robot; The positioning and navigation module specifically includes: The positioning submodule is used to match the local point cloud map of the current scene with the omnidirectional three-dimensional photovoltaic field map to locate the current position of the photovoltaic robot; The marker determination submodule is used to determine the location of the first target photovoltaic module to be transported, as well as the target location for transport, in the omnidirectional three-dimensional photovoltaic field map; The path planning submodule is used to plan a first movement path of the photovoltaic robot before transporting the photovoltaic robot, and a second movement path during transporting the photovoltaic robot, based on the current position of the photovoltaic robot, the position of the first target photovoltaic module, and the target position to be transported. The mobile control submodule is used to drive the photovoltaic robot to the position of the first target photovoltaic module according to the first mobile path; The pose acquisition submodule is used to acquire the current pose of the robotic arm of the photovoltaic robot; The grasping control submodule is used to control the photovoltaic robot to grasp the first target photovoltaic module by means of the robotic arm, based on the current pose of the robotic arm of the photovoltaic robot and the position of the first target photovoltaic module. The mobile control submodule is further configured to, after grasping the first target photovoltaic module, control the photovoltaic robot to transport the first target photovoltaic module to the target location according to the second movement path.

6. A photovoltaic robot positioning and navigation device based on SLAM technology according to any one of claims 4 or 5, characterized in that, The photovoltaic robot is also equipped with a cleaning brush for cleaning the second target photovoltaic module; The positioning and navigation device further includes: A cleaning control module is used to clean the second target photovoltaic module by means of a cleaning brush installed on the photovoltaic robot after the photovoltaic robot arrives at the destination where the second target photovoltaic module is located; Furthermore, during the cleaning of the second target photovoltaic module, the distance between the brush handle of the cleaning brush and the surface of the second target photovoltaic module is obtained in real time by the lidar or ultrasonic sensor installed on the photovoltaic robot, so as to control the pressure applied by the cleaning brush to the second target photovoltaic module.

Images