Glass curtain wall cleaning robot control system combined with unmanned aerial vehicle aerial photography modeling

Abstract

The invention relates to cleaning robot control, in particular to a glass curtain wall cleaning robot control system combined with unmanned aerial vehicle aerial photography modeling, and the system is characterized in that an unmanned aerial vehicle collects radar data and image data of a building facade and uploads the collected data to an upper computer; the cleaning robot receives the target cleaning path sent by the upper computer, cleans the building facade according to the target cleaning path, detects obstacle information in real time in the cleaning process and uploads the obstacle information to the upper computer; the upper computer is used for performing high-precision three-dimensional modeling on the building facade according to the radar data and the image data to obtain a building facade model; according to the building facade model, the geometric complexity, the real-time position of the cleaning robot and the endurance condition, performing cleaning path planning on each cleaning robot, and sending a target cleaning path to the corresponding cleaning robot; according to the technical scheme, the defects that the cleaning effect on the glass curtain wall of the high-rise building is poor, and the cleaning efficiency is low can be effectively overcome.

Description

Technical Field

[0001] The present invention relates to the control of cleaning robots, and particularly to a control system for a glass curtain wall cleaning robot combined with aerial photography and modeling by a drone. Background Art

[0002] With the acceleration of the urbanization process, high-rise buildings are increasing day by day. Glass curtain walls are widely used in various high-rise buildings due to their beautiful appearance and good lighting properties. However, the cleaning of glass curtain walls faces many challenges. Traditional manual cleaning methods have problems such as high risks in high-altitude operations, low efficiency, and high costs. With the development of robot technology, glass curtain wall cleaning robots have gradually become a viable alternative.

[0003] Although existing glass curtain wall cleaning robots have overcome the disadvantages of manual cleaning to a certain extent, there are still some problems. For example, most cleaning robots can only clean according to a preset cleaning path and cannot intelligently adjust the cleaning path according to the actual environmental changes of the glass curtain wall (such as a window suddenly opening and causing the cleaning robot to encounter an obstacle). In addition, for complex-shaped glass curtain walls, such as arc-shaped and irregular-shaped ones, existing cleaning robots often have difficulty adapting, resulting in poor cleaning effects.

[0004] The rise of drone aerial photography and modeling technology has brought new development opportunities for glass curtain wall cleaning robots. By using a drone to conduct aerial photography and modeling of the building facade, a detailed three-dimensional model of the glass curtain wall can be obtained, including information such as its shape, size, and pollution degree. However, in the existing technology, drones are only used for preliminary exploration and cannot provide real-time assistance for the operation of cleaning robots. Therefore, it is necessary to develop a control system for a glass curtain wall cleaning robot combined with drone aerial photography and modeling to achieve intelligent path planning, adaptive cleaning, and efficient operation of the cleaning robot, and effectively improve the cleaning quality and efficiency of the glass curtain wall. Summary of the Invention

[0005] (1) Technical Problems to be Solved

[0006] In view of the above-mentioned disadvantages of the existing technology, the present invention provides a control system for a glass curtain wall cleaning robot combined with drone aerial photography and modeling, which can effectively overcome the defects of poor cleaning effect and low cleaning efficiency of the existing technology for the glass curtain walls of high-rise buildings.

[0007] (2) Technical Solutions

[0008] To achieve the above object, the present invention is realized through the following technical solutions:

[0009] A control system for a glass curtain wall cleaning robot combined with drone aerial photography and modeling includes a host computer, a drone, a cleaning robot, and a mobile terminal;

[0010] The drone collects radar data and image data of the building facade and uploads the collected data to the host computer;

[0011] The cleaning robot receives the target cleaning path sent by the host computer, cleans the building facade according to the target cleaning path, and detects obstacle information in real time during the cleaning process and uploads the obstacle information to the host computer;

[0012] The host computer performs high-precision 3D modeling on the building facade based on the radar data and image data to obtain a building facade model; plans the cleaning paths for each cleaning robot according to the building facade model, geometric complexity, real-time position and battery life of the cleaning robot, and sends the target cleaning path to the corresponding cleaning robot; controls the drone to perform local resweeping on the area where the obstacle is located after receiving the obstacle information to update the building facade model and the target cleaning paths of each cleaning robot in real time; statistically analyzes the cleaning task data in real time and sends the cleaning task data to the mobile terminal;

[0013] The mobile terminal receives and visually displays the cleaning task data, and at the same time allows the user to adjust the target cleaning paths of each cleaning robot.

[0014] Preferably, a lidar and a high-definition RGB camera are arranged on the drone;

[0015] The lidar collects radar data of the building facade and uploads the radar data to the host computer;

[0016] The high-definition RGB camera collects image data of the building facade and uploads the image data to the host computer.

[0017] Preferably, a contact sensor and an obstacle detection module are arranged on the cleaning robot;

[0018] The contact sensor detects the contact signal with the obstacle in real time during the process of cleaning the building facade according to the target cleaning path and sends the contact signal to the obstacle detection module;

[0019] The obstacle detection module extracts the obstacle information according to the obstacle detection signal and uploads it to the host computer after packaging it with the detection position of the contact signal.

[0020] Preferably, the host computer includes a 3D model construction module, a cleaning path planning module and a cleaning task data statistics module;

[0021] The 3D model construction module performs high-precision 3D modeling on the building facade based on the radar data and image data to obtain a building facade model, and after partitioning the cleaning units of the building facade model, sends it to the cleaning path planning module;

[0022] The cleaning path planning module plans the cleaning paths for each cleaning robot according to the building facade model, geometric complexity, real-time position and battery life of the cleaning robot, and sends the target cleaning paths to the corresponding cleaning robots;

[0023] The cleaning task data statistics module statistically analyzes the cleaning task data in real time and sends the cleaning task data to the mobile terminal;

[0024] Among them, after receiving the obstacle information, the host computer judges the area where the obstacle is located according to the detection position of the contact signal, and controls the drone to perform local resweeping on the area where the obstacle is located to update the building facade model and the target cleaning paths of each cleaning robot in real time.

[0025] Preferably, the three-dimensional model construction module performs high-precision three-dimensional modeling on the building facade according to radar data and image data to obtain the building facade model, including:

[0026] Register the three-dimensional point cloud data, and fuse the three-dimensional point cloud data collected at different times and different angles into a unified coordinate system;

[0027] Coarsely register the image data to ensure that the image data is spatially aligned with the three-dimensional point cloud data;

[0028] Perform fine registration on the three-dimensional point cloud data and the image data to ensure that each point in the three-dimensional point cloud data is spatially aligned with the corresponding pixel point in the image data;

[0029] Map the registered image texture onto the three-dimensional point cloud data to generate a high-precision building facade model including geometric structure and surface texture.

[0030] Preferably, after mapping the registered image texture onto the three-dimensional point cloud data to generate a high-precision building facade model including geometric structure and surface texture, it includes:

[0031] Perform model simplification, redundant point removal, and geometric structure repair and improvement on the building facade model to improve the display effect of the model while ensuring the accuracy and integrity of the model;

[0032] Perform refined processing on the key areas of the building facade model to enhance the detail performance ability of the model;

[0033] Among them, the key areas include doors, windows and decorative lines.

[0034] Preferably, the cleaning path planning module plans the cleaning paths for each cleaning robot according to the building facade model, geometric complexity, real-time position and battery life of the cleaning robot, including:

[0035] Integrate the building facade model, geometric complexity, real-time position, and battery life of cleaning robots into a unified environment;

[0036] Determine the initial positions and target cleaning units of each cleaning robot, and allocate cleaning tasks according to the battery life of the cleaning robots and the requirements of the cleaning tasks, ensuring that each cleaning robot can complete its own cleaning tasks within the battery life range;

[0037] Combined with the kinematic constraints and battery life of the cleaning robots, use an algorithm suitable for multi-agent path planning to perform cleaning path planning for each cleaning robot to obtain the target cleaning paths of each cleaning robot.

[0038] Preferably, after combining the kinematic constraints and battery life of the cleaning robots and using an algorithm suitable for multi-agent path planning to perform cleaning path planning for each cleaning robot to obtain the target cleaning paths of each cleaning robot, it includes:

[0039] Remove redundant nodes and adjust the path directions of the target cleaning paths of each cleaning robot to improve the smoothness and executability of the paths;

[0040] Further fine-tune the target cleaning paths of each cleaning robot in combination with the geometric complexity of the building facade to ensure that each cleaning robot can reach the corresponding target cleaning unit smoothly;

[0041] Further adjust the target cleaning paths of each cleaning robot in real time in combination with the battery life of the cleaning robots to ensure that each cleaning robot can complete its own cleaning tasks smoothly.

[0042] Preferably, the cleaning task data statistics module performs real-time statistics on the cleaning task data and sends the cleaning task data to the mobile terminal, including:

[0043] Perform real-time statistics on the current working time, completed task volume, remaining working time, and remaining task volume of each cleaning robot, and send the cleaning task data of each cleaning robot to the mobile terminal.

[0044] Preferably, a local area network is formed between the host computer, the drone, the cleaning robot, and the mobile terminal through wireless communication or a mobile phone network card.

[0045] (III) Beneficial Effects

[0046] Compared with the prior art, in a control system of a glass curtain wall cleaning robot combined with UAV aerial photography modeling provided by the present invention, a host computer performs high-precision three-dimensional modeling on the building facade according to radar data and image data collected by the UAV to obtain a building facade model, and then plans cleaning paths for each cleaning robot according to the building facade model, geometric complexity, real-time position and battery life of the cleaning robot, and sends the target cleaning paths to the corresponding cleaning robots. The cleaning robots clean the building facade according to the target cleaning paths, and detect obstacle information in real time during the cleaning process and upload the obstacle information to the host computer. After receiving the obstacle information, the host computer controls the UAV to perform local resweeping on the area where the obstacle is located to update the building facade model and the target cleaning paths of each cleaning robot in real time. Therefore, high-precision three-dimensional modeling of the building facade can be carried out by using the UAV, and real-time update can be combined with obstacle information to provide real-time assistance for the operation of the cleaning robot, so that the cleaning robot can better adapt to the glass curtain wall, and at the same time, the target cleaning paths can be intelligently adjusted according to the actual environmental changes of the glass curtain wall, thereby effectively improving the cleaning quality and efficiency of the glass curtain wall. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0048] Figure 1 It is a schematic diagram of the system of the present invention;

[0049] Figure 2 It is a schematic flow diagram of the three-dimensional model construction module in the present invention for performing high-precision three-dimensional modeling on the building facade to obtain a building facade model;

[0050] Figure 3 It is a schematic flow diagram of the cleaning path planning module in the present invention for planning cleaning paths for each cleaning robot. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] In order to make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0052] A control system for a glass curtain wall cleaning robot combined with UAV aerial photography modeling, as Figure 1 shown, includes a host computer, a UAV, a cleaning robot and a mobile terminal;

[0053] The UAV collects radar data and image data of the building facade and uploads the collected data to the host computer;

[0054] The cleaning robot receives the target cleaning path sent by the host computer, cleans the building facade according to the target cleaning path, and detects obstacle information in real time during the cleaning process, and uploads the obstacle information to the host computer;

[0055] The host computer performs high-precision 3D modeling on the building facade according to the radar data and image data to obtain a building facade model; plans the cleaning paths for each cleaning robot according to the building facade model, geometric complexity, real-time position and battery life of the cleaning robot, and sends the target cleaning path to the corresponding cleaning robot; controls the UAV to perform local re-scanning on the area where the obstacle is located after receiving the obstacle information to update the building facade model and the target cleaning paths of each cleaning robot in real time; statistically analyzes the cleaning task data in real time and sends the cleaning task data to the mobile terminal;

[0056] The mobile terminal receives and visually displays the cleaning task data, and at the same time allows the user to adjust the target cleaning paths of each cleaning robot.

[0057] In the technical solution of this application, a lidar and a high-definition RGB camera are set on the UAV;

[0058] The lidar collects radar data of the building facade and uploads the radar data to the host computer;

[0059] The high-definition RGB camera collects image data of the building facade and uploads the image data to the host computer.

[0060] In the technical solution of this application, a contact sensor and an obstacle detection module are set on the cleaning robot;

[0061] The contact sensor detects the contact signal with the obstacle in real time during the process of cleaning the building facade according to the target cleaning path, and sends the contact signal to the obstacle detection module;

[0062] The obstacle detection module extracts the obstacle information according to the obstacle detection signal and uploads it to the host computer in a package with the detection position of the contact signal.

[0063] In the technical solution of this application, the host computer includes a 3D model construction module, a cleaning path planning module and a cleaning task data statistics module;

[0064] 3D model construction module, which performs high-precision 3D modeling on the building facade based on radar data and image data to obtain a building facade model, and divides the building facade model into cleaning units (the area of each cleaning unit is 5-10m 2 ) and then sends them to the cleaning path planning module;

[0065] Cleaning path planning module, which plans the cleaning paths for each cleaning robot according to the building facade model, geometric complexity, real-time position and battery life of the cleaning robot, and sends the target cleaning paths to the corresponding cleaning robots;

[0066] Cleaning task data statistics module, which statistically analyzes the cleaning task data in real time and sends the cleaning task data to the mobile terminal;

[0067] Among them, after receiving the obstacle information, the host computer judges the area where the obstacle is located according to the detection position of the contact signal, and controls the drone to perform local re-scanning on the area where the obstacle is located to update the building facade model and the target cleaning paths of each cleaning robot in real time (it can also output the obstacle crossing timing according to the area where the obstacle is located and execute the sling obstacle crossing action. For example, if a window suddenly opens and causes a convex area to appear in the building facade model, this convex area can be marked as the "area to cross the obstacle", and an obstacle crossing access node is reserved in the target cleaning path. The cleaning robot stops moving at a specified distance from the "area to cross the obstacle" and waits for the host computer to send an obstacle crossing instruction).

[0068] ① The 3D model construction module performs high-precision 3D modeling on the building facade based on radar data and image data to obtain a building facade model, as Figure 2 shown, including:

[0069] Register the 3D point cloud data, and fuse the 3D point cloud data collected at different times and different angles into a unified coordinate system;

[0070] Coarsely register the image data to ensure that the image data is spatially aligned with the 3D point cloud data;

[0071] Perform fine registration on the 3D point cloud data and the image data to ensure that each point in the 3D point cloud data is spatially aligned with the corresponding pixel point in the image data;

[0072] Map the registered image texture onto the 3D point cloud data to generate a high-precision building facade model containing geometric structure and surface texture.

[0073] Specifically, after mapping the registered image texture onto the 3D point cloud data to generate a high-precision building facade model containing geometric structure and surface texture, as Figure 2 shown, including:

[0074] Simplify the building facade model, remove redundant points, and repair and improve the geometric structure to enhance the display effect of the model while ensuring the accuracy and integrity of the model;

[0075] Refine the key areas of the building facade model to improve the detail performance of the model;

[0076] Among them, the key areas include doors, windows, and decorative lines.

[0077] ② The cleaning path planning module plans the cleaning paths for each cleaning robot according to the building facade model, geometric complexity, real-time position, and battery life of the cleaning robot, as Figure 3 shown, including:

[0078] Integrate the building facade model, geometric complexity, real-time position, and battery life of the cleaning robot into a unified environment;

[0079] Determine the initial positions and target cleaning units of each cleaning robot, and allocate cleaning tasks according to the battery life of the cleaning robot and the cleaning task requirements to ensure that each cleaning robot can complete its own cleaning task within the battery life;

[0080] Combined with the kinematic constraints and battery life of the cleaning robot, use an algorithm suitable for multi-agent path planning to plan the cleaning paths for each cleaning robot to obtain the target cleaning paths of each cleaning robot.

[0081] Specifically, after using an algorithm suitable for multi-agent path planning to plan the cleaning paths for each cleaning robot combined with the kinematic constraints and battery life of the cleaning robot to obtain the target cleaning paths of each cleaning robot, as Figure 3 shown, including:

[0082] Remove redundant nodes and adjust the path directions of the target cleaning paths of each cleaning robot to improve the smoothness and executability of the paths;

[0083] Further fine-tune the target cleaning paths of each cleaning robot in combination with the geometric complexity of the building facade to ensure that each cleaning robot can reach the corresponding target cleaning unit smoothly;

[0084] Further adjust the target cleaning paths of each cleaning robot in real time in combination with the battery life of the cleaning robot to ensure that each cleaning robot can complete its own cleaning task smoothly.

[0085] ③ The cleaning task data statistics module statistically analyzes the cleaning task data in real time and sends the cleaning task data to the mobile terminal, including:

[0086] Real-time statistics are carried out on the current working time, the amount of tasks completed, the remaining working time, and the remaining amount of tasks of each cleaning robot, and the cleaning task data of each cleaning robot are sent to the mobile terminal.

[0087] In the technical solution of this application, a local area network is formed among the host computer, the drone, the cleaning robot, and the mobile terminal through wireless communication or a mobile phone network card to facilitate data transmission among them.

[0088] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A control system for a glass curtain wall cleaning robot combined with UAV aerial photography modeling, characterized in that: It includes a host computer, a drone, a cleaning robot, and a mobile terminal; The drone collects radar data and image data of the building facade and uploads the collected data to the host computer; The cleaning robot receives the target cleaning path sent by the host computer, cleans the building facade according to the target cleaning path, and in the process of cleaning, it detects obstacle information in real time and uploads the obstacle information to the host computer; The host computer performs high-precision 3D modeling on the building facade based on the radar data and image data to obtain a building facade model; According to the building facade model, geometric complexity, the real-time position and battery life of the cleaning robot, it plans the cleaning paths for each cleaning robot and sends the target cleaning path to the corresponding cleaning robot; after receiving the obstacle information, it controls the drone to perform local resweeping on the area where the obstacle is located to update the building facade model and the target cleaning paths of each cleaning robot in real time; It statistically analyzes the cleaning task data in real time and sends the cleaning task data to the mobile terminal; The mobile terminal receives and visually displays the cleaning task data, and at the same time allows the user to adjust the target cleaning paths of each cleaning robot.

2. The glass curtain wall cleaning robot control system combined with UAV aerial photography modeling according to claim 1, characterized in that: A lidar and a high-definition RGB camera are provided on the drone; The lidar collects radar data of the building facade and uploads the radar data to the host computer; The high-definition RGB camera collects image data of the building facade and uploads the image data to the host computer.

3. The glass curtain wall cleaning robot control system combined with UAV aerial photography modeling according to claim 2, characterized in that: A contact sensor and an obstacle detection module are provided on the cleaning robot; The contact sensor, in the process of cleaning the building facade according to the target cleaning path, detects the contact signal with the obstacle in real time and sends the contact signal to the obstacle detection module; The obstacle detection module extracts the obstacle information according to the obstacle detection signal and uploads it to the host computer in a package with the detection position of the contact signal.

4. The glass curtain wall cleaning robot control system combined with UAV aerial photography modeling according to claim 3, characterized in that: The host computer includes a 3D model construction module, a cleaning path planning module, and a cleaning task data statistics module; The 3D model construction module performs high-precision 3D modeling on the building facade based on the radar data and image data to obtain a building facade model, and after partitioning the cleaning units of the building facade model, it sends them to the cleaning path planning module; The cleaning path planning module plans the cleaning paths for each cleaning robot according to the building facade model, geometric complexity, the real-time position and battery life of the cleaning robot, and sends the target cleaning path to the corresponding cleaning robot; The cleaning task data statistics module statistically analyzes the cleaning task data in real time and sends the cleaning task data to the mobile terminal; Among them, after the host computer receives the obstacle information, it judges the area where the obstacle is located according to the detection position of the contact signal, and controls the drone to perform local resweeping on the area where the obstacle is located to update the building facade model and the target cleaning paths of each cleaning robot in real time.

5. The control system of the glass curtain wall cleaning robot combined with UAV aerial photography modeling according to claim 4, characterized in that: The 3D model construction module performs high-precision 3D modeling on the building facade based on the radar data and image data to obtain a building facade model, including: Registering the 3D point cloud data, and fusing the 3D point cloud data collected at different times and different angles into a unified coordinate system; Perform coarse registration on the image data to ensure that the image data is spatially aligned with the 3D point cloud data; Perform fine registration of 3D point cloud data and image data to ensure that each point in the 3D point cloud data is spatially aligned with the corresponding pixel in the image data; The registered image texture is mapped onto the 3D point cloud data to generate a high-precision building facade model that includes geometric structure and surface texture.

6. The glass curtain wall cleaning robot control system combined with UAV aerial photography modeling according to claim 5, characterized in that: After mapping the registered image texture onto the three-dimensional point cloud data to generate a high-precision building facade model including geometric structure and surface texture, the method includes: Simplify the building facade model, remove redundant points, and repair and improve the geometric structure to improve the display effect of the model while ensuring the accuracy and integrity of the model; Refine the key areas of the building facade model to improve the model's detail expression capabilities; Among them, key areas include doors, windows and decorative moldings.

7. The control system of the glass curtain wall cleaning robot combined with UAV aerial photography modeling according to claim 4, characterized in that: The cleaning path planning module plans the cleaning path for each cleaning robot based on the building facade model, geometric complexity, the real-time position and endurance of the cleaning robot, including: Integrate building facade models, geometric complexity, and the real-time location and battery life of cleaning robots into a unified environment; Determine the initial position and target cleaning unit of each cleaning robot, and assign cleaning tasks based on the cleaning robot's battery life and cleaning task requirements to ensure that each cleaning robot can complete its cleaning task within its battery life; Combined with the kinematic constraints and endurance of the cleaning robots, an algorithm suitable for multi-agent path planning is used to plan the cleaning path of each cleaning robot and obtain the target cleaning path of each cleaning robot.

8. The glass curtain wall cleaning robot control system combined with UAV aerial photography modeling according to claim 7, characterized in that: The cleaning path planning for each cleaning robot is performed by combining the kinematic constraints and endurance of the cleaning robot and adopting an algorithm suitable for multi-agent path planning. After obtaining the target cleaning path of each cleaning robot, the following steps are included: Remove redundant nodes and adjust the path direction of each cleaning robot's target cleaning path to improve the smoothness and executability of the path; The target cleaning path of each cleaning robot is further fine-tuned based on the geometric complexity of the building facade to ensure that each cleaning robot successfully reaches the corresponding target cleaning unit; Furthermore, the target cleaning path of each cleaning robot is adjusted in real time based on the battery life of the cleaning robot to ensure that each cleaning robot successfully completes its cleaning task.

9. The glass curtain wall cleaning robot control system combined with UAV aerial photography modeling according to claim 4, characterized in that: The cleaning task data statistics module performs real-time statistics on the cleaning task data and sends the cleaning task data to the mobile terminal, including: The current working time, completed tasks, remaining working time and remaining tasks of each cleaning robot are counted in real time, and the cleaning task data of each cleaning robot is sent to the mobile terminal.

10. The control system of the glass curtain wall cleaning robot combined with UAV aerial photography modeling according to any one of claims 1-9, characterized in that: The host computer, the drone, the cleaning robot and the mobile terminal form a local area network through wireless communication or a mobile phone network card.

Images