UAV building exterior wall intelligent inspection method and system
By generating a high-precision environmental map of the building and determining the optimal field of view, combined with the Mask R-CNN algorithm model, the problem of difficult to balance efficiency and accuracy in drone building exterior wall patrol is solved, and efficient and accurate externalities are achieved Wall disease identification and inspection.
Patent Information
- Application Number
- CN202410709447.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-03
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-06-03
AI Technical Summary
Existing drone building exterior wall patrol technology is difficult to balance the patrol efficiency and the accuracy of identification results. Inappropriate path planning leads to collision obstacles or unclear images during flight, affecting patrol efficiency and accuracy.
Generate a high-precision environmental map of the building, determine the global target inspection area, determine the optimal field of view based on camera parameters and target distance, divide the coverage area and determine the inspection point, collect image information through drones for disease identification, and combine with Mask R -CNN algorithm model for image recognition.
The efficiency and accuracy of building exterior wall inspections are improved, image quality is ensured and the number of inspection points is reduced, and the inspection efficiency is improved while improving the accuracy of inspection results while accurately identifying diseases.
Smart Images

Figure CN118295437B_ABST
Abstract
Description
Technical Field
[0001] The disclosed embodiments relate to the technical field of intelligent building detection, and more particularly to a method and system for intelligent inspection of building exterior walls by drones. Background Art
[0002] The exterior walls of urban buildings are mainly divided into decorated tile walls, glass curtain walls and undecorated cement mortar painted walls. The exterior walls of ordinary high-rise residential buildings and other low-rise buildings are mainly decorated with tiles or cement mortar painted, and super high-rise office buildings are mainly decorated with glass curtain walls. No matter what form the exterior wall is, it will gradually age. The exterior wall facing bricks will collapse due to adhesive materials and construction defects. The glass curtain wall and cement mortar exterior wall will crack and peel due to rain, strong winds and earthquakes. In order to ensure the safety of structural performance and extend the life of the building, a lot of manpower and material resources are required every year for daily operation and maintenance. The exterior wall inspection has become an important part of the daily operation and maintenance of buildings. The traditional exterior wall inspection methods mainly include visual inspection and hammering, but the existing manual operation and maintenance mode can no longer meet the growing demand.
[0003] In order to explore new operation and maintenance models to meet the growing demand for operation and maintenance, the industry has widely used emerging technologies for exterior wall inspection. In related technologies, drones are used to inspect building exterior walls. It is necessary to set up a planning path, generate inspection tasks and execute them by drones, and collect exterior wall images for image recognition of exterior wall diseases. However, there are still some problems in related technologies: it is impossible to achieve a balance between inspection efficiency and the accuracy of inspection recognition results. Summary of the invention
[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a method and system for intelligent inspection of building exterior walls using a drone.
[0005] In a first aspect, an embodiment of the present disclosure provides a method for intelligent inspection of building exterior walls by using a drone, which is applied to a drone inspection platform and includes:
[0006] Generate a high-precision environment map of the building, and determine a global target inspection area of the building exterior wall based on the high-precision environment map of the building;
[0007] Obtaining camera parameters of a camera carried by the drone, and determining an optimal field of view parameter based on the camera parameters and a target distance, wherein the optimal field of view parameter represents the size of an image range of an exterior wall covered by the camera carried by the drone when photographing the exterior wall in a real environment; wherein the target distance represents a vertical distance from an optimal field of view represented by the optimal field of view parameter to the camera of the drone, and the target distance is at least related to the camera parameters and the image quality of the wall captured by the drone;
[0008] Based on the optimal field of view parameters, multiple optimal field of view ranges covering the global target inspection area are determined, multiple projection points corresponding to the centers of the multiple optimal field of view ranges in the global target inspection area of the building exterior wall are determined, and the multiple projection points are used as multiple inspection points;
[0009] An inspection path for the drone is generated based on the multiple inspection points, and an inspection task is generated based on the inspection path and sent to the drone; image information of the building collected and transmitted when the drone performs the inspection task is received, and the quality of the building's exterior wall is determined based on the image information.
[0010] In one embodiment, generating a high-precision building environment map includes:
[0011] Acquire a building information model (BIM) of a building, acquire geometric information and non-geometric information of the building from the BIM, extract the outer boundary information of the building from the geometric information, and reconstruct the three-dimensional outer boundary of the building based on the outer boundary information of the building and the non-geometric information to obtain a high-precision environment map of the building.
[0012] In one embodiment, the method further comprises:
[0013] When it is determined based on the image information that there is a defect in the quality of the building's exterior wall, the coordinates of the defect position in the image information are located, the defect position coordinates are converted into world coordinates, and the local target inspection area of the UAV is determined based on the world coordinates. Within the local target inspection area, the focal length of the UAV's camera is controlled to be changed to specifically view the defect information.
[0014] In one embodiment, the method further comprises:
[0015] The image information is input into a pre-trained image recognition model to output building exterior wall disease information; wherein the building exterior wall disease information at least includes the area size and area location coordinates of the disease, and the image recognition model is pre-trained based on a Mask R-CNN algorithm model with sample images of building exterior walls.
[0016] In one embodiment, determining a plurality of optimal viewing ranges covering the global target inspection area based on the optimal viewing range parameters includes:
[0017] Based on the optimal field of view parameter, the global target inspection area is divided into a plurality of rectangular sub-areas of the same size; wherein the area size of each rectangular sub-area matches the optimal field of view size represented by the optimal field of view parameter;
[0018] The optimal field of view represented by the optimal field of view parameter is covered for each rectangular sub-area to obtain multiple optimal field of view ranges covering the global target inspection area.
[0019] In one embodiment, the global target inspection area of the building exterior wall includes multiple different inspection areas, and the multiple different inspection areas are divided by the shape and structure of the building; the method also includes: determining the width of each of the multiple different inspection areas, and removing the inspection areas whose width is less than a preset safety distance.
[0020] In one embodiment, when the camera parameters are determined, the target distance is negatively correlated with the image quality.
[0021] In a second aspect, the present disclosure provides a drone building exterior wall intelligent inspection system, including:
[0022] An area generation module is used to generate a high-precision environment map of the building, and determine the global target inspection area of the building exterior wall based on the high-precision environment map of the building;
[0023] A parameter determination module, used to obtain camera parameters of a camera carried by a drone, and determine an optimal field of view parameter based on the camera parameters and a target distance, wherein the optimal field of view parameter represents the size of an image range of an exterior wall covered by the camera carried by the drone when photographing the exterior wall in a real environment; wherein the target distance represents a vertical distance from an optimal field of view represented by the optimal field of view parameter to the camera of the drone, and the target distance is at least related to the camera parameters and the image quality of the wall collected by the drone;
[0024] A patrol point determination module, used to determine multiple optimal visual ranges covering the global target patrol area based on the optimal visual range parameters, determine multiple projection points corresponding to the centers of the multiple optimal visual ranges in the global target patrol area of the building exterior wall, and use the multiple projection points as multiple patrol points;
[0025] The inspection processing module is used to generate an inspection path for the drone based on the multiple inspection points, generate an inspection task based on the inspection path and send it to the drone; receive image information of the building collected and transmitted when the drone performs the inspection task, and determine the quality of the building's exterior wall based on the image information.
[0026] In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for intelligent inspection of building exterior walls by a drone of any of the above-mentioned embodiments.
[0027] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, including:
[0028] Processor; and
[0029] Memory for storing computer programs;
[0030] Wherein, the processor is configured to execute the drone building exterior wall intelligent inspection method described in any of the above embodiments by executing the computer program.
[0031] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:
[0032] The method and system for intelligent inspection of building exterior walls by using a drone provided in the embodiments of the present disclosure generate a high-precision environment map of the building, determine a global target inspection area of the building exterior wall based on the high-precision environment map of the building; obtain camera parameters of a camera carried by the drone, determine an optimal field of view parameter based on the camera parameters and a target distance, wherein the optimal field of view parameter represents the size of the image range of the exterior wall covered by the camera carried by the drone when photographing the exterior wall in a real environment; wherein the target distance represents the vertical distance from the optimal field of view represented by the optimal field of view parameter to the camera of the drone, and the target distance is at least related to the camera parameters and the image quality of the wall collected by the drone; based on the optimal field of view parameter, determine multiple optimal field of view covering the global target inspection area, determine multiple projection points corresponding to the centers of the multiple optimal field of view in the global target inspection area of the building exterior wall, and use the multiple projection points as multiple inspection points; generate an inspection path of the drone based on the multiple inspection points, generate an inspection task based on the inspection path and send it to the drone; receive image information of the building collected and transmitted by the drone when performing the inspection task, and determine the quality of the building exterior wall based on the image information. The solution of this embodiment provides a reliable basis for path planning based on the high-precision environmental map of the building, so that the inspection area can be accurately determined and the inspection points can be finally determined to generate inspection tasks. When determining the inspection points and inspection paths, the optimal field of view of the camera is considered and associated with the quality of the acquired image and the camera parameters. The inspection area is divided accordingly to set inspection points with appropriate positions and numbers. While ensuring that clear images are acquired, an excessive number of inspection points are not set. In this way, the inspection efficiency can be improved as much as possible under the premise of accurately identifying the defects of the building's exterior walls. That is, while improving the inspection efficiency of the building's exterior walls, the accuracy of the inspection results of the building's exterior walls can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0035] Figure 1 This is a flow chart of the intelligent inspection method for building exterior walls using a drone according to an embodiment of the present disclosure;
[0036] Figure 2 Schematic diagram of the process of determining the optimal field of view parameters according to the embodiment of the present disclosure;
[0037] Figure 3 This is a schematic diagram of the inspection area covered by the optimal field of view according to the embodiment of the present disclosure;
[0038] Figure 4 A schematic diagram of a process for determining inspection points in an embodiment of the present disclosure;
[0039] Figure 5 This is a schematic diagram of a drone building exterior wall intelligent inspection system according to an embodiment of the present disclosure;
[0040] Figure 6 Schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0041] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0043] It should be understood that, in the following, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0044] The related art cannot achieve a balance between inspection efficiency and inspection recognition result accuracy. There are many reasons for this. For example, improper path planning may lead to collision with obstacles during flight and repeated flight, or poor flight path, such as too many inspection points and too long flight time, which may reduce inspection efficiency. At the same time, improper path planning, such as inappropriate inspection point locations, may lead to unclear images collected, which may reduce the accuracy of inspection recognition results. To this end, the disclosed embodiments provide a method and system for intelligent inspection of building exterior walls by drones.
[0045] Figure 1 The present invention is a flowchart of a method for intelligent inspection of building exterior walls by using a drone in an embodiment of the present invention. The method can be applied to a drone (Unmanned Aerial Vehicle) inspection platform such as a computer or server that can communicate with a drone, such as wireless communication (such as 5G network communication, etc.). The method may include the following steps:
[0046] Step S101: Generate a high-precision building environment map, and determine a global target inspection area of the building exterior wall based on the high-precision building environment map.
[0047] At present, some unmanned aerial vehicles represented by UAVs do not have the ability to recognize the environment. It is necessary to convert the building environment to be inspected into a digital environment map that can be understood by the machine. The generated high-precision building environment map can provide a high-precision real building environment, which can provide a reliable basis for subsequent path planning, making path planning more accurate. Exemplarily, a high-precision building environment map can be generated for the building to be inspected, and then the global target inspection area of the building's exterior wall is determined based on the high-precision environment map, that is, the inspection object, that is, the building's exterior wall, is selected from the high-precision building environment map, and the global target inspection area of the building's exterior wall is determined, such as selecting a part of the entire exterior wall as the target inspection area, but it is not limited to this.
[0048] Step S102: Obtain camera parameters of the camera carried by the drone, and determine optimal field of view parameters based on the camera parameters and the target distance, wherein the optimal field of view parameters represent the size of the image range of the exterior wall covered by the camera carried by the drone when photographing the exterior wall in a real environment; wherein the target distance represents the vertical distance from the optimal field of view represented by the optimal field of view parameters to the camera of the drone, and the target distance is at least related to the camera parameters and the image quality of the wall captured by the drone.
[0049] For example, the camera parameters may include the focal length F l , the size of the camera sensor, such as width S w and height S h . refer to Figure 2 As shown, the optimal field of view parameters of the camera can be determined by the following formula:
[0050]
[0051] Among them, FOV h Indicates the horizontal FOV, FOV v Indicates the vertical FOV, W d Represents the target distance, which is set as needed to determine the optimal field of view parameter FOV. The selection and determination of the optimal FOV is crucial to the efficiency and completeness of inspection image data collection, because it affects the inspection efficiency and the accuracy of the inspection results. For example, the determination of FOV determines how many inspection points are needed to completely cover the outer wall, affecting the results of path planning. At the same time, FOV is related to the target distance W. d When the camera is used to cover the inspection area of the exterior wall, W d In one embodiment, when the camera parameters are determined, the target distance is negatively correlated with the image quality, that is, the image quality collected by the camera is negatively correlated with W. d Negatively correlated, W d The larger the value, the lower the image quality, resulting in poorer recognition and quantification of exterior wall defects, that is, low accuracy of inspection results. d A balance must be struck between accuracy and image quality to ensure that exterior wall inspection efficiency is maximized while still being able to accurately identify and quantify defects.
[0052] Taking the above reasons into consideration in this embodiment, after the drone model is determined, the camera parameters of the camera it carries are fixed. Therefore, the optimal FOV can be determined by setting a reasonable target distance based on the image quality requirements. Selecting the optimal FOV can determine, for example, the number of subsequent inspection points and the efficiency and completeness of image data collection. That is, selecting the optimal FOV can improve the efficiency of exterior wall inspections while improving the accuracy of exterior wall inspection results.
[0053] Step S103: Based on the optimal field of view parameters, determine multiple optimal field of view ranges covering the global target inspection area, determine multiple projection points corresponding to the centers of the multiple optimal field of view ranges in the global target inspection area of the building exterior wall, and use the multiple projection points as multiple inspection points.
[0054] Exemplary, reference Figure 3 As shown in , after determining the optimal FOV, multiple optimal field of view ranges covering the global target inspection area can be determined in the global target inspection area of the building exterior wall in the building high-precision environment map. The global target inspection area usually requires multiple different optimal field of view ranges to be fully covered. For example, for the height T shown in the figure h and width T w The surface of the outer wall in the global target inspection area gradually covers multiple optimal field of view ranges of size Ri. Figure 4 As shown in , for example, taking the surface of the outer wall in the global target inspection area TIA as a reference, determine the multiple projection points corresponding to the centers of multiple optimal field of view ranges (FOV centers) in the global target inspection area of the building's outer wall. It should be noted here that after the optimal FOV is determined, the multiple optimal field of view ranges covering the global target inspection area are the same in size, but the positions may be different (not shown). For example, when there are protrusions or obstacles on the surface of the outer wall in the global target inspection area, the optimal field of view range covering the protrusion or obstacle area is the surface covering the protrusion or obstacle away from the wall. Therefore, the positions of multiple inspection points determined based on multiple projection points may not be exactly the same. The above method can determine the appropriate position and number of inspection points, providing a basis for subsequent path planning.
[0055] Among them, multiple inspection points are a group of locations where the UAV best hovers to take pictures to identify the disease. The relevant technology focuses on optimizing the inspection path to obtain the minimum number of inspection points, but this will sacrifice the coverage area to improve efficiency, and will not be able to collect images of some exterior wall areas, resulting in missed inspections of diseases, affecting the accuracy of the inspection results. In this embodiment, in order to ensure that the accuracy of the exterior wall inspection results is improved while improving the efficiency of the exterior wall inspection, the exterior wall is fully covered when the inspection points are generated. At the same time, in order to ensure image quality and UAV safety, the solution based on the above-mentioned projection points allows the UAV working plane to be parallel to the global target inspection area TIA to ensure image quality and UAV safety, avoid repeated flights that affect the inspection efficiency, thereby improving the accuracy of the exterior wall inspection results while improving the efficiency of the exterior wall inspection.
[0056] Step S104: Generate an inspection path for the drone based on the multiple inspection points, generate an inspection task based on the inspection path and send it to the drone; receive image information of the building collected and transmitted when the drone performs the inspection task, and determine the quality of the building's exterior wall based on the image information.
[0057] Exemplarily, after determining multiple inspection points, a path planning algorithm can be used to generate an inspection path. The path planning algorithm can be, for example, an A* algorithm or a genetic algorithm. After obtaining the inspection path, a UAV inspection task with the lowest flight cost can be generated so that the drone collects exterior wall image data in the real world, and identifies and judges the quality of the exterior wall of the building, such as the presence of defects such as cracks, based on the exterior wall image data of the building collected and transmitted by the drone. Among them, the inspection task may include but is not limited to the hovering position of each inspection point, such as world coordinates, camera gimbal attitude angle, such as yaw angle, focal length, photo shooting instructions, etc.
[0058] The solution of this embodiment provides a reliable basis for path planning based on the high-precision environmental map of the building, so that the inspection area can be accurately determined and the inspection points can be finally determined to generate inspection tasks. When determining the inspection points and inspection paths, the optimal field of view of the camera is considered and associated with the quality of the acquired image and the camera parameters. The inspection area is divided accordingly to set inspection points with appropriate positions and numbers. While ensuring that clear images are acquired, an excessive number of inspection points are not set. In this way, the inspection efficiency can be improved as much as possible under the premise of accurately identifying the defects of the building's exterior walls. That is, while improving the inspection efficiency of the building's exterior walls, the accuracy of the inspection results of the building's exterior walls can be improved.
[0059] It should be noted that the high-precision environment map of the building provides a basis for subsequent path planning, which ultimately affects the inspection efficiency of the building exterior walls and the accuracy of the inspection results as described above. In order to improve the inspection efficiency of the building exterior walls and the accuracy of the inspection results, in one embodiment, the step of generating a high-precision environment map of the building in step S101 may include: obtaining a building information model BIM of the building, obtaining geometric information and non-geometric information of the building from the BIM, extracting the outer boundary information of the building from the geometric information, and reconstructing the three-dimensional outer boundary of the building based on the outer boundary information of the building and the non-geometric information to obtain a high-precision environment map of the building.
[0060] Exemplarily, geometric information may include, but is not limited to, the location, size, and spatial topological relationship of the components of a building. Non-geometric information may include, but is not limited to, the unique identification ID, name, function and performance technical parameters, specifications, materials, construction sections or methods, etc. of the components of a building. Geometric information is the key to extracting and reconstructing the boundaries of a building. It can accurately describe the spatial relationship of components and realize component positioning. Non-geometric information can fully describe all the properties of components, which is conducive to the accurate reconstruction of buildings. The method for extracting geometric information can be the element bounding box method, the element UV coordinate system method, etc., without limitation. In this embodiment, the element UV coordinate system method can be used to extract geometric information from BIM.
[0061] BIM has rich information and millimeter-level precision, and can provide a high-precision real building environment. Although BIM has been widely used in building surface inspections, including planning paths and visual management, there is currently no research that can fully utilize the geometric and non-geometric information of the BIM model to construct a high-precision environmental map of the building and plan subsequent inspection paths accordingly. The solution of this embodiment builds a high-precision environmental map based on BIM, and can establish a high-precision environmental map relatively quickly and at low cost, because existing methods for identifying boundaries and inspection objects, such as remote sensing image recognition or laser radar scanning recognition, cannot strike a balance between cost, time, and accuracy. In actual engineering, establishing a high-precision environmental map often requires higher costs and additional time. In addition, the geometric and non-geometric information of the building obtained based on BIM in this embodiment can more accurately construct a high-precision environmental map of the building, provide an accurate basic map for subsequent path planning, and make the path planning results more accurate, thereby improving the efficiency of building exterior wall inspections and the accuracy of inspection results.
[0062] In one embodiment, the method may also include: when it is determined based on the image information that there is a defect in the quality of the building's exterior wall, locating the coordinates of the defect position in the image information, converting the defect position coordinates into world coordinates, determining a local target inspection area of the drone based on the world coordinates, and controlling the change of the focal length of the drone's camera within the local target inspection area to specifically view the defect information.
[0063] Specifically, the position coordinates of the disease in the acquired image need to be converted into world coordinates. This is a complex process involving the conversion between multiple coordinate systems, including the conversion from pixel coordinates to image physical coordinates, from image physical coordinates to camera coordinates, and from camera coordinates to world coordinates.
[0064] For example, the defects may be cracks in the exterior wall, peeling of the wall skin, hollowing, etc., without limitation. When the drone performs the inspection task, it can judge in real time based on the collected image recognition whether there are problems such as defects in the exterior wall quality. When it is determined that there are defects, it can locate the area that needs to be inspected, that is, the local target inspection area, and change the position of the drone and / or the camera focal length to further focus on inspecting the defect information.
[0065] The farther the drone is from the inspection area, the larger the area that can be inspected at each inspection point, and the shorter the time to complete the inspection. However, the farther the drone is from the inspection area, the smaller the image of the exterior wall disease in the camera, which leads to inaccurate identification and quantification results of the disease. In order to fully utilize the advantages of high efficiency of long-distance inspections and overcome the disadvantages of low accuracy of inspection results, the embodiments of the present disclosure propose a "global-local" mechanism for adaptively adjusting inspection tasks to identify diseases at close range to obtain more detailed information. The "global-local" adaptive inspection mechanism can fully utilize the advantages of high efficiency of long-distance inspections and overcome the disadvantages of low accuracy of inspection results, thereby improving the efficiency of exterior wall inspections while improving the accuracy of exterior wall inspection results.
[0066] In one embodiment, the method may further include: inputting the image information into a pre-trained image recognition model to output building exterior wall disease information; wherein the building exterior wall disease information at least includes the area size and area location coordinates of the disease, and the image recognition model is pre-trained based on a Mask R-CNN algorithm model with sample images of the building exterior wall.
[0067] Specifically, during the inspection process of the drone, each time the drone arrives at an inspection point, it will hover and control the camera to take photos and transmit them to the platform. Then, the platform calls the image recognition model to identify the collected images and outputs the building exterior wall disease information such as the area size of the disease and the coordinates of the area location. The image recognition model used to identify the disease is pre-trained based on the Mask R-CNN algorithm model based on sample images of the building exterior wall. In other embodiments, the drone can recognize the image by itself, such as performing edge computing based on the image recognition model to transmit the recognition results back to the platform.
[0068] Deep learning methods have better performance in various computer vision tasks and can automatically extract rich features in images. In this embodiment, a deep learning method is used. Based on the recognition of various types of diseases in the external wall disease inspection process, the corresponding Mask R-CNN algorithm model training is selected to identify and locate various diseases in the image. The Mask R-CNN algorithm model can adapt to multiple instance segmentations. The image recognition model trained accordingly can accurately identify various external wall diseases and improve the accuracy of the external wall inspection results.
[0069] In one embodiment, the step of determining multiple optimal viewing ranges covering the global target inspection area based on the optimal viewing range parameters in step S103 may specifically include: dividing the global target inspection area into multiple rectangular sub-areas of the same size based on the optimal viewing range parameters; wherein the area size of each of the rectangular sub-areas matches the size of the optimal viewing range represented by the optimal viewing range parameters; and covering the optimal viewing range represented by the optimal viewing range parameters for each of the rectangular sub-areas to obtain multiple optimal viewing ranges covering the global target inspection area.
[0070] Please continue to refer to Figure 3 and Figure 4 As shown, the area size of each rectangular sub-region is the same as the optimal field of view represented by the optimal field of view parameter FOV, and the global target inspection area is divided into multiple rectangular sub-regions of the same size, and each rectangular sub-region corresponds to an optimal field of view represented by the optimal field of view parameter FOV, thereby obtaining multiple optimal field of view covering the global target inspection area.
[0071] In one embodiment, the global target inspection area of the building exterior wall includes multiple different inspection areas, and the multiple different inspection areas are divided by the shape and structure of the building; the method also includes: determining the width of each of the multiple different inspection areas, and removing the inspection areas whose width is less than a preset safety distance.
[0072] Exemplarily, due to the complexity of the building structure and shape, buildings are usually irregular polyhedrons rather than cubes or cuboids, and it is impossible to use a fixed global target inspection area to cover the entire exterior wall. In order to ensure maximum coverage of the exterior wall during the inspection, the surface can be divided into multiple different inspection areas according to the shape and structure of the building. However, there are some small and unimportant inspection areas corresponding to local exterior wall parts on the original exterior wall, and their existence not only prevents the camera from covering it within a safe distance, but also greatly reduces the inspection efficiency. For this reason, in this embodiment, relatively small inspection areas will be deleted, and two adjacent inspection areas divided by them can be merged, wherein specifically, the inspection areas with a width less than the preset safety distance can be removed, which has little effect on the inspection range and the accuracy of the inspection results, but can improve the inspection efficiency.
[0073] It should be noted that, although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc. In addition, it is also easy to understand that these steps may be, for example, executed synchronously or asynchronously in multiple modules / processes / threads.
[0074] like Figure 5 As shown, the embodiment of the present disclosure provides a drone building exterior wall intelligent inspection system, including:
[0075] The area generation module 501 is used to generate a high-precision environment map of the building, and determine the global target inspection area of the building exterior wall based on the high-precision environment map of the building;
[0076] The parameter determination module 502 is used to obtain the camera parameters of the camera carried by the drone, and determine the optimal field of view parameters based on the camera parameters and the target distance, wherein the optimal field of view parameters represent the size of the image range of the exterior wall covered by the camera carried by the drone when photographing the exterior wall in a real environment; wherein the target distance represents the vertical distance from the optimal field of view represented by the optimal field of view parameter to the camera of the drone, and the target distance is at least related to the camera parameters and the image quality of the wall collected by the drone;
[0077] A patrol point determination module 503 is used to determine multiple optimal visual ranges covering the global target patrol area based on the optimal visual range parameters, determine multiple projection points corresponding to the centers of the multiple optimal visual ranges in the global target patrol area of the building exterior wall, and use the multiple projection points as multiple patrol points;
[0078] The inspection processing module 504 is used to generate an inspection path for the drone based on the multiple inspection points, generate an inspection task based on the inspection path and send it to the drone; receive image information of the building collected and transmitted when the drone performs the inspection task, and determine the quality of the building's exterior wall based on the image information.
[0079] In one embodiment, the area generation module generates a high-precision environment map of the building, including: obtaining a building information model (BIM) of the building, obtaining geometric information and non-geometric information of the building from the BIM, extracting the outer boundary information of the building from the geometric information, and reconstructing the three-dimensional outer boundary of the building based on the outer boundary information of the building and the non-geometric information to obtain a high-precision environment map of the building.
[0080] In one embodiment, the system may also include a local inspection module, which is used to: when it is determined based on the image information that there is a defect in the quality of the building's exterior wall, locate the coordinates of the defect position in the image information, convert the defect position coordinates into world coordinates, determine the local target inspection area of the drone based on the world coordinates, and control the change of the focal length of the drone's camera within the local target inspection area to specifically view the defect information.
[0081] In one embodiment, the system may also include a recognition module, which is used to: input the image information into a pre-trained image recognition model to output building exterior wall disease information; wherein the building exterior wall disease information at least includes the area size and area location coordinates of the disease, and the image recognition model is pre-trained based on a MaskR-CNN algorithm model with sample images of the building exterior wall.
[0082] In one embodiment, the inspection point determination module determines a plurality of optimal viewing ranges covering the global target inspection area based on the optimal viewing range parameter, including:
[0083] Based on the optimal field of view parameter, the global target inspection area is divided into a plurality of rectangular sub-areas of the same size; wherein the area size of each rectangular sub-area matches the optimal field of view size represented by the optimal field of view parameter;
[0084] The optimal field of view represented by the optimal field of view parameter is covered for each rectangular sub-area to obtain multiple optimal field of view ranges covering the global target inspection area.
[0085] In one embodiment, the global target inspection area of the building exterior wall includes multiple different inspection areas, and the multiple different inspection areas are divided by the shape and structure of the building; the system may also include a deletion module for: determining the width of each of the multiple different inspection areas, and removing the inspection areas whose width is less than a preset safety distance.
[0086] In one embodiment, when the camera parameters are determined, the target distance is negatively correlated with the image quality.
[0087] Regarding the system in the above embodiment, the specific manner in which each module performs operations and the corresponding technical effects brought about have been described in detail in the embodiment of the method, and will not be elaborated here.
[0088] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of a module or unit described above can be further divided into multiple modules or units for concretization. The components displayed as modules or units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying creative work.
[0089] The embodiments of the present disclosure also provide a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for intelligent inspection of building exterior walls by using a drone as described in any of the embodiments above is implemented.
[0090] Exemplarily, the readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.
[0091] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.
[0092] The present disclosure also provides an electronic device, including a processor and a memory, wherein the memory is used to store a computer program, wherein the processor is configured to execute the drone building exterior wall intelligent inspection method in any of the above embodiments by executing the computer program.
[0093] Refer to the following Figure 6The electronic device 600 according to this embodiment of the present invention is described. Figure 6 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0094] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.
[0095] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method embodiment section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 The steps of the method shown in .
[0096] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .
[0097] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.
[0098] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0099] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. In addition, the electronic device 600 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 through a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0100] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the method steps of the above-mentioned embodiments according to the implementation methods of the present disclosure.
[0101] It should be noted that, in this article, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0102] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A drone building exterior wall intelligent inspection method, characterized in that: The method is applied to the UAV inspection platform, including: Generate a high-precision environment map of the building, and determine a global target inspection area of the building's exterior wall based on the high-precision environment map of the building; the global target inspection area is a part of the building's exterior wall; the high-precision environment map of the building includes a high-precision real building environment; Obtaining camera parameters of a camera carried by the drone, and determining an optimal field of view parameter based on the camera parameters and a target distance, wherein the optimal field of view parameter represents the size of an image range of an exterior wall covered by the camera carried by the drone when photographing the exterior wall in a real environment; wherein the target distance represents a vertical distance from an optimal field of view represented by the optimal field of view parameter to the camera of the drone, and the target distance is at least related to the camera parameters and the image quality of the wall captured by the drone; Based on the optimal field of view parameters, multiple optimal field of view ranges covering the global target inspection area are determined, multiple projection points corresponding to the centers of the multiple optimal field of view ranges in the global target inspection area of the building exterior wall are determined, and the multiple projection points are used as multiple inspection points; Generate an inspection path for the drone based on the multiple inspection points, generate an inspection task based on the inspection path and send it to the drone; receive image information of the building collected and transmitted by the drone when performing the inspection task, and determine the quality of the building's exterior wall based on the image information; the high-precision environmental map of the building provides an accurate basic map for the inspection path planning; Among them, the generating of high-precision environment map of the building includes: obtaining the building information model BIM of the building, obtaining geometric information and non-geometric information of the building from the BIM, extracting the outer boundary information of the building from the geometric information, and reconstructing the three-dimensional outer boundary of the building based on the outer boundary information of the building and the non-geometric information to obtain the high-precision environment map of the building; the geometric information includes the position, size and spatial topological relationship of the components of the building; the non-geometric information describes all attributes of the components of the building, including the unique identification ID, name, function and performance technical parameters, specification model, material or material, construction section or construction method of the components of the building.
2. The method according to claim 1, characterized in that The method further includes: When it is determined based on the image information that there is a defect in the quality of the building's exterior wall, the coordinates of the defect position in the image information are located, the defect position coordinates are converted into world coordinates, and the local target inspection area of the UAV is determined based on the world coordinates. Within the local target inspection area, the focal length of the UAV's camera is controlled to be changed to specifically view the defect information.
3. The method according to claim 2, characterized in that The method further includes: The image information is input into a pre-trained image recognition model to output building exterior wall disease information; wherein the building exterior wall disease information at least includes the area size and area location coordinates of the disease, and the image recognition model is pre-trained based on a Mask R-CNN algorithm model with sample images of building exterior walls.
4. The method according to claim 1, characterized in that: The determining, based on the optimal field of view parameter, a plurality of optimal field of view ranges covering the global target inspection area comprises: Based on the optimal field of view parameter, the global target inspection area is divided into a plurality of rectangular sub-areas of the same size; wherein the area size of each rectangular sub-area matches the optimal field of view size represented by the optimal field of view parameter; The optimal field of view represented by the optimal field of view parameter is covered for each rectangular sub-area to obtain multiple optimal field of view ranges covering the global target inspection area.
5. The method according to claim 1, characterized in that The global target inspection area of the building exterior wall includes multiple different inspection areas, and the multiple different inspection areas are divided by the shape and structure of the building; the method also includes: determining the width of each of the multiple different inspection areas, and removing the inspection areas whose width is less than a preset safety distance.
6. The method according to claim 1, characterized in that When the camera parameters are determined, the target distance is negatively correlated with the image quality.
7. An intelligent inspection system for building exterior walls using drones, characterized in that: include: An area generation module is used to generate a high-precision environment map of the building, and determine the global target inspection area of the building exterior wall based on the high-precision environment map of the building; The global target inspection area is a part of the outer wall of the building; the high-precision building environment map includes a high-precision real building environment; A parameter determination module, used to obtain camera parameters of a camera carried by a drone, and determine an optimal field of view parameter based on the camera parameters and a target distance, wherein the optimal field of view parameter represents the size of an image range of an exterior wall covered by the camera carried by the drone when photographing the exterior wall in a real environment; wherein the target distance represents a vertical distance from an optimal field of view represented by the optimal field of view parameter to the camera of the drone, and the target distance is at least related to the camera parameters and the image quality of the wall collected by the drone; A patrol point determination module, used to determine multiple optimal visual ranges covering the global target patrol area based on the optimal visual range parameters, determine multiple projection points corresponding to the centers of the multiple optimal visual ranges in the global target patrol area of the building exterior wall, and use the multiple projection points as multiple patrol points; The inspection processing module is used to generate an inspection path for the drone based on the multiple inspection points, generate an inspection task based on the inspection path and send it to the drone; receive image information of the building collected and transmitted by the drone when performing the inspection task, and determine the quality of the building's exterior wall based on the image information; the high-precision environmental map of the building provides an accurate basic map for the inspection path planning; Among them, the area generation module is specifically used to: obtain the building information model BIM of the building, obtain the geometric information and non-geometric information of the building from the BIM, extract the outer boundary information of the building from the geometric information, and reconstruct the three-dimensional outer boundary of the building based on the outer boundary information of the building and the non-geometric information to obtain a high-precision environmental map of the building; the geometric information includes the position, size and spatial topological relationship of the components of the building; the non-geometric information describes all attributes of the components of the building, including the unique identification ID, name, function and performance technical parameters, specifications, materials or materials, construction sections or construction methods of the components of the building.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the drone building exterior wall intelligent inspection method according to any one of claims 1 to 6 is implemented.
9. An electronic device, characterized in that: include: processor; as well as Memory for storing computer programs; Wherein, the processor is configured to execute the drone building exterior wall intelligent inspection method as described in any one of claims 1 to 6 by executing the computer program.
Citation Information
Cited By
Unmanned aerial vehicle building intelligent inspection method and system
CN120747419B
