Large-scale steel structure air-ground collaborative automatic inspection path planning method and inspection system

Abstract

The present invention relates to the technical field of automatic inspection of steel structures, and discloses a large-scale steel structure air-ground collaborative automatic inspection path planning method and inspection system, comprising the following steps: obtaining three-dimensional point cloud data of large-scale steel structures, determining a point cloud set that can fall within the field of view of the drone according to the maximum flight altitude and maximum field of view of a given drone, and obtaining the inspection range of the drone; the point cloud set outside the inspection range of the drone is the inspection range of the wall-climbing robot; and planning the inspection paths of the drone and the wall-climbing robot within the inspection range of the drone and the inspection range of the wall-climbing robot, respectively. Based on two unmanned devices, the present invention realizes high-coverage, high-efficiency, and high-precision unmanned inspection of steel structures.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic inspection of steel structures, and in particular relates to a path planning method and an inspection system for automatic inspection of large-scale steel structures in collaboration between air and ground. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] A major threat to the safe service of large steel structures is the peeling and corrosion of surface paint. However, due to their large size, large size, complex connections among numerous rods, and staggered obstructions, it is difficult for a single automated inspection equipment to achieve a balance between inspection efficiency, coverage, and accuracy.

[0004] At present, steel structure inspection mainly relies on the following methods: manual inspection, drone inspection, ultrasonic inspection and infrared thermal imaging inspection. Manual inspection is often time-consuming and labor-intensive, and there are many blind spots in inspection, especially when working at high altitudes, there are great safety hazards, and it is difficult to meet the needs of large-scale or high-frequency inspections; although drone inspection can cover high altitudes and complex areas, it is difficult to use in bad weather due to weather and lighting conditions, the battery life is limited, and it is easy to produce collection dead angles; ultrasonic inspection is highly dependent on equipment, the inspection speed is slow, and the operator must be in close contact with the structure, so it is difficult to implement in high altitudes and dangerous environments; infrared thermal imaging inspection has high requirements for temperature difference, high equipment cost, and it is difficult to make accurate judgments on all types of defects, and it is easily affected by ambient temperature. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a large-scale steel structure air-ground collaborative automatic inspection path planning method, which realizes full coverage periodic and efficient inspection of large steel structures by allocating inspection tasks and planning paths for drones and wall-climbing robots.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0007] A large-scale steel structure air-ground collaborative automatic inspection path planning method comprises the following steps:

[0008] Obtain the 3D point cloud data of large steel structures, and determine the point cloud set that can fall within the field of view of the drone according to the maximum flight altitude and maximum field of view of the drone, and obtain the inspection range of the drone; the point cloud set outside the inspection range of the drone is the inspection range of the wall-climbing robot;

[0009] The inspection paths of the UAV and the wall-climbing robot are planned within the inspection range of the UAV and the wall-climbing robot respectively.

[0010] In some embodiments, planning an inspection path for a drone within the inspection range of the drone includes:

[0011] According to the point cloud data within the inspection range of the UAV, the outer layer data is extracted and expanded by a set distance to obtain the UAV inspection flight surface by fitting;

[0012] Clustering the point cloud data of steel structures within the inspection range of the drone to obtain point cloud data of multiple steel sections, extracting the center line based on the point cloud data of each steel section to obtain a center line space network; performing three-dimensional grid division on the center line space network, and extracting a detection viewpoint for each grid through which the center line passes;

[0013] According to the set distance threshold, all intersection points are clustered to obtain multiple intersection clusters, and the cluster center of each intersection cluster is used as a fixed inspection point for the UAV.

[0014] In some embodiments, after obtaining the inspection fixed-point positions of the drone, the shortest path planning is also performed based on all the inspection fixed-point positions of the drone.

[0015] In some embodiments, within the inspection range of the wall-climbing robot, planning the inspection path of the wall-climbing robot includes:

[0016] The point cloud data of the steel structure within the inspection range of the wall-climbing robot is clustered to obtain the point cloud data of multiple steel sections, and the center line is extracted according to the point cloud data of each steel section to obtain the center line space network; the center line space network is divided into three-dimensional grids, and a detection viewpoint is extracted for each grid through which the center line passes;

[0017] For each detection viewpoint, determine a plurality of steel segments that can be connected to the detection viewpoint without obstacles, and obtain a set of steel segments that can be viewed at the detection viewpoint;

[0018] Analyze the set of steel segments corresponding to each detection viewpoint, and take the minimum number of steel segments as the goal to obtain the steel segment corresponding to each viewpoint;

[0019] These steel segments are recorded as the steel segments to be crawled, and the intersection between the detection viewpoint and the corresponding steel segment is recorded as the inspection fixed point position of the wall-climbing robot.

[0020] In some embodiments, after obtaining the steel section to be crawled and the inspection fixed point position, the shortest path planning is also performed.

[0021] In some embodiments, the method further includes: judging whether a single device is capable of completing the inspection task based on the performance parameters of the drone and the wall-climbing robot; if not, further calculating the required number of drones and the required number of wall-climbing robots, and allocating the inspection path.

[0022] In some embodiments, the method for determining the number of drones is:

[0023] The surface of the steel structure within the inspection range of the drone is expanded by a set distance to obtain the flight surface of the drone inspection, wherein the set distance is the distance between the drone and the steel structure during the inspection;

[0024] The area of ​​the UAV flight surface is taken as the total area, and combined with the field of view area that can be covered by a single flight of a given UAV, the required number of UAVs is obtained.

[0025] In some embodiments, the method for determining the number of wall-climbing robots is: determining the required number of wall-climbing robots according to the planned path length of the wall-climbing robots and the endurance of a single wall-climbing robot.

[0026] One or more embodiments provide a large-scale steel structure air-ground collaborative automatic inspection system, including a drone, a wall-climbing robot, and a server, wherein the server is configured as follows:

[0027] Execute the large-scale steel structure automatic inspection path planning method to obtain the paths of the UAV and the wall-climbing robot, and send them to the UAV and the wall-climbing robot respectively, and control the UAV and the wall-climbing robot to perform the inspection tasks in parallel according to the corresponding paths;

[0028] When suspected damage location information is received from a drone, the line of sight direction is calculated based on the point cloud area where the suspected damage is located, and the position that the wall-climbing robot needs to reach is determined based on the line of sight direction; based on the position to be reached, the nearest wall-climbing robot is found, and a path is planned for the wall-climbing robot from the current position to the target position, and the planned path is sent to the wall-climbing robot; after the suspected damage detection is completed, the original path is returned to continue the inspection work.

[0029] In some embodiments, when a drone / wall-climbing robot detects suspected damage, the point cloud area where the suspected damage point is located is determined based on the current position of the drone / wall-climbing robot, the relative distance to the suspected damage, and the current shooting orientation of the drone / wall-climbing robot.

[0030] In one or more of the above technical solutions, for large steel structure buildings, the inspection range of drones and wall-climbing robots is first divided, and then inspection paths are planned within their respective inspection ranges, and then the drones and wall-climbing robots are controlled to perform defect detection according to the planned paths, thereby achieving high-coverage, high-efficiency, and high-precision unmanned inspection of large steel structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0032] Figure 1 This is an application scenario diagram of the path planning and inspection system in an embodiment of the present invention;

[0033] Figure 2 This is a flow chart of the UAV inspection path planning in an embodiment of the present invention;

[0034] Figure 3 This is a flow chart of the inspection path planning of the wall-climbing robot in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0036] In the description of the embodiments of the present application, the term “including” and similar terms should be understood as open inclusion, that is, “including but not limited to.” The term “based on” should be understood as “based at least in part on.”

[0037] As described in the background technology, large steel structures have the characteristics of large height, wide range, numerous rods, and complex connections, which makes manual inspection time-consuming and laborious, and there are many blind spots in the work. There is still a lack of an automatic inspection system with strong applicability. In order to realize unmanned comprehensive inspection of large steel structures, this application introduces a wall-climbing robot, and provides a comprehensive, efficient and intelligent inspection system and method for large steel structures based on collaborative inspection of multiple unmanned equipment, and carries out comprehensive inspection of large steel structures based on two unmanned equipment with complementary functions, namely, unmanned aerial vehicles and wall-climbing robots. For large steel structure buildings, the inspection ranges of unmanned aerial vehicles and wall-climbing robots are first divided, and then the inspection paths are planned within their respective inspection ranges, and then the unmanned aerial vehicles and wall-climbing robots are controlled to perform defect detection according to the planned paths. In addition, in addition to detecting defects within its own inspection area, the wall-climbing robot also serves as a supplement to the defect detection of unmanned aerial vehicles, and performs close inspections on suspected rust damage points scanned by unmanned aerial vehicles, thereby realizing high coverage, high efficiency, and high precision unmanned inspection of large steel structures.

[0038] Figure 1The following are examples of application scenarios of this application. The drone can be an existing drone device, and the wall-climbing robot can be an existing wall-climbing robot for steel structure buildings, as long as it is equipped with a visual system and can be installed with a defect detection algorithm for steel structure buildings, without specific limitation.

[0039] As an example, for the distribution characteristics of the steel structures to be inspected at the inspection site of large steel structures, the problem of false detection and missed detection of rust damage points may be caused by the lack of collection angles within the collection range. Based on this, one or more embodiments provide a gimbal with a viewing angle adjustment function. The gimbal carries a multi-modal information collection module and an electric control module. Through the controlled movement of multiple motors, the basic viewing angle adjustment function of the heavy-duty gimbal in the horizontal direction of 360 degrees and the vertical direction of 90 degrees is completed, and a larger range can be covered through the viewing angle adjustment. In view of the problem that the spatial motion position information of the unmanned aerial vehicle is unknown, the GNSS positioning module and the IMU positioning module are introduced to obtain the position information of the unmanned equipment in real time, so as to realize the precise positioning of the unmanned equipment in the three-dimensional air construction model; the multi-modal information collection module includes an image processing module and a laser radar sensor. The image processing module is used to collect images of the steel structure and then perform defect analysis. The laser radar sensor is used to detect the relative distance between the steel structure segment in real time. When a defect is detected, the steel segment where the defect is located is determined according to the relative distance and the shooting angle.

[0040] As an example, a wall-climbing robot used in corrosion detection of large steel structures is equipped with an image acquisition module, an ultrasonic thickness gauge, and an environmental humidity sensor. Among them, the image acquisition module is used to obtain visible light images of corrosion damage on the surface of high-quality steel structures. Since the causes of corrosion damage to large steel structures are mostly water seepage and leakage, which can cause abnormally high environmental humidity, an environmental humidity sensor is used to quickly and sensitively detect the humidity data on the surface of the steel structure at close range. The suspected leakage points are identified and corresponding warnings are issued through the collected environmental humidity data, and the identification of corrosion damage is assisted at the same time; in view of the difference in steel material thickness caused by different degrees of corrosion damage to large steel structures, an ultrasonic thickness gauge is introduced to detect the thickness of the corrosion damage site, focusing on breakthroughs in lightweight intelligent ultrasonic thickness measurement technology, and realizing important data collection and intelligent judgment of the severity of corrosion damage to large steel structures.

[0041] One or more embodiments of the present invention provide a large-scale steel structure air-ground collaborative automatic inspection method, comprising the following steps:

[0042] Step 1: Obtain the 3D point cloud data of the large steel structure. According to the given flight altitude and field of view of the drone, determine the point cloud set that can fall within the field of view of the drone, and obtain the inspection range of the drone. The point cloud set outside the inspection range of the drone is the inspection range of the wall-climbing robot.

[0043] Step 2: Plan the inspection paths of the UAV and the wall-climbing robot within the inspection range of the UAV and the wall-climbing robot respectively.

[0044] In step 1, firstly, a spatial coordinate system is constructed with the set position as the origin to obtain coordinate information of the point cloud data; as an example, the ground center point of the steel structure building can be used as the origin.

[0045] The inspection range of the drone is mainly analyzed based on the drone's detection accuracy, monitoring range and other performance parameters. The flight altitude of the drone is the maximum distance between the drone and the steel structure, which is related to the resolution of the visual system on board. Within the maximum flight altitude of the drone, the altitude at which a defect of a given size can be clearly photographed shall prevail. The field of view is related to the field of view of the visual system and the degree of freedom of the gimbal. When the parameters of the drone, its gimbal and the visual system are determined, the flight altitude and field of view can be obtained by calculation.

[0046] The field of view of different cameras determines the shape and size of their detection area. The field of view of a camera is usually expressed in horizontal and vertical viewing angles. Based on the field of view and working distance, the field of view of the camera can be modeled as a three-dimensional volume. Specifically, it can be modeled in the following way: Assume that the distance between the drone and the structure is , the horizontal viewing angle is , the vertical viewing angle is , the camera is at a distance The horizontal and vertical coverage widths at are:

[0047] ,

[0048] ,

[0049] The above model is combined with the real-time location information of the drone to calculate its coverage in space, match it with the three-dimensional point cloud of the steel structure surface, and mark the points covered in the drone's field of view. After the inspection task is completed, the marked point cloud set can be represented as the covered area, while the unmarked point cloud set represents the inspection blind area.

[0050] In step 2, the path planning for the UAV and the wall-climbing robot includes two levels: planning the operation route and planning the shooting angle. Planning the operation route ensures comprehensive coverage of the inspection, and the planning of the shooting angle calculates the inspection positions where the UAV performs fixed-point shooting, thereby taking into account both the comprehensiveness of the inspection and work efficiency.

[0051] like Figure 2As shown in the figure, the inspection path planning for the drone within the inspection range of the drone specifically includes:

[0052] (1) Based on the point cloud data within the inspection range of the drone, the outer layer data is extracted and expanded by a set distance to obtain the drone inspection flight surface. The set distance is the distance between the drone and the steel structure during the inspection. It can be understood that when the ground center point of the steel structure building is taken as the origin, the point farthest from the coordinate origin in the same ray direction starting from the coordinate origin is recorded as the outer layer data.

[0053] (2) Clustering the point cloud data of the steel structure within the inspection range of the UAV to obtain point cloud data of multiple steel sections, extracting the center line based on the point cloud data of each steel section to obtain a center line space network; dividing the center line space network into a three-dimensional grid (cube grid), and extracting a detection viewpoint for each grid through which the center line passes;

[0054] (3) For each detection viewpoint, calculate the intersection point where the distance between the detection viewpoint and the inspection flight surface of the UAV is the shortest, and record the direction from the detection viewpoint to the intersection point as the line of sight direction;

[0055] (4) According to the set distance threshold, all intersection points are clustered to obtain multiple intersection clusters. The cluster center of each intersection cluster is used as a fixed inspection point for the drone. The drone takes pictures of the corresponding viewpoint at the fixed inspection point.

[0056] (5) Perform shortest path planning based on all inspection locations of the drone.

[0057] By extracting the center line within the inspection range and dividing it into three-dimensional grids, a viewpoint is extracted in each three-dimensional grid. This ensures that the field of view of the subsequent drone can cover all steel structure sections within the inspection range during inspection, avoiding missed views or defects.

[0058] After obtaining all the inspection fixed-point positions of the drone, it is necessary to organize these inspection fixed-point positions into a coherent path through path planning. As an example, the ant colony algorithm can be used for path planning.

[0059] Since the wall-climbing robot works by attaching to the surface of the steel structure, its detection field of view is limited. Therefore, in order to take into account both work efficiency and energy saving, it is necessary to make the crawling path as short as possible while ensuring that the field of view of the wall-climbing robot can cover the entire inspection range.

[0060] Based on this, Figure 3 As shown in the figure, within the inspection range of the wall-climbing robot, the inspection path planning of the wall-climbing robot specifically includes:

[0061] (1) Clustering the point cloud data of the steel structure within the inspection range of the wall-climbing robot to obtain point cloud data of multiple steel segments, extracting the center line based on the point cloud data of each steel segment to obtain a center line space network; dividing the center line space network into a three-dimensional grid (cube grid), and extracting a detection viewpoint for each grid through which the center line passes;

[0062] (2) For each detection viewpoint, determine the multiple steel segments that can be connected to the detection viewpoint without obstacles. That is, for each detection viewpoint, a set of steel segments that can be visible to it can be obtained. These steel segment sets are analyzed, and the steel segment with the most visible viewpoints is preferentially selected. The steel segment with the least steel segments is taken as the goal, and the steel segment corresponding to each viewpoint is obtained;

[0063] (3) These steel segments are recorded as the steel segments to be climbed, and the intersection between the detection viewpoint and the corresponding steel segment is recorded as the inspection fixed point position of the wall-climbing robot, which is used to shoot the viewpoint on the corresponding line of sight.

[0064] (4) Based on the steel sections to be crawled and the fixed inspection positions on each steel section, the shortest path planning is performed. It can be understood that the shortest path is a coherent path covering all the steel sections to be crawled and the fixed inspection positions on each steel section.

[0065] Since it is necessary to ensure that the wall-climbing robot passes through all the sections to be crawled and is allowed to pass through the steel sections repeatedly, the above path planning problem can be equivalent to an arc routing problem, that is, it is required to find a shortest path so that each steel section to be crawled in the steel structure is passed at least once. As an example, the following method can be used: an undirected road network graph is established based on all the steel sections to be crawled, the endpoints of each steel section are nodes of the graph, the steel section is the edge connecting two nodes, the length of each steel section determines the weight of the edge, and the solution for solving problems under existing large-scale road networks is used for path planning.

[0066] The path is solved based on the above method, and all fixed inspection positions in each steel section are smoothly connected to generate the final path.

[0067] Based on this, the inspection range division and inspection route planning of drones and wall-climbing robots are realized. For large steel structures, a single drone or a single wall-climbing robot cannot meet the daily inspection needs due to its limitations such as endurance. It may require multiple devices to cooperate to achieve this.

[0068] Based on this, for large steel structures, it also includes:

[0069] Step 3: Based on the performance parameters of the drone and wall-climbing robot, determine whether a single device can complete the inspection task. If not, further calculate the required number of drones and wall-climbing robots, and allocate the inspection path.

[0070] Specifically, the overall area and height of the steel structure are the primary factors affecting the number of inspections. Larger and taller structures require more drones to cover the entire area. The complexity of the structure, such as the density of beams and columns and the number of corners, will increase the difficulty of inspection. Therefore, complex structures often require more drones to reduce blind spots and increase repeated coverage to ensure comprehensive inspections.

[0071] The method for determining the number of drones is:

[0072] The area of ​​the drone flight surface obtained above is taken as the total area, combined with the field of view area that can be covered by a single flight of a given drone, to obtain the required number of drones.

[0073] Assume that the single flight coverage area of ​​the drone is , the total area is , then the minimum number of drones required can be approximately calculated by the following formula:

[0074]

[0075] in is the overlap coefficient between drones to ensure seamless connection between areas. It is the difficulty coefficient of steel structure inspection.

[0076] The method for allocating the inspection path of the drone includes: segmenting the inspection path of the drone according to the required number of drones, obtaining the inspection path of each drone, and setting the location of the nest of each drone. Reasonable deployment of drone nests helps to improve inspection efficiency and reduce flight distance and energy consumption. The location of the nest is selected at the edge of the structure and the closest to its inspection path.

[0077] The number of wall-climbing robots required for inspection is determined based on the size of the blind area of ​​the steel structure inspection, the spatial range, and the number of suspected rust spots. The height of the blind area, the distance between structures, and the spatial complexity determine the number of robots required for each area. The method for determining the number of wall-climbing robots is as follows:

[0078] The number of wall-climbing robots required is determined based on the path length of the wall-climbing robot planned in step 2 and the endurance of a single wall-climbing robot. Based on this, the wall-climbing robot can stay longer at points with complex structures and many points to be observed, so as to inspect these areas in detail.

[0079] Assume the path length is , and the density of suspected rust spots is , then the initial number of wall-climbing robots required is It can be estimated by the following formula:

[0080]

[0081] in, represents the robot's climbing speed, Indicates the robot's battery life. Represents the spatial complexity of the blind area of ​​the steel structure.

[0082] The method for allocating inspection paths of wall-climbing robots includes: segmenting the inspection paths of the wall-climbing robots according to the number of wall-climbing robots required to obtain the inspection paths of each wall-climbing robot;

[0083] On the basis of realizing the above path planning, one or more embodiments of the present invention further provide a large-scale steel structure air-ground collaborative automatic inspection system, including a drone, a wall-climbing robot and a server:

[0084] The server is configured to plan the inspection range and inspection path of the UAV and the wall-climbing robot, and send the inspection path to the UAV and the wall-climbing robot respectively;

[0085] The drone is configured to inspect the large steel structure within the drone inspection range according to the set inspection path, identify and locate suspected damage in the large steel structure, and feedback to the server;

[0086] The wall-climbing robot is configured to inspect large steel structures within the inspection range of the wall-climbing robot according to a set inspection path, identify and locate suspected damage in the large steel structure, and feed back to the server.

[0087] When faced with the parallel tasks of blind spot inspection by drones and suspected rust point detection, the principle of giving priority to the suspected rust point detection is adopted. According to the location of the suspected rust point detected by the drone, the nearest wall-climbing robot is dispatched to approach for inspection. After the suspected rust point detection is completed, the wall-climbing robot will continue to detect the inspection blind spot. Specifically, when the server receives the suspected damage location information sent by the drone, it calculates the line of sight direction according to the center coordinates of the suspected damage, and determines the position that the wall-climbing robot needs to reach according to the line of sight direction; according to the position to be reached, the nearest wall-climbing robot is found, and the path between the current position and the target position of the wall-climbing robot is planned, and the planned path is sent to the wall-climbing robot; after the suspected damage detection is completed, return to the previous path to continue the inspection work.

[0088] Drones can be used to locate suspected damage in large steel structures including:

[0089] After the suspected damage is identified, the three-dimensional point cloud area where the suspected damage point is located is determined according to the coordinate position of the drone, the relative distance to the suspected damage and the current shooting orientation of the drone.

[0090] The method for real-time acquisition of the coordinate position of the UAV includes: using a fusion positioning technology of a global navigation satellite system (GNSS) and an inertial measurement unit (IMU) to acquire the position of the UAV in real time.

[0091] The wall-climbing robot locates suspected damage in large steel structures including:

[0092] After the suspected damage is identified, the steel section where the suspected damage point is located is determined according to the coordinate position of the wall-climbing robot, the relative distance to the suspected damage and the current shooting orientation of the drone.

[0093] The method for real-time acquisition of the coordinate position of the wall-climbing robot includes: using a fusion positioning technology of a global navigation satellite system (GNSS) and an inertial measurement unit (IMU) to acquire the position of the wall-climbing robot in real time.

[0094] In one or more of the above embodiments, by combining the different maneuverability and detection accuracy of different heterogeneous unmanned equipment, a multi-unmanned equipment air-to-ground collaborative inspection method for large steel structures is obtained, including an inspection task allocation method for drones and wall-climbing robots, and an inspection path planning method within their respective inspection ranges, thereby achieving full coverage, periodic, and efficient inspection of surface damage such as corrosion of large steel structures, thereby accurately locating the target location of the damage.

[0095] At the same time, through the parallel collaborative inspections of drones and wall-climbing robots, timely discovery, timely reporting and timely repairs can be achieved, so as to discover and eliminate safety hazards of large steel structures as quickly as possible.

[0096] Although adopting specific order to describe each operation, this should be understood as requiring such operation to be performed in the specific order shown or in sequential order, or requiring all illustrated operations to be performed to obtain desired results. Under certain environment, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the application. Some features described in the context of separate embodiments can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in the mode of any suitable sub-combination.

[0097] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the scope of protection of the present application is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are only exemplary forms.

Claims

1. A large-scale steel structure air-ground collaborative automatic inspection path planning method, characterized in that: The following steps are involved: Obtain the 3D point cloud data of large steel structures, and determine the point cloud set that can fall within the field of view of the drone according to the maximum flight altitude and maximum field of view of the drone, and obtain the inspection range of the drone; the point cloud set outside the inspection range of the drone is the inspection range of the wall-climbing robot; Plan the inspection paths of the drone and the wall-climbing robot within the inspection range of the drone and the wall-climbing robot respectively; Planning the inspection path for the drone within the inspection range includes: According to the point cloud data within the inspection range of the UAV, the outer layer data is extracted and expanded by a set distance to obtain the UAV inspection flight surface by fitting; Clustering the point cloud data of steel structures within the inspection range of the drone to obtain point cloud data of multiple steel sections, extracting the center line based on the point cloud data of each steel section to obtain a center line space network; performing three-dimensional grid division on the center line space network, and extracting a detection viewpoint for each grid through which the center line passes; For each detection viewpoint, the intersection point with the inspection flight surface of the UAV is calculated when the distance between the viewpoint and the UAV is the shortest. All the intersection points are clustered according to the set distance threshold to obtain multiple intersection clusters. The cluster center of each intersection cluster is used as a fixed inspection point of the UAV. Within the inspection range of the wall-climbing robot, the inspection path planning of the wall-climbing robot includes: The point cloud data of the steel structure within the inspection range of the wall-climbing robot is clustered to obtain the point cloud data of multiple steel sections, and the center line is extracted according to the point cloud data of each steel section to obtain the center line space network; the center line space network is divided into three-dimensional grids, and a detection viewpoint is extracted for each grid through which the center line passes; For each detection viewpoint, determine a plurality of steel segments that can be connected to the detection viewpoint without obstacles, and obtain a set of steel segments that can be viewed at the detection viewpoint; Analyze the set of steel segments corresponding to each detection viewpoint, and take the minimum number of steel segments as the goal to obtain the steel segment corresponding to each viewpoint; These steel segments are recorded as the steel segments to be crawled, and the intersection between the detection viewpoint and the corresponding steel segment is recorded as the inspection fixed point position of the wall-climbing robot.

2. The large-scale steel structure air-ground collaborative automatic inspection path planning method according to claim 1 is characterized in that: After obtaining the inspection point positions of the drone, the shortest path planning is also performed based on all the inspection point positions of the drone.

3. The large-scale steel structure air-ground collaborative automatic inspection path planning method according to claim 1 is characterized in that: After obtaining the steel section to be crawled and the fixed inspection point position, the shortest path planning is also carried out.

4. The large-scale steel structure air-ground collaborative automatic inspection path planning method according to claim 1 is characterized in that: The method also includes: judging whether a single device can complete the inspection task according to the performance parameters of the drone and the wall-climbing robot, and if not, further calculating the required number of drones and the required number of wall-climbing robots, and allocating the inspection path.

5. The large-scale steel structure air-ground collaborative automatic inspection path planning method as claimed in claim 4 is characterized in that: The method for determining the number of drones is: The surface of the steel structure within the inspection range of the drone is expanded by a set distance to obtain the flight surface of the drone inspection, wherein the set distance is the distance between the drone and the steel structure during the inspection; The area of ​​the UAV flight surface is taken as the total area, and combined with the field of view area that can be covered by a single flight of a given UAV, the required number of UAVs is obtained.

6. The large-scale steel structure air-ground collaborative automatic inspection path planning method as claimed in claim 4 is characterized in that: The method for determining the number of wall-climbing robots is as follows: the number of wall-climbing robots required is determined based on the planned path length of the wall-climbing robots and the endurance of a single wall-climbing robot.

7. A large-scale steel structure air-ground collaborative automatic inspection system, characterized in that: The system comprises a drone, a wall-climbing robot and a server, wherein the server is configured as follows: Execute the large-scale steel structure air-ground collaborative automatic inspection path planning method as described in any one of claims 1 to 6, obtain the paths of the UAV and the wall-climbing robot, and send them to the UAV and the wall-climbing robot respectively, and control the UAV and the wall-climbing robot to perform inspection tasks in parallel according to the corresponding paths; When suspected damage location information sent by the UAV is received, the line of sight direction is calculated according to the point cloud area where the suspected damage is located, and the position that the wall-climbing robot needs to reach is determined according to the line of sight direction; according to the position to be reached, the nearest wall-climbing robot is found, and the path from the current position to the target position of the wall-climbing robot is planned, and the planned path is sent to the wall-climbing robot; after the suspected defect detection is completed, return to the original path to continue the inspection work.

8. The large-scale steel structure air-ground collaborative automatic inspection system according to claim 7 is characterized in that: When the UAV / wall-climbing robot detects suspected damage, the point cloud area where the suspected damage point is located is determined based on the current position of the UAV / wall-climbing robot, the relative distance to the suspected damage, and the current shooting orientation of the UAV / wall-climbing robot.

Images