Unmanned aerial vehicle global continuous intelligent inspection method and system for bridge support under signal limited space condition

Abstract

The invention provides an unmanned aerial vehicle global continuous intelligent inspection method and system for a bridge support under a signal limited space condition, and the method is characterized in that the method comprises the following steps: S1, constructing a global prior map; s2, preparing an unmanned aerial vehicle; s3, task planning; s4, intelligent inspection: based on the preset inspection parameters, utilizing the unmanned aerial vehicle to autonomously fly and hover at an inspection waypoint to collect a bridge support image, performing real-time positioning and mapping when the unmanned aerial vehicle autonomously flies, and executing dynamic obstacle avoidance, and / or guided flight in a signal limited space, and / or inspection optimization; s5, data processing and defect identification; and S6, generating an inspection report. According to the invention, global continuous intelligent routing inspection can be carried out on the bridge support under the condition of a signal limited space, the precision is high, the reliability is good, and routing inspection optimization can be carried out, so that the endurance time of the unmanned aerial vehicle is prolonged and the image acquisition quality is improved under the condition that the global continuous routing inspection operation of the unmanned aerial vehicle is not influenced.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of bridge engineering structure health monitoring and unmanned aerial vehicle application, and particularly relates to a method and system for realizing full autonomous and high-precision global continuous intelligent inspection of bridge bearings by an unmanned aerial vehicle in a complex restricted space with missing or weak navigation signals. BACKGROUND

[0002] Bridge bearings are key load transfer components connecting the upper structure and the lower structure of a bridge, and their health conditions are directly related to the safety of the whole bridge. The traditional manual inspection method has the disadvantages of low efficiency, high risk, strong subjectivity and difficulty in comprehensive coverage.

[0003] At present, the use of unmanned aerial vehicles for bridge inspection has become an industry trend. However, the existing technology is mainly applied to open spaces such as the bottom of the beam and the surface of the pier, and relies on GPS, Beidou and other positioning and navigation. When the inspection target is the bearing located under the pier cap beam or the bridge span connection, the unmanned aerial vehicle will enter a typical "signal restricted space", which is characterized by: the navigation signal is completely shielded or extremely weak, causing the unmanned aerial vehicle to be unable to rely on satellite positioning. The space structure is complex and narrow, with a large number of obstacles (such as embedded parts and maintenance channels). There is electromagnetic interference that may affect remote control and image transmission signals. In particular, when the unmanned aerial vehicle inspects the bridge bearings of a multi-layer, through-type overpass bridge structure or when the bridge is near a high-rise building, the problem of "signal restricted space" for the unmanned aerial vehicle is more obvious.

[0004] Therefore, the existing unmanned aerial vehicle inspection technology cannot work safely and effectively in such an environment, and there is an urgent need for an intelligent inspection solution that is not completely dependent on external signals and has high autonomy. SUMMARY

[0005] The present application aims to overcome the shortcomings of the prior art and provide a global continuous intelligent inspection method for unmanned aerial vehicles to autonomously position, avoid obstacles, collect images and intelligently identify defects of bridge bearings in a signal restricted space.

[0006] The present application is implemented through the following technical solutions.

[0007] In a first aspect, the present application provides a global continuous intelligent inspection method for bridge bearings by an unmanned aerial vehicle in a signal restricted space, characterized in that it comprises the following steps: S1, constructing a global prior map A three-dimensional point cloud model of the bridge bearing inspection area is constructed as a global prior map; S2, unmanned aerial vehicle preparation The global prior map is loaded into the unmanned aerial vehicle and a fusion sensor is carried on board; S3, task planning based on the global prior map, preset the inspection parameters according to the type, number and position of the bridge support; the inspection parameters include inspection path, inspection waypoint, gimbal angle and shooting parameter; S4, intelligent inspection based on the preset inspection parameters, the unmanned aerial vehicle autonomously flies and hovers at the inspection waypoint to collect images of the bridge support, while performing real-time positioning and mapping, dynamic obstacle avoidance, guided flight in signal-limited space, and / or inspection optimization when the unmanned aerial vehicle autonomously flies; S5, data processing and defect identification S6, generating an inspection report.

[0008] Preferably, in step S2, the fusion sensor includes a laser radar, a vision sensor and an inertial measurement unit.

[0009] Preferably, the real-time positioning and mapping specifically includes the following steps: when the unmanned aerial vehicle flies along the inspection path, the fusion sensor is used to collect fusion data; based on the fusion data, SLAM is used to obtain real-time positioning information and real-time attitude information of the unmanned aerial vehicle, and a local obstacle map around the unmanned aerial vehicle is constructed; the local obstacle map is matched with the global prior map, and the real-time positioning information is corrected through the matching result to correct the cumulative error of SLAM.

[0010] Preferably, the guided flight in signal-limited space specifically includes the following steps: when the unmanned aerial vehicle flies into the signal-limited space, the preset inspection path is compared with the corrected real-time positioning information to generate a control instruction to guide the unmanned aerial vehicle to fly in the signal-limited space.

[0011] Preferably, the dynamic obstacle avoidance specifically includes the following steps: the local obstacle map is compared with the loaded global prior map to detect dynamic obstacles in the local obstacle map that are not recorded in the global prior map, and real-time dynamic obstacle avoidance is triggered to make the unmanned aerial vehicle bypass the dynamic obstacles and then return to the preset inspection path; the dynamic obstacles are updated to the loaded global prior map, and the preset inspection path is corrected and updated based on the bypass path.

[0012] Preferably, the inspection optimization specifically includes the following steps: when the unmanned aerial vehicle flies into the signal-limited space, the starting position and the ending position of the signal-limited space are recorded; dividing a corresponding signal-restricted area in the global prior map based on the start position and the end position; judging whether there is an inspection waypoint in the signal-restricted area, when there is an inspection waypoint, analyzing and calculating an equivalent waypoint area of each inspection waypoint, comparing the equivalent waypoint area with the signal-restricted area, if the equivalent waypoint area does not all fall into the signal-restricted area, moving the inspection waypoint to an optimized inspection waypoint which is not located in the signal-restricted area but in the equivalent waypoint area, if the equivalent waypoint area all falls into the signal-restricted area, not moving the inspection waypoint, when there is no inspection waypoint, adjusting an inspection path falling into the signal-restricted area to outside the signal-restricted area.

[0013] Preferably, The step S2 of preparing the unmanned aerial vehicle further comprises: configuring a lightweight AI image analysis model in the unmanned aerial vehicle system. The step S4 of intelligent inspection further comprises: using the lightweight AI image analysis model to identify whether the bridge support image is qualified, if not, controlling the pan-tilt camera to re-shoot at the original pan-tilt angle or / and to additionally re-shoot at the pan-tilt angle until a qualified bridge support image is obtained.

[0014] Preferably, the step S5 of data processing and defect identification specifically comprises: stitching the bridge support image to generate a panoramic image or a three-dimensional model of the bridge support; using a deep learning model to analyze the panoramic image or the three-dimensional model to identify and quantify the defect type and grade of the support.

[0015] Preferably, the defect type comprises: cracking, aging, bulging, uneven compression and slipping of rubber support; rusting, weld cracking, pin falling and displacement exceeding limit of steel support.

[0016] In a second aspect, the present application provides an unmanned aerial vehicle global continuous intelligent inspection system for bridge support under signal-restricted space condition, comprising an unmanned aerial vehicle and a ground station, characterized in that further comprising: a modeling module configured in the ground station, a fusion sensor carried on the unmanned aerial vehicle, a fusion positioning module, an inspection optimization module and an image processing module configured in the unmanned aerial vehicle system. The modeling module is used to construct a global prior map. The fusion sensor is used to collect fusion data of the environment around the unmanned aerial vehicle. The fusion positioning module comprises: The fusion number is used to acquire real-time positioning information of the unmanned aerial vehicle by SLAM and construct a local obstacle map around the unmanned aerial vehicle, the local obstacle map is matched with the global prior map, and the real-time positioning information is corrected by correcting errors accumulated by SLAM according to the matching result; When the unmanned aerial vehicle flies into a signal-limited space, the preset inspection path is compared with the corrected real-time positioning information, a control instruction is generated, and the unmanned aerial vehicle is guided to fly in the signal-limited space. The local obstacle map is compared with the loaded global prior map, a dynamic obstacle not recorded in the global prior map is detected in the local obstacle map, real-time dynamic obstacle avoidance is triggered, the unmanned aerial vehicle returns to the preset inspection path after bypassing the dynamic obstacle, the dynamic obstacle is updated into the loaded global prior map, and the preset inspection path is corrected and updated based on the bypass path. The inspection optimization module is used to record a starting position and an ending position of a signal-limited space when the unmanned aerial vehicle flies into the signal-limited space, a corresponding signal-limited area is divided in the global prior map based on the starting position and the ending position, it is judged whether there is an inspection waypoint in the signal-limited area, when there is an inspection waypoint, an equivalent waypoint area of each inspection waypoint is calculated and analyzed, the equivalent waypoint area is compared with the signal-limited area, if the equivalent waypoint area does not fall into the signal-limited area, the inspection waypoint is moved to an optimized inspection waypoint which is not located in the signal-limited area but located in the equivalent waypoint area, if the equivalent waypoint area falls into the signal-limited area, the inspection waypoint is not moved, and when there is no inspection waypoint, an inspection path falling into the signal-limited area is adjusted to be outside the signal-limited area. The image processing module is used to identify whether the bridge support image is qualified by using a preconfigured lightweight AI image analysis model, if not, the gimbal camera is controlled to be rephotographed at the original gimbal angle or / and additional gimbal angle rephotography is performed until a qualified bridge support image is obtained. The data processing module comprises: The bridge support image is spliced to generate a panoramic image or a three-dimensional model of the bridge support. The panoramic image or the three-dimensional model is analyzed by using a pre-established deep learning model to identify and quantify the defect type and grade of the support, and then an inspection report is generated.

[0017] Compared with the prior art, the present application has the following advantages: 1) Global continuous flight operation: completely get rid of the dependence on GPS and remote control signals, and realize global continuous flight operation in an extreme signal-limited environment.

[0018] 2) High precision and high reliability: Through the fusion positioning scheme of "prior model + real-time SLAM", the cumulative error of pure SLAM is effectively overcome, and the positioning accuracy can reach the centimeter level, far exceeding that of pure inertial navigation.

[0019] 3) High safety: It has the ability to build local maps in real time and dynamically avoid obstacles, which can deal with unknown obstacles and greatly improve the safety of working in a limited space.

[0020] 4) Intelligent Optimized Inspection: During each inspection operation, the inspection can be optimized to minimize the number of times the UAV crosses signal-restricted spaces and reduce the number of inspection waypoints set up in signal-restricted spaces. This improves the UAV's endurance and image acquisition quality without affecting its continuous inspection operations across the entire domain. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention, but not all embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the inspection path in the embodiment; Figure 3 This is a schematic diagram showing the flight altitude and position of the drone; Figure 4 This is a schematic diagram illustrating the dynamic obstacle avoidance, guided flight, and inspection optimization in a signal-constrained space when using the method of this invention to inspect the bridge bearings of a target bridge. Figure 5 for Figure 4 Enlarged view of point B in the middle; Figure 6 for Figure 4 Enlarged view of point C in the middle; Figure 7 for Figure 4 Enlarged view of point D in the middle; Figure 8 for Figure 4 Enlarged view of point E in the middle; Figure 9 A schematic diagram of the inspection path and inspection waypoints optimized for dynamic obstacle avoidance and inspection. Figure 10 This is a structural block diagram of the system of the present invention; The meanings of the labels in the above diagram are as follows: 1-existing bridge, 2-target bridge, 3-existing building, 4-bridge support, 5-signal restricted area, 6-equivalent waypoint area, 7-bridge pier, 8-UAV, 9-dynamic obstacle, 10-inspection path. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0023] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0024] As illustrated herein, unless the context clearly indicates otherwise, the words “a,” “an,” “an,” and / or “the” do not specifically refer to the singular and may also include the plural. Generally speaking, the terms “comprising” and “including” only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0025] The definitions used herein, such as the terms “having,” “may have,” “comprising,” or “may include,” indicate the presence of the corresponding function, operation, element, etc., and do not limit the presence of one or more other functions, operations, elements, etc. Furthermore, it should be understood that the terms “comprising” or “having” as used herein indicate the presence of the features, figures, steps, operations, elements, components, or combinations thereof described in the specification, without excluding the presence or addition of one or more other features, figures, steps, operations, elements, components, or combinations thereof.

[0026] The definitions of "first" and "second" in this document, and the descriptions of "first" and "second" appearing in this document, are for illustrative purposes and to distinguish the objects being described. They do not indicate any order or limit on the number of devices, and do not constitute any limitation on this document. For example, a first element may be referred to as a second element without departing from the scope of this disclosure, and similarly, a second element may be referred to as a first element.

[0027] The definition of connection in this document will be understood to mean that when an element (e.g., a first element) is “connected to” or “operationally or communicatively coupled to” another element (e.g., a second element), the element may be directly connected or coupled to the other element, and there may be an intermediate element (e.g., a third element) between the element and the other element. Conversely, it will be understood that when an element (e.g., a first element) is “directly connected to” or “directly coupled to” another element (e.g., a second element), there is no intermediate element (e.g., a third element) between the element and the other element. Example 1

[0028] This embodiment provides a method for continuous intelligent inspection of bridge supports using unmanned aerial vehicles (UAVs) in signal-constrained spaces. Please refer to [link to relevant documentation]. Figure 1 It includes the following steps: S1. Construct a global prior map High-precision 3D point cloud models of bridge bearing inspection areas are constructed using existing bridge design BIM models, drawings, or laser scanning as global prior maps. S2, Drone Preparation The global prior map is loaded into the drone and equipped with fusion sensors, and a lightweight AI image analysis model is configured in the drone system; the fusion sensors include LiDAR, visual sensors (RGB camera) and inertial measurement unit (IMU). S3, Task Planning Based on the aforementioned global prior map, inspection parameters are preset according to the type, quantity, and orientation of the bridge supports; these inspection parameters include the inspection path, inspection waypoints, gimbal angle, and shooting parameters; for example, such as... Figure 2 As shown, in the bridge support inspection area, target bridge 2 crosses existing bridge 1 and is adjacent to existing buildings 3 on both sides. The preset inspection path 10 includes inspection waypoints A1, A2, A3, ..., A20; Figure 3 As shown, UAV 8 flies at the height of bridge support 4 below target bridge 2; as Figure 2 As shown, when the UAV hovers over inspection waypoint A1, the target position captured by the gimbal camera on the UAV is one side of support Z1 and Z2. When the UAV hovers over inspection waypoint A2, the target position captured by the gimbal camera on the UAV is the other side of support Z1 and Z2, and so on. S4, Intelligent Inspection Based on the preset inspection parameters, the UAV is used to fly autonomously and hover at the inspection waypoint to collect images of the bridge support. At the same time, real-time positioning and mapping are performed during the autonomous flight of the UAV, and dynamic obstacle avoidance, and / or guided flight in signal-constrained spaces, and / or inspection optimization are performed. in: The specific steps for acquiring bridge bearing images are as follows: When the drone autonomously flies to the inspection point, it hovers precisely and controls the gimbal camera to adjust to the preset gimbal angle. According to the preset shooting parameters, the high-resolution zoom camera automatically focuses and shoots the surface of the support, such as rubber support and steel support, to acquire high-definition images of the bridge support. The lightweight AI image analysis model is used to identify and determine whether the collected bridge bearing images are qualified, such as whether they are clear or have obvious defects. If they are not qualified, the gimbal camera is controlled to retake the picture at the original gimbal angle or / and the gimbal angle is added to retake the picture until a qualified bridge bearing image is obtained. The specific steps for real-time positioning and mapping are as follows: As the drone flies along the inspection path, it utilizes a fusion sensor—namely, LiDAR, a visual sensor (RGB camera), and an inertial measurement unit (IMU)—to collaboratively collect and fuse data about the drone's surrounding environment, resulting in fused data. Based on the fused data, SLAM (Simultaneous Localization and Mapping) technology is used to obtain the real-time positioning and attitude information of the UAV, and to build a local obstacle map of the surrounding area. The local obstacle map is matched with the global prior map using the ICP iterative nearest point algorithm. The accumulated error of SLAM is corrected by the matching result, and the real-time positioning information is corrected to ensure that the UAV always knows its precise position on the global prior map. The specific steps for guided flight in signal-confined spaces are as follows: When the drone flies into a signal-restricted space, the preset inspection path is compared with the corrected real-time positioning information to generate control commands to guide the drone to fly in the signal-restricted space. In this invention, signal limitation refers to the loss or weakness of the UAV's navigation signal, which prevents the UAV from positioning and navigating using navigation systems such as GPS and Beidou. This invention uses a combination of a global prior map and real-time SLAM fusion with multiple sensors to position and guide the UAV, enabling autonomous flight in environments with limited navigation signals, thus not affecting the UAV's continuous inspection of bridge supports across the entire area. The specific steps of the dynamic obstacle avoidance are as follows: The local obstacle map is compared with the loaded global prior map to detect dynamic obstacles in the local obstacle map that are not recorded in the global prior map, such as temporary scaffolding and set templates. The APF artificial potential field method or the DWA dynamic window method are used to trigger real-time dynamic obstacle avoidance, so that the UAV can return to the preset inspection path after bypassing the dynamic obstacle. The dynamic obstacles are updated to the loaded global prior map, and the preset inspection path is corrected and updated based on the detour path. For example, such as Figure 4 and Figure 8 As shown, after flying over inspection waypoint A18, the UAV detects dynamic obstacle 9. The UAV then detours around obstacle 9 and returns to the preset inspection path. Simultaneously, the preset inspection path is corrected and updated based on the detour path. The updated inspection path 10 is shown below. Figure 9 As shown; The specific steps for optimizing the inspection process are as follows: When the drone flies into a signal-limited space, record the starting and ending positions where the drone's navigation signal is limited; Based on the starting position and the ending position, corresponding signal-restricted areas are delineated in the global prior map; To determine if a waypoint exists within the signal-restricted area, if so, the equivalent waypoint area for each waypoint is calculated and compared with the signal-restricted area. If the equivalent waypoint area does not entirely fall within the signal-restricted area, the waypoint is moved to an optimized waypoint located outside the signal-restricted area but within the equivalent waypoint area. If the equivalent waypoint area entirely falls within the signal-restricted area, the waypoint remains stationary. If no waypoint exists, the inspection path within the signal-restricted area is adjusted to be outside the area. Optimized waypoints are preferentially placed on the inspection path. If an optimized waypoint cannot be placed on the inspection path, the inspection path between the optimized waypoint and its upstream and downstream waypoints is adjusted. Furthermore, when a waypoint is moved to an optimized waypoint, its corresponding gimbal parameters and shooting parameters should be adjusted using conventional methods. In this invention, when the UAV enters a signal-restricted space, the starting position of the navigation signal restriction is the last position recorded by the navigation system before the UAV's navigation signal is restricted. When the UAV leaves the signal-restricted space, the ending position of the navigation signal restriction is the position initially recorded by the navigation system when the UAV's navigation signal is restored. The equivalent waypoint area refers to the area where the UAV can hover under the condition that the bridge support image corresponding to the target position of the preset inspection waypoint can be acquired and the bridge support image can be clear. Typically, the equivalent waypoint area is a radial area centered on the preset inspection waypoint, such as a circular area centered on the preset inspection waypoint. To facilitate understanding of the inspection optimization process, for example, such as Figure 4 and Figure 5As shown, after the UAV crosses the inspection waypoint A2, it flies in the narrow space between the existing bridge 1 and the target bridge 2. At this time, the UAV will enter the signal-restricted space. The UAV records the starting position P1 and the ending position P1' of the navigation signal restriction, and the circular area with the line connecting the starting position P1 and the ending position P1' as the diameter is the signal-restricted area S1. There is an inspection waypoint A3 in the signal-restricted area S1. The equivalent waypoint area M3 corresponding to the inspection waypoint A3 does not fall entirely within the signal-restricted area S1. Then, the inspection waypoint A3 is moved to the optimized inspection waypoint A3' which is not located in the signal-restricted area S1 but is located in the equivalent waypoint area M3. A3' is located on the inspection path, and there is no need to adjust the inspection path between the optimized inspection waypoint A3' and its upstream and downstream inspection waypoints A2 and A4. like Figure 4 and Figure 6 As shown, when the UAV continues to fly and approaches inspection waypoint A5, there is a tall existing building 3 on one side of the target bridge 2. At this time, the UAV will enter a signal-restricted space. The UAV records the starting position P2 and the ending position P2' where the navigation signal is restricted, and the circular area drawn with the line connecting the starting position P2 and the ending position P2' as the diameter is designated as the signal-restricted area S2. Inspection waypoints A5, A6, and A7 exist within the signal-restricted area S2. Among them, inspection waypoints A6 and A7 correspond to inspection waypoints A5, A6, and A7 respectively. Since both equivalent waypoint areas M6 and M7 fall entirely within signal-restricted area S2, inspection waypoints A6 and A7 will not be moved. However, since the equivalent waypoint area M5 corresponding to inspection waypoint A5 does not fall entirely within signal-restricted area S2, inspection waypoint A5 will be moved to the optimized inspection waypoint A5', which is not located within signal-restricted area S2 but is located within equivalent waypoint area M5. Furthermore, A5' is located on the inspection path, and there is no need to adjust the inspection path between the optimized inspection waypoint A5' and its upstream and downstream inspection waypoints A4 and A6. like Figure 4 and Figure 7 As shown, when the UAV continues to fly and approaches the inspection waypoint A10, there is a tall existing building 3 on one side of the target bridge 2. At this time, the UAV will enter a signal-restricted space. The UAV records the starting position P3 and the ending position P3' where the navigation signal is restricted, and a circular area with the line connecting the starting position P3 and the ending position P3' as the diameter is designated as the signal-restricted zone S3. If there is no inspection waypoint within the signal-restricted zone S3, the inspection path that falls into the signal-restricted zone S3 will be adjusted to outside the signal-restricted zone S3. The inspection path and inspection waypoint settings after the above inspection optimization are as follows: Figure 9 As shown; Typically, bridge bearing inspections should be conducted at regular intervals. When a bridge is near a tall building or is a multi-story, through-type structure, the inspection area may contain many areas with limited navigation signals. Based on the method of this invention, when using a UAV for intelligent inspection for the first time, it will traverse many signal-limited areas when inspecting along the initially preset inspection path. However, this invention uses a global prior map combined with real-time SLAM fusion of multiple sensors to locate and guide the UAV, enabling autonomous flight in environments with limited navigation signals, thus not affecting the UAV's continuous inspection of the entire bridge bearing area. However, the real-time SLAM fusion method using multiple sensors consumes significantly more energy for data computation, transmission, and synchronization than traditional GPS navigation, thus drastically reducing the UAV's endurance. Furthermore, in areas with many dynamic objects such as pedestrians and vehicles, and in poor lighting conditions, the data acquisition accuracy of the visual sensors decreases, and noise from other onboard sensors can cause local positioning deviations in the lidar. These factors reduce the UAV's positioning accuracy in signal-limited spaces, thus affecting image acquisition quality. Furthermore, this invention can optimize the inspection process during each inspection operation, thereby minimizing the number of times the UAV crosses signal-restricted spaces during subsequent inspections and reducing the number of inspection waypoints set up in signal-restricted spaces. This improves the UAV's endurance and image acquisition quality without affecting its continuous inspection operation across the entire domain. S5, Data Processing and Defect Identification After the inspection is completed, the images of the bridge bearings collected by the UAV inspection are stitched together to generate a panoramic view or three-dimensional model of the bridge bearings. Using deep learning models, such as YOLO, the panoramic image or 3D model is analyzed to identify and quantify the defect types and levels of the bearings; the defect types include: cracking, aging, bulging, uneven compression, and slippage of rubber bearings; corrosion, weld cracking, pin detachment, and excessive displacement of steel bearings.

[0029] S6. Generate inspection report Generate a structured inspection report that includes the location, type, size, and severity level of defects. Example 2

[0030] Based on the method described in Embodiment 1, this embodiment provides an intelligent unmanned aerial vehicle (UAV) inspection system for bridge bearings under signal-constrained spatial conditions. Please refer to [link to relevant documentation]. Figure 10 This includes drones, ground workstations, modeling modules configured in the ground workstations, fusion sensors mounted on the drones, and fusion positioning modules, inspection optimization modules, and image processing modules configured in the drone system. The modeling module includes a function to construct a global prior map and load the global prior map into the UAV system. The fusion sensor is used to collect fused data of the environment surrounding the UAV; The fusion positioning module includes: The system is used to obtain real-time positioning information of the UAV using SLAM based on the fused data and construct a local obstacle map of the surrounding area. The local obstacle map is matched with the global prior map, and the accumulated error of SLAM is corrected by the matching result to correct the real-time positioning information. This is used to compare the preset inspection path with the corrected real-time positioning information when the drone flies into a signal-restricted space, generate control commands, and guide the drone to fly in the signal-restricted space. This is used to compare the local obstacle map with the loaded global prior map, detect dynamic obstacles in the local obstacle map that are not recorded in the global prior map, trigger real-time dynamic obstacle avoidance, so that the UAV can bypass the dynamic obstacle and return to the preset inspection path. At the same time, the dynamic obstacle is updated to the loaded global prior map, and the preset inspection path is corrected and updated based on the bypass path. The inspection optimization module is used to record the starting and ending positions of the UAV's navigation signal when it flies into a signal-restricted space. Based on the starting and ending positions, it divides the corresponding signal-restricted area in the global prior map, determines whether there is an inspection waypoint within the signal-restricted area, and if there is an inspection waypoint, it analyzes and calculates the equivalent waypoint area for each inspection waypoint. It compares the equivalent waypoint area with the signal-restricted area. If the equivalent waypoint area does not fall entirely within the signal-restricted area, it moves the inspection waypoint to an optimized inspection waypoint that is not located in the signal-restricted area but is located in the equivalent waypoint area. If the equivalent waypoint area falls entirely within the signal-restricted area, it does not move the inspection waypoint. If there is no inspection waypoint, the inspection path that falls within the signal-restricted area is adjusted to outside the signal-restricted area. The image processing module is used to identify and determine whether the bridge bearing image is qualified using a pre-configured lightweight AI image analysis model. If it is not qualified, the module controls the gimbal camera to retake the image at the original gimbal angle or / and add gimbal angles to retake the image until a qualified bridge bearing image is obtained. The data processing module includes: Used to stitch together the images of the bridge bearings to generate a panoramic view or a three-dimensional model of the bridge bearings; It is used to analyze panoramic or 3D models using pre-built deep learning models, identify and quantify the defect types and levels of supports, and then generate inspection reports.

[0031] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0032] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0033] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0034] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0035] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0036] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0037] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0038] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0039] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0040] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0041] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0042] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0043] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for continuous intelligent inspection of bridge bearings using unmanned aerial vehicles (UAVs) in signal-constrained spaces, characterized in that: Includes the following steps: S1. Construct a global prior map Construct a 3D point cloud model of the bridge bearing inspection area as a global prior map; S2, Drone Preparation The global prior map is loaded into the drone and equipped with fusion sensors; S3, Task Planning Based on the global prior map, inspection parameters are preset according to the type, quantity, and orientation of the bridge bearings; The inspection parameters include the inspection path, inspection waypoints, gimbal angle, and shooting parameters. S4, Intelligent Inspection Based on the preset inspection parameters, the UAV is used to fly autonomously and hover at the inspection waypoint to collect images of the bridge support. At the same time, real-time positioning and mapping are performed during the autonomous flight of the UAV, and dynamic obstacle avoidance, and / or guided flight in signal-constrained spaces, and / or inspection optimization are performed. S5. Data Processing and Defect Identification S6. Generate inspection report.

2. The method for continuous intelligent inspection of bridge supports under signal-constrained spatial conditions using unmanned aerial vehicles (UAVs) as described in claim 1, characterized in that, In step S2, the fusion sensor includes a lidar, a vision sensor, and an inertial measurement unit.

3. The method for continuous intelligent inspection of bridge supports under signal-constrained spatial conditions using unmanned aerial vehicles (UAVs) as described in claim 1, characterized in that... The specific steps for real-time positioning and mapping are as follows: When the drone flies along the inspection path, it uses fusion sensors to collect fused data; Based on the fused data, SLAM is used to obtain the real-time positioning and attitude information of the UAV, and to build a local obstacle map of the surrounding area. The local obstacle map is matched with the global prior map, and the accumulated errors of SLAM are corrected by using the matching results to modify the real-time positioning information.

4. The method for continuous intelligent inspection of bridge supports under signal-constrained spatial conditions using unmanned aerial vehicles (UAVs), as described in claim 3, is characterized in that... The specific steps for guided flight in signal-confined spaces are as follows: When the drone flies into a signal-restricted space, the preset inspection path is compared with the corrected real-time positioning information to generate control commands to guide the drone to fly in the signal-restricted space.

5. The method for continuous intelligent inspection of bridge supports under signal-constrained spatial conditions using unmanned aerial vehicles (UAVs), as described in claim 3, is characterized in that... The specific steps of the dynamic obstacle avoidance are as follows: The local obstacle map is compared with the loaded global prior map to detect dynamic obstacles in the local obstacle map that are not recorded in the global prior map, and real-time dynamic obstacle avoidance is triggered so that the UAV can bypass the dynamic obstacles and return to the preset inspection path. The dynamic obstacles are updated to the loaded global prior map, and the preset inspection path is corrected and updated based on the detour path.

6. The method for continuous intelligent inspection of bridge supports under signal-constrained spatial conditions using unmanned aerial vehicles (UAVs), as described in claim 3, is characterized in that... The specific steps for optimizing the inspection process are as follows: When the drone flies into a signal-limited space, record the starting and ending positions where the drone's navigation signal is limited; Based on the starting position and the ending position, corresponding signal-restricted areas are delineated in the global prior map; Determine whether there is a patrol waypoint within the signal-restricted area. If a patrol waypoint exists, analyze and calculate the equivalent waypoint area for each patrol waypoint. Compare the equivalent waypoint area with the signal-restricted area. If the equivalent waypoint area does not fall entirely within the signal-restricted area, move the patrol waypoint to an optimized patrol waypoint that is not located within the signal-restricted area but is located within the equivalent waypoint area. If the equivalent waypoint area falls entirely within the signal-restricted area, do not move the patrol waypoint. If no inspection waypoint exists, the inspection path that falls within the signal-restricted area will be adjusted to be outside the signal-restricted area.

7. The method for continuous intelligent inspection of bridge supports under signal-constrained spatial conditions using unmanned aerial vehicles (UAVs), as described in claim 1, is characterized in that: The drone preparation step S2 also includes: configuring a lightweight AI image analysis model in the drone system; The intelligent inspection step S4 also includes: using a lightweight AI image analysis model to identify and determine whether the bridge bearing image is qualified. If it is not qualified, the gimbal camera is controlled to retake the picture at the original gimbal angle or / and add gimbal angles to retake the picture until a qualified bridge bearing image is obtained.

8. The method for continuous intelligent inspection of bridge supports under signal-constrained spatial conditions using unmanned aerial vehicles (UAVs), as described in claim 1, is characterized in that... The specific steps of data processing and defect identification in step S5 include: The bridge bearing images are stitched together to generate a panoramic view or 3D model of the bridge bearings. Deep learning models are used to analyze panoramic or 3D images to identify and quantify the types and levels of defects in the supports.

9. The method for continuous intelligent inspection of bridge supports under signal-constrained spatial conditions using unmanned aerial vehicles (UAVs), as described in claim 8, is characterized in that... The defect types include: Rubber bearings exhibit cracking, aging, bulging, uneven compression, and slippage. Steel supports are corroded, welds are cracked, pins are missing, and displacement exceeds limits.

10. A UAV-based continuous intelligent inspection system for bridge bearings under signal-constrained spatial conditions, comprising a UAV and a ground workstation, characterized in that, Also includes: The modeling module is configured in the ground workstation, the fusion sensor is mounted on the UAV, and the fusion positioning module, inspection optimization module and image processing module are configured in the UAV system. The modeling module is used to construct a global prior map; The fusion sensor is used to collect fused data of the environment surrounding the UAV; The fusion positioning module includes: The system is used to obtain real-time positioning information of the UAV using SLAM based on the fused data and construct a local obstacle map of the surrounding area. The local obstacle map is matched with the global prior map, and the accumulated error of SLAM is corrected by the matching result to correct the real-time positioning information. This is used to compare the preset inspection path with the corrected real-time positioning information when the drone flies into a signal-restricted space, generate control commands, and guide the drone to fly in the signal-restricted space. This is used to compare the local obstacle map with the loaded global prior map, detect dynamic obstacles in the local obstacle map that are not recorded in the global prior map, trigger real-time dynamic obstacle avoidance, so that the UAV can bypass the dynamic obstacle and return to the preset inspection path. At the same time, the dynamic obstacle is updated to the loaded global prior map, and the preset inspection path is corrected and updated based on the bypass path. The inspection optimization module is used to record the starting and ending positions of the UAV's navigation signal when it flies into a signal-restricted space. Based on the starting and ending positions, it divides the corresponding signal-restricted area in the global prior map, determines whether there is an inspection waypoint within the signal-restricted area, and if there is an inspection waypoint, it analyzes and calculates the equivalent waypoint area for each inspection waypoint. It compares the equivalent waypoint area with the signal-restricted area. If the equivalent waypoint area does not fall entirely within the signal-restricted area, it moves the inspection waypoint to an optimized inspection waypoint that is not located in the signal-restricted area but is located in the equivalent waypoint area. If the equivalent waypoint area falls entirely within the signal-restricted area, it does not move the inspection waypoint. If there is no inspection waypoint, the inspection path that falls within the signal-restricted area is adjusted to outside the signal-restricted area. The image processing module is used to identify and determine whether the bridge bearing image is qualified using a pre-configured lightweight AI image analysis model. If it is not qualified, the module controls the gimbal camera to retake the image at the original gimbal angle or / and add gimbal angles to retake the image until a qualified bridge bearing image is obtained. The data processing module includes: Used to stitch together the images of the bridge bearings to generate a panoramic view or a three-dimensional model of the bridge bearings; It is used to analyze panoramic or 3D models using pre-built deep learning models, identify and quantify the defect types and levels of supports, and then generate inspection reports.

Images