A monitoring and analysis system for autonomous inspection operations of unmanned aerial vehicles (UAVs)

Abstract

This invention discloses an autonomous unmanned aerial vehicle (UAV) inspection operation monitoring and analysis system, belonging to the field of UAV inspection operation analysis technology. The invention processes collected railway track image data, railway catenary thermal imaging data, and 3D point cloud data along the railway line to obtain visual indices, functional indices, and structural indices. The inspection result generation module compares these indices with thresholds and performs corresponding operations, such as marking the priority of defect maintenance or structural maintenance, and pushing relevant data to maintenance personnel. This fully leverages the value of the data, provides guidance for optimizing subsequent inspection work, and solves the problem of existing technologies lacking efficient analysis and processing mechanisms, thus failing to fully exploit the value of the data.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) inspection operation analysis technology, specifically to an autonomous UAV inspection operation monitoring and analysis system. Background Technology

[0002] As a vital national transportation infrastructure, the safe operation of railways is of paramount importance. Traditional railway inspections mainly rely on manual inspections, with inspectors patrolling along the railway tracks. This method is not only inefficient and labor-intensive, but also highly susceptible to environmental factors, making it prone to missed inspections. This is especially true in areas with complex terrain and harsh environments, such as mountains, tunnels, and bridges, where manual inspections are even more difficult and risky. With the continuous development of technology, drone technology has been gradually applied to the field of railway inspection. Drones can quickly and efficiently inspect railway lines, greatly improving inspection efficiency and coverage.

[0003] Although drone inspections have been initially applied in railway inspections, the following shortcomings still exist:

[0004] On the one hand, during the inspection process, the large amount of data obtained by drones lacks an efficient analysis and processing mechanism, which makes it impossible to fully assess the status of railway facilities and fully explore the value of the data to guide the optimization of subsequent inspection work.

[0005] On the other hand, the confidence level of the current railway facility status cannot be determined based on the flight data of the drone during the inspection process, and this cannot be used as a basis to determine whether the drone inspection operation needs to be carried out again, which leads to the inability to guarantee the accuracy of the analysis results.

[0006] To address this, a monitoring and analysis system for autonomous drone inspection operations has been developed. Summary of the Invention

[0007] The purpose of this invention is to solve the problems pointed out in the background art by proposing an autonomous inspection operation monitoring and analysis system for unmanned aerial vehicles (UAVs).

[0008] The objective of this invention can be achieved through the following technical solution: a monitoring and analysis system for autonomous inspection operations of unmanned aerial vehicles (UAVs), comprising:

[0009] Inspection target acquisition module: During the autonomous flight of the UAV according to the planned inspection route, image data of railway tracks in the target area are acquired;

[0010] Inspection target analysis module: Receives image data of railway tracks in the target area, and after analysis and processing, obtains the visual index of railway tracks in the target area;

[0011] Specifically:

[0012] Identify the defect area corresponding to different defects in railway tracks. The defect area includes the crack area, the damaged area, and the missing area.

[0013] The crack area, damaged area, and missing area corresponding to t evaluation areas are statistically analyzed, normalized, multiplied by the corresponding set weight coefficients, and then summed to obtain t sets of evaluation indices. The visual index is obtained by summing the t sets of evaluation indices.

[0014] Inspection result generation module: Receives the visual index of the railway track in the target area and compares it with the corresponding set threshold index, and executes the corresponding steps based on the comparison result;

[0015] Specifically:

[0016] If the visual index is higher than the corresponding set threshold index, the evaluation indices corresponding to the t evaluation areas will be arranged in descending order as the priority order for defect maintenance of each evaluation area.

[0017] At the same time, the crack area, damaged area and missing area of ​​each assessment area corresponding to the railway track of the current target area are extracted from the analysis of the previous UAV inspection operation, and used as the historical crack area, historical damaged area and historical missing area of ​​each assessment area;

[0018] The crack area, damaged area, and missing area of ​​each assessment area at the current time point are combined with the historical crack area, historical damaged area, and historical missing area to obtain the visual dynamic change value of each assessment area.

[0019] Define a change dataset corresponding to the visual dynamic change values. The change dataset stores the value range of each group of visual dynamic change values, and each value range corresponds to a dynamic change rating. The dynamic change rating includes an improvement rating, no significant change rating, and a deterioration rating. Mark the dynamic change rating of each assessment area. After marking, push the marking results to the operation and maintenance personnel and trigger an alarm signal.

[0020] In a preferred embodiment of the present invention, the specific process of identifying different defects in railway tracks is as follows:

[0021] For rail cracks, the crack profile is determined by contour detection and geometric calculation, thereby obtaining the crack region of the rail crack;

[0022] Image segmentation was used to separate the damaged areas of the sleepers from the railway track image data;

[0023] The missing areas in the ballast region are separated from the railway track image data through image segmentation;

[0024] The railway line corresponding to the current inspection process of the UAV is divided into t evaluation areas, t=1,2,...,r, where r is the total number of evaluation areas.

[0025] In a preferred embodiment of the present invention, the inspection target acquisition module and the inspection target analysis module are further used for:

[0026] During the autonomous flight of the drone according to the planned inspection route, thermal imaging data of the railway catenary of the railway track in the target area and three-dimensional point cloud data along the target area are collected.

[0027] The system receives thermal imaging data of the railway catenary in the target area and three-dimensional point cloud data along the railway line in the target area, and then analyzes and processes the data to obtain the functional index and structural index of the railway track in the target area.

[0028] In a preferred embodiment of the present invention, obtaining the functional index and structural index of the railway track in the target area specifically involves:

[0029] For the thermal imaging data of the railway catenary in the target area, the components of the catenary are identified and located, and then the types of components are classified. After classification, the regional images of each catenary component are extracted from the thermal imaging data.

[0030] Calculate the average temperature of each group of regional images, calculate the ratio of the average temperature of each contact wire component to the corresponding set temperature threshold, set the weight corresponding to each component, and take the average of the ratios of each group and the corresponding weights to obtain the functional index.

[0031] The centerline is determined by fitting the three-dimensional point cloud data. The distance from the point clouds on both sides to the centerline is calculated to obtain the track gauge value. For each set of track gauge values ​​corresponding to t evaluation areas, the difference between each set and the set standard track gauge is calculated and the absolute value is taken to obtain the track gauge deviation value. The average value of each set of track gauge deviation values ​​in the same evaluation area is calculated to obtain the track gauge evaluation value. The track gauge evaluation values ​​of t evaluation areas are summed to obtain the structure index.

[0032] In a preferred embodiment of the present invention, the step of performing the corresponding steps based on the comparison result of the functional index and the corresponding threshold index specifically includes:

[0033] If the functional index is higher than the corresponding set threshold index, the ambient temperature of the target area is first obtained, and the temperature reference range corresponding to high temperature weather is set. If the ambient temperature is within the temperature reference range, a high temperature signal is pushed to the operation and maintenance personnel; otherwise, the functional index and thermal imaging data of the railway catenary in the target area are pushed to the operation and maintenance personnel, and an alarm signal is triggered at the same time.

[0034] In a preferred embodiment of the present invention, the step of performing the corresponding steps based on the comparison result of the structure index and the corresponding threshold index specifically includes:

[0035] If the structural index is higher than the corresponding set threshold index, the track gauge assessment values ​​corresponding to the t assessment areas will be arranged in descending order. The arrangement position will be used as the priority order for structural maintenance of each assessment area, and the information will be sent to the maintenance personnel, while triggering an alarm signal.

[0036] As a preferred embodiment of the present invention, it further includes:

[0037] The autonomous flight module for unmanned aerial vehicles includes a positioning and navigation unit, an obstacle avoidance and perception unit, and a flight control unit.

[0038] The positioning and navigation unit uses a global satellite navigation system and an inertial navigation system to control the UAV to fly along a preset railway inspection route;

[0039] The obstacle avoidance unit uses pre-deployed lidar on the drone to perceive the environmental information around the drone in real time. When an obstacle is detected in front, it controls the drone to safely bypass the obstacle.

[0040] The flight control unit is used to control the UAV to perform inspections according to pre-planned inspection requirements; the inspection requirements include inspection area, inspection route, flight altitude, flight speed, and inspection time;

[0041] The re-monitoring execution module: When the re-monitoring signal is triggered, it evaluates the drone's attitude data, flight speed, and flight altitude after completing the current inspection route, and extracts the drone's attitude data during the current inspection operation, including pitch angle, roll angle, and yaw angle; it counts the number of times the attitude data exceeds the corresponding preset threshold range, and divides it by the flight time consumed by the current inspection operation to obtain the abnormal frequency of the current inspection operation; it extracts the flight speed at each time point of the current inspection operation, takes the average value to obtain the average flight speed, calculates the deviation rate between the flight speed and the preset route speed to obtain the speed deviation rate of the current inspection operation; it extracts the actual flight altitude of the current inspection operation, calculates the deviation rate between the actual flight altitude and the preset flight altitude to obtain the altitude deviation rate of the current inspection operation.

[0042] By comprehensively analyzing the frequency of anomalies, speed deviation, and altitude deviation, a confidence index is obtained for the UAV in completing the current inspection route.

[0043] If the confidence index is higher than the corresponding set threshold index, a confirmation re-monitoring signal will be sent to the operation and maintenance personnel. If the operation and maintenance personnel confirm again or the confidence index is lower than the set threshold index, the drone will be controlled again to perform the target area inspection operation.

[0044] In a preferred embodiment of the present invention, after triggering the alarm signaling, the operation and maintenance personnel may selectively trigger the re-monitoring signaling, and after triggering the re-examination signaling, it is sent to the re-monitoring execution module.

[0045] In a preferred embodiment of the present invention, obtaining the confidence index of the UAV in completing the current inspection route is specifically as follows:

[0046] Anomaly frequency, velocity deviation, and altitude deviation are used as a set of inverse indicators. A reference dataset is set corresponding to the set of inverse indicators. The reference dataset includes the allowable anomaly frequency, allowable velocity deviation, and allowable altitude deviation.

[0047] The ratios of abnormal frequency, speed deviation rate, and height deviation rate to the allowable abnormal frequency, speed deviation rate, and height deviation rate are calculated to obtain the frequency exceeding the allowable value, speed exceeding the allowable value, and height exceeding the allowable value.

[0048] The frequency excess, speed excess, and altitude excess are multiplied by their respective weighting coefficients and then summed to obtain the overall excess value. The confidence index of the UAV in completing the current inspection route is obtained by dividing the overall excess value by an integer.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] This invention processes collected railway track image data, railway catenary thermal imaging data, and 3D point cloud data along the railway line to obtain visual indices, functional indices, and structural indices, respectively. The inspection result generation module compares these indices with thresholds and performs corresponding operations, such as marking the priority of defect maintenance or structural maintenance and pushing relevant data to maintenance personnel. This fully taps into the value of the data, provides guidance for optimizing subsequent inspection work, and solves the problem of existing technologies lacking efficient analysis and processing mechanisms and failing to fully tap into the value of data.

[0051] This invention evaluates the attitude data, flight speed, flight altitude, and other status parameters of the UAV after completing the current inspection route when a re-monitoring signal is triggered, and obtains a confidence index. Based on the confidence index, it determines whether to conduct another inspection or notify maintenance personnel for confirmation. This achieves comprehensive and accurate monitoring of the UAV's operational status, and improves the intelligence level of UAV inspection and the accuracy of data analysis.

[0052] This invention evaluates railway tracks and overhead contact lines from multiple dimensions by calculating visual, functional, and structural indices. The visual index considers defects such as rail cracks, sleeper damage, and missing ballast; the functional index analyzes the functional status of overhead contact line components based on temperature; and the structural index assesses track structural stability through gauge deviation. This comprehensive evaluation method can more accurately grasp the overall condition of railway facilities and promptly identify potential safety hazards. Attached Figure Description

[0053] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0054] Figure 1 This is a schematic diagram of the principle of the present invention;

[0055] Figure 2 This is a schematic diagram of obstacle avoidance during the autonomous flight of the UAV in this invention. Detailed Implementation

[0056] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Please see Figure 1-2 As shown, an autonomous inspection operation monitoring and analysis system for unmanned aerial vehicles (UAVs) includes an autonomous flight module for UAVs, an inspection target acquisition module, an inspection target analysis module, an inspection result generation module, and a follow-up monitoring execution module.

[0058] The autonomous flight module of an unmanned aerial vehicle (UAV) includes a positioning and navigation unit, a perception and obstacle avoidance unit, and a flight control unit.

[0059] The positioning and navigation unit uses a combination of Global Navigation Satellite System (GNSS) and Inertial Navigation System (INS) to provide the UAV with accurate positioning and navigation information. Multiple differential base stations are set up along the railway line, and real-time dynamic differential positioning (RTK) technology is used to further improve the positioning accuracy of the UAV and ensure that it can fly accurately along the preset railway inspection route.

[0060] The obstacle avoidance unit utilizes pre-deployed environmental perception devices on the drone, including but not limited to lidar, ultrasonic sensors, and visual sensors, to perceive the surrounding environment in real time. By fusing and processing this sensor data, a three-dimensional environmental model of the drone's surroundings is constructed, enabling rapid identification and precise location of obstacles along the railway line. When an obstacle is detected ahead, the obstacle avoidance system automatically plans an obstacle avoidance path, controlling the drone to safely bypass the obstacle and ensuring flight safety.

[0061] The flight control unit is used to control the UAV to perform inspections according to pre-planned inspection requirements; the inspection requirements include inspection area, inspection route, flight altitude, flight speed, inspection time, etc.

[0062] Based on GIS railway electronic maps and 3D terrain data, collision-free flight paths are generated to avoid no-fly zones (such as stations and residential areas) and high-risk areas (such as mountain rockfall sections).

[0063] The inspection target acquisition module is used to collect image data of railway tracks, thermal imaging data of railway catenary, and three-dimensional point cloud data along the target area during the autonomous flight of the UAV according to the planned inspection route, using a pre-loaded high-definition visible light camera, infrared thermal imager, and laser scanner, and transmit them to the inspection target analysis module.

[0064] High-definition visible light camera: 20 megapixels, 1 / 2.3-inch CMOS sensor, supports 4K images;

[0065] Infrared thermal imager: resolution 640×512, temperature measurement range -20~500℃, accuracy ±2℃;

[0066] Laser scanner: scanning rate 200kHz, ranging accuracy ±5mm;

[0067] The inspection target analysis module is used to receive image data of railway tracks in the target area, and after analysis and processing, obtain the visual index of railway tracks in the target area.

[0068] Specifically:

[0069] The 4K image (3840×2160 pixels) was reconstructed to a higher resolution using a bicubic interpolation algorithm, improving the recognition of track details (such as 0.5mm level crack detection).

[0070] Non-local mean filtering (NLM) is used to remove image sensor noise while preserving edge details (suitable for the small pixel noise characteristics of 1 / 2.3-inch CMOS).

[0071] Based on UAV POS data (position + attitude), the image data of railway tracks is converted into a railway engineering coordinate system (X-axis extends along the track, Y-axis is perpendicular to the track) using a homography matrix; after locating the main track area, irrelevant backgrounds (such as sky, vegetation, etc.) are excluded to reduce the amount of computation.

[0072] A defect identification model is pre-built and trained on a dataset containing 200,000+ orbital defect samples, which can effectively detect orbital defects;

[0073] Based on UAV POS data, railway track image data is converted into a railway engineering coordinate system using a homography matrix, ensuring a precise correspondence between the image and the actual track position. This facilitates accurate location and analysis of track defects. Automatically locating the main track area and eliminating irrelevant background reduces unnecessary computation, enabling the system to process image data more efficiently and improving the overall efficiency of inspection operations. This allows for the completion of inspection and analysis of the target area in a shorter time.

[0074] Defects in railway track image data are identified using a pre-built defect identification model and classified according to defect categories, including rail cracks, sleeper damage, and missing ballast.

[0075] For rail cracks, the crack profile is determined by contour detection and geometric calculation, thereby obtaining the crack region of the rail crack. The position of the crack region on the track is determined according to the railway engineering coordinate system. The position and area of ​​the rail crack on the track are input into a pre-constructed 3D railway model for display. The crack area is obtained by statistically analyzing the pixels of the damaged area and converting them based on the image resolution.

[0076] The three-dimensional model of the railway is a complete model when the railway is put into use;

[0077] For example, the construction process is as follows:

[0078] Based on the processed DEM and DOM data, a three-dimensional terrain model of the railway line is generated in modeling software (such as SketchUp and Bentley ContextCapture) to restore the surrounding environment such as ground undulations, roads, and vegetation.

[0079] Based on the track point cloud data, the centerline and cross-sectional profile of the rail are extracted to generate a 3D model of the rail (including rail head, rail web, and rail base) that matches the actual size.

[0080] Based on the actual spacing and specifications, generate sleeper models in batches and precisely connect them with the rails, while adding detailed models such as fasteners and shims;

[0081] Construct a ballast bed model to recreate the distribution pattern of the ballast (this can be done by fitting point cloud data or by using a predefined ballast bed template).

[0082] Add models of railway ancillary facilities such as overhead contact lines (including supports, catenary wires, and contact wires), signals, turnouts, bridges, and tunnels. These models can be generated by adjusting parameters based on a standard component library (such as the overhead contact line model library commonly used in the railway industry) to ensure that they are consistent with the location and specifications of the actual facilities.

[0083] High-resolution texture photos associated with the model are applied to the corresponding model surface (such as the metal texture of the rails and the wood texture of the sleepers), and lighting and shadow rendering are performed to make the model visually close to the real railway scene.

[0084] The damaged areas of the sleepers are separated from the railway track image data through image segmentation, and the location of the damaged areas on the track is determined using the railway engineering coordinate system. The location and area of ​​the damaged areas on the track are then input into a pre-constructed 3D railway model for display. The damaged area is obtained by counting the pixels of the damaged areas and converting them based on the image resolution.

[0085] The missing regions in the ballast area are separated from the railway track image data through image segmentation, and the position of the missing regions on the track is determined by the railway engineering coordinate system. The position and area of ​​the missing regions on the track are then input into a pre-constructed 3D railway model for display. The missing area is obtained by counting the pixels of the missing regions and converting them based on the image resolution.

[0086] The railway line corresponding to the current inspection process of the drone is divided into t evaluation areas, t=1,2,...,r, where r is the total number of evaluation areas;

[0087] After normalizing the crack area, damaged area and missing area corresponding to each of the t evaluation areas, multiply them by the corresponding set weight coefficients and then sum them to obtain the evaluation index corresponding to each of the t evaluation areas. The evaluation index of the t evaluation areas is then summed to obtain the visual index of the railway track in the target area.

[0088] The weighting coefficients for crack area, damaged area, and missing area in different assessment regions are set according to the importance of the divided regions;

[0089] It is also used to receive thermal imaging data of railway catenary and to analyze and process it to obtain the functional index of the railway catenary in the target area.

[0090] Specifically:

[0091] For thermal imaging data of railway catenary in the target area, pre-trained target detection models, such as the YOLO series models, are used to identify and locate the components of the catenary. After location, the component types are classified, such as brackets, droppers, insulators, etc.

[0092] After segmentation, regional images of each contact wire component are extracted from the thermal imaging data. The average temperature is calculated for each extracted regional image of the contact wire component. In the thermal imaging image, there is a certain mapping relationship between pixel values ​​and temperature. The pixel values ​​can be converted into actual temperature values ​​according to the calibration parameters of the thermal imaging equipment.

[0093] Based on the operating standards and experience of railway overhead contact systems, normal temperature thresholds are set for different overhead contact system components;

[0094] The average temperature calculated for each contact network component area is compared with the corresponding set temperature threshold, with the average temperature as the numerator and the temperature threshold as the denominator. Based on the importance of each contact network component to the overall function of the railway contact network, a corresponding weight is assigned to each component. The calculated ratios of each group are multiplied by the corresponding weights, and then the average value is taken to obtain the functional index of the railway contact network in the target area.

[0095] By classifying the contact wire components and extracting images of their respective areas to calculate the average temperature, the shortcomings of existing technologies that may perform general temperature analysis on the entire contact wire area are avoided. Different components have different temperature change characteristics during normal operation and when a fault occurs, and targeted temperature extraction can more accurately reflect the actual working status of each component.

[0096] It is also used to receive three-dimensional point cloud data along the target area and perform analysis and processing to obtain the structural index of the target area;

[0097] Specifically:

[0098] The three-dimensional point cloud data along the target area is fitted, and the orbital plane is fitted using the RANSAC (Random Sample Consensus) algorithm. The centerline is then determined based on the plane's normal vector and the point cloud distribution.

[0099] Based on the extracted track centerline, the track gauge is calculated. By finding the point clouds on both sides of the track, the distance from the point clouds on both sides to the centerline is calculated, thus obtaining the track gauge value.

[0100] For each set of track gauge values ​​corresponding to t evaluation areas, the difference between the difference and the set standard track gauge is calculated and the absolute value is taken to obtain the track gauge deviation value; the average value of each set of track gauge deviation values ​​is taken to obtain the track gauge evaluation value corresponding to each t evaluation area.

[0101] For each set of track gauge values, the difference between the value and the set standard track gauge is calculated and the absolute value is taken to obtain the track gauge deviation value; the average value of each set of track gauge deviation values ​​is taken to obtain the track gauge evaluation value of the target area.

[0102] The structural index of the railway track in the target area is obtained by summing the track gauge evaluation values ​​of t evaluation areas.

[0103] It not only calculates the track gauge assessment value for each evaluation area, but also the track gauge assessment value for the target area, providing a comprehensive assessment of the track gauge from local to global perspectives. By analyzing and averaging the track gauge deviation values ​​from multiple evaluation areas, it is possible to more accurately understand the track gauge variations at different locations and the overall track gauge status.

[0104] The inspection result generation module is used to receive the visual index, functional index and structural index of the railway track in the target area, compare them with the corresponding set threshold index, and execute the corresponding steps based on the comparison results.

[0105] Specifically:

[0106] S1: If the visual index is higher than the corresponding set threshold index, first extract the evaluation index corresponding to the t evaluation areas from the railway 3D model after the input is completed, arrange them in descending order, and mark them on the railway 3D model according to the arrangement position as the priority order for defect maintenance of each evaluation area.

[0107] Simultaneously, the crack area, damaged area, and missing area of ​​each assessment area corresponding to the railway track in the current target area are extracted from the analysis of the previous UAV inspection operation. These are used as the historical crack area, historical damaged area, and historical missing area of ​​each assessment area. express;

[0108] The crack area, damaged area, and missing area of ​​each assessment area at the current time point are used as follows: express;

[0109] Using formula The visual dynamic change values ​​of each assessment area were calculated; where These are the preset weighting coefficients;

[0110] A change dataset corresponding to the visual dynamic change values ​​is set. The change dataset stores the value range of each group of visual dynamic change values, and each value range corresponds to a dynamic change rating. The dynamic change rating includes an improvement rating, no significant change rating, and a deterioration rating. The dynamic change rating of each assessment area is also marked on the railway 3D model. After marking, the railway 3D model is pushed to the operation and maintenance personnel, and an alarm signal is triggered at the same time.

[0111] Based on the visual dynamic change values, maintenance personnel can quickly identify which areas have significant status changes and which areas are relatively stable, thereby developing targeted maintenance plans. At the same time, this quantitative value can also serve as an indicator for evaluating the effectiveness of maintenance. After the maintenance work is completed, the effectiveness of the maintenance measures can be evaluated by comparing the visual dynamic change values ​​before and after the maintenance.

[0112] S2: If the functional index is higher than the corresponding set threshold index, first obtain the ambient temperature of the target area, set the temperature reference range corresponding to high temperature weather, and if the ambient temperature is within the temperature reference range, push the high temperature signal to the operation and maintenance personnel; otherwise, push the functional index and thermal imaging data of the railway catenary in the target area to the operation and maintenance personnel, and trigger the alarm signal at the same time.

[0113] S3: If the structural index is higher than the corresponding set threshold index, the track gauge evaluation values ​​corresponding to the t evaluation areas will be arranged in descending order on the railway 3D model after the input is completed, and the arrangement position will be marked on the railway 3D model as the priority order for structural maintenance of each evaluation area. The marked railway 3D model will be pushed to the operation and maintenance personnel, and an alarm signal will be triggered at the same time.

[0114] S4: After triggering the alarm signaling, the operation and maintenance personnel can selectively trigger the re-monitoring signaling, and after triggering the re-monitoring signaling, it is sent to the re-monitoring execution module;

[0115] The re-monitoring execution module is used to evaluate the status parameters of the UAV after completing the current inspection route when a re-monitoring signal is triggered, and obtain the confidence index of the UAV in the process of completing the current inspection route. If the confidence index is higher than the corresponding set threshold index, a confirmation re-monitoring signal is pushed to the operation and maintenance personnel. If the operation and maintenance personnel confirm again or the confidence index is lower than the set threshold index, the UAV is controlled to perform the target area inspection operation again. The status parameters include UAV attitude data, flight speed, and flight altitude.

[0116] Specifically:

[0117] First, obtain the battery level of the drone before and after executing the current inspection route, and calculate the difference to obtain the battery power consumed by the drone for a single inspection route, which is recorded as the unit power consumption.

[0118] A preset value of the battery power that the drone needs to retain outside of operation is added to the power consumption of a single drone as a threshold battery power. The remaining battery power of the drone when the monitoring signal is triggered is compared with the threshold battery power. If it is lower than the threshold battery power, the drone is replaced; otherwise, it is put into use.

[0119] Extract the attitude data of the UAV during the current inspection operation from the UAV flight data, including pitch angle, roll angle and yaw angle; count the number of times the attitude data exceeds the corresponding preset threshold range, and divide it by the flight time consumed in the current inspection operation to obtain the abnormal frequency of the current inspection operation.

[0120] Extract the flight speed at each time point in the current inspection operation from the UAV flight data, take the average value to obtain the average flight speed, calculate the deviation rate between the flight speed and the preset route speed, and obtain the speed deviation rate of the current inspection operation.

[0121] Extract the actual flight altitude of the current inspection operation from the drone flight data, calculate the deviation rate between the actual flight altitude and the preset flight altitude, and obtain the altitude deviation rate of the current inspection operation.

[0122] Fluctuations and anomalies in attitude data, such as pitch, roll, and yaw angles, can lead to camera instability, resulting in blurry images or angular deviations, thus affecting data quality.

[0123] Flight speed deviation rate: if the speed is too high, it may result in insufficient image acquisition intervals and missed details; if it is too slow, it may increase the impact of vibration.

[0124] High-precision control is crucial; inaccurate height will affect the sensor's resolution and coverage, impacting the accuracy of thermal imaging or point cloud data.

[0125] Extract the frequency of anomalies, the velocity deviation rate, and the height deviation rate as a set of inverse indicators. Set a reference dataset corresponding to the set of inverse indicators. The reference dataset includes the allowable frequency of anomalies, the allowable velocity deviation rate, and the allowable height deviation rate.

[0126] The abnormal frequency, speed deviation rate, and height deviation rate are calculated as ratios to the permissible abnormal frequency, speed deviation rate, and height deviation rate, respectively. The abnormal frequency, speed deviation rate, and height deviation rate are used as the numerators, and the permissible abnormal frequency, speed deviation rate, and height deviation rate are used as the denominators to obtain the frequency exceeding the permissible value, speed exceeding the permissible value, and height exceeding the permissible value.

[0127] The frequency exceeding the allowable value, speed exceeding the allowable value, and altitude exceeding the allowable value are multiplied by the corresponding set weight coefficients, and then summed to obtain the comprehensive exceeding value. The confidence index of the UAV in completing the current inspection route is obtained by dividing the comprehensive exceeding value by an integer.

[0128] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A monitoring and analysis system for autonomous inspection operations of unmanned aerial vehicles (UAVs), characterized in that, include: Inspection target acquisition module: During the autonomous flight of the UAV according to the planned inspection route, image data of railway tracks in the target area are acquired; Inspection target analysis module: Receives image data of railway tracks in the target area, and after analysis and processing, obtains the visual index of railway tracks in the target area; Specifically: Identify the defect area corresponding to different defects in railway tracks. The defect area includes the crack area, the damaged area, and the missing area. The crack area, damaged area, and missing area corresponding to t evaluation areas are statistically analyzed, normalized, multiplied by the corresponding set weight coefficients, and then summed to obtain t sets of evaluation indices. The visual index is obtained by summing the t sets of evaluation indices. Inspection result generation module: Receives the visual index of the railway track in the target area and compares it with the corresponding set threshold index, and executes the corresponding steps based on the comparison result; Specifically: If the visual index is higher than the corresponding set threshold index, the evaluation indices corresponding to the t evaluation areas will be arranged in descending order as the priority order for defect maintenance of each evaluation area. At the same time, the crack area, damaged area and missing area of ​​each assessment area corresponding to the railway track of the current target area are extracted from the analysis of the previous UAV inspection operation, and used as the historical crack area, historical damaged area and historical missing area of ​​each assessment area; The crack area, damaged area, and missing area of ​​each assessment area at the current time point are combined with the historical crack area, historical damaged area, and historical missing area to obtain the visual dynamic change value of each assessment area. A change dataset corresponding to the visual dynamic change values ​​is defined. The change dataset stores the value range of each group of visual dynamic change values, and each value range corresponds to a dynamic change rating. The dynamic change rating includes a rating of improvement, a rating of no significant change, and a rating of deterioration. Mark the dynamic changes in the rating of each assessment area; Once the marking is complete, the marking results will be pushed to the operations and maintenance personnel, and an alarm signal will be triggered at the same time.

2. The UAV autonomous inspection operation monitoring and analysis system according to claim 1, characterized in that, The specific process for identifying different defects in railway tracks is as follows: For rail cracks, the crack profile is determined by contour detection and geometric calculation, thereby obtaining the crack region of the rail crack; Image segmentation was used to separate the damaged areas of the sleepers from the railway track image data; The missing areas in the ballast region are separated from the railway track image data through image segmentation; The railway line corresponding to the current inspection process of the UAV is divided into t evaluation areas, t=1,2,...,r, where r is the total number of evaluation areas.

3. The UAV autonomous inspection operation monitoring and analysis system according to claim 1, characterized in that, The inspection target acquisition module and the inspection target analysis module are also used for: During the autonomous flight of the drone according to the planned inspection route, thermal imaging data of the railway catenary of the railway track in the target area and three-dimensional point cloud data along the target area are collected. The system receives thermal imaging data of the railway catenary in the target area and three-dimensional point cloud data along the railway line in the target area, and then analyzes and processes the data to obtain the functional index and structural index of the railway track in the target area.

4. The UAV autonomous inspection operation monitoring and analysis system according to claim 3, characterized in that, The functional and structural indices of the railway tracks in the target area are obtained, specifically: For the thermal imaging data of the railway catenary in the target area, the components of the catenary are identified and located, and then the types of components are classified. After classification, the regional images of each catenary component are extracted from the thermal imaging data. Calculate the average temperature of each group of regional images, calculate the ratio of the average temperature of each contact wire component to the corresponding set temperature threshold, set the weight corresponding to each component, and take the average of the ratios of each group and the corresponding weights to obtain the functional index. The centerline is determined by fitting the three-dimensional point cloud data. The distance from the point clouds on both sides to the centerline is calculated to obtain the track gauge value. For each set of track gauge values ​​corresponding to t evaluation areas, the difference between each set and the set standard track gauge is calculated and the absolute value is taken to obtain the track gauge deviation value. The average value of each set of track gauge deviation values ​​in the same evaluation area is calculated to obtain the track gauge evaluation value. The track gauge evaluation values ​​of t evaluation areas are summed to obtain the structure index.

5. The UAV autonomous inspection operation monitoring and analysis system according to claim 4, characterized in that, Based on the comparison results between the functional index and the corresponding threshold index, the following steps are performed: If the functional index is higher than the corresponding set threshold index, the ambient temperature of the target area is first obtained, and the temperature reference range corresponding to high temperature weather is set. If the ambient temperature is within the temperature reference range, a high temperature signal is pushed to the operation and maintenance personnel; otherwise, the functional index and thermal imaging data of the railway catenary in the target area are pushed to the operation and maintenance personnel, and an alarm signal is triggered at the same time.

6. The UAV autonomous inspection operation monitoring and analysis system according to claim 4, characterized in that, Based on the comparison results between the structure index and the corresponding threshold index, the following steps are performed: If the structural index is higher than the corresponding set threshold index, the track gauge assessment values ​​corresponding to the t assessment areas will be arranged in descending order. The arrangement position will be used as the priority order for structural maintenance of each assessment area, and the information will be sent to the maintenance personnel, while triggering an alarm signal.

7. The UAV autonomous inspection operation monitoring and analysis system according to claim 1, characterized in that, Also includes: The autonomous flight module for unmanned aerial vehicles includes a positioning and navigation unit, an obstacle avoidance and perception unit, and a flight control unit. The positioning and navigation unit uses a global satellite navigation system and an inertial navigation system to control the UAV to fly along a preset railway inspection route; The obstacle avoidance unit uses pre-deployed lidar on the drone to perceive the environmental information around the drone in real time. When an obstacle is detected in front, it controls the drone to safely bypass the obstacle. The flight control unit is used to control the drone to perform inspections according to pre-planned inspection requirements; Inspection requirements include inspection area, inspection route, flight altitude, flight speed, and inspection time; The re-monitoring execution module: When the re-monitoring signal is triggered, it evaluates the drone's attitude data, flight speed, and flight altitude after completing the current inspection route, and extracts the drone's attitude data during the current inspection operation, including pitch angle, roll angle, and yaw angle; it counts the number of times the attitude data exceeds the corresponding preset threshold range, and divides it by the flight time consumed by the current inspection operation to obtain the abnormal frequency of the current inspection operation; it extracts the flight speed at each time point of the current inspection operation, takes the average value to obtain the average flight speed, calculates the deviation rate between the flight speed and the preset route speed to obtain the speed deviation rate of the current inspection operation; it extracts the actual flight altitude of the current inspection operation, calculates the deviation rate between the actual flight altitude and the preset flight altitude to obtain the altitude deviation rate of the current inspection operation. By comprehensively analyzing the frequency of anomalies, speed deviation, and altitude deviation, a confidence index is obtained for the UAV in completing the current inspection route. If the confidence index is higher than the corresponding set threshold index, a confirmation re-monitoring signal will be sent to the operation and maintenance personnel. If the operation and maintenance personnel confirm again or the confidence index is lower than the set threshold index, the drone will be controlled again to perform the target area inspection operation.

8. The UAV autonomous inspection operation monitoring and analysis system according to claim 7, characterized in that, After the alarm signal is triggered, the operation and maintenance personnel can selectively trigger the re-monitoring signal, which is then sent to the re-monitoring execution module.

9. The UAV autonomous inspection operation monitoring and analysis system according to claim 7, characterized in that, The confidence index of the drone in completing the current inspection route is obtained as follows: Anomaly frequency, velocity deviation, and altitude deviation are used as a set of inverse indicators. A reference dataset is set corresponding to the set of inverse indicators. The reference dataset includes the allowable anomaly frequency, allowable velocity deviation, and allowable altitude deviation. The ratios of abnormal frequency, speed deviation rate, and height deviation rate to the allowable abnormal frequency, speed deviation rate, and height deviation rate are calculated to obtain the frequency exceeding the allowable value, speed exceeding the allowable value, and height exceeding the allowable value. The frequency excess, speed excess, and altitude excess are multiplied by their respective weighting coefficients and then summed to obtain the overall excess value. The confidence index of the UAV in completing the current inspection route is obtained by dividing the overall excess value by an integer.

Images