Unmanned aerial vehicle intelligent inspection method and system based on FAST structure anomaly target identification

Abstract

The application discloses an unmanned aerial vehicle intelligent inspection method and system based on FAST structure abnormal target identification, and comprises the following steps: S1, establishing an unmanned aerial vehicle control platform; S2, setting an unmanned aerial vehicle flight path scheme in combination with a multi-source data acquisition type and telescope structure characteristics; S3, collecting infrared images and visible light images by using an unmanned aerial vehicle payload for a FAST complex structure abnormal target; obtaining a visible-infrared image pair after registration; S4, labeling a visible light image of abnormal target data, and forming a multi-source FAST structure abnormal target data set by combining an abnormal target label after labeling with the visible-infrared image pair; S5, pre-processing data in the FAST structure abnormal target data set to form an amplified data set; and S6, intelligently identifying and verifying the FAST structure abnormal target according to a multi-source fusion abnormal target detection model. The application first applies the unmanned aerial vehicle technology to the structure inspection of astronomical observation equipment, and can greatly improve the inspection efficiency of a radio telescope.

Description

Technical Field

[0001] This invention relates to the field of astronomy, and in particular to an intelligent inspection method and system for unmanned aerial vehicles (UAVs) based on the identification of structural anomalies in the FAST telescope. Background Technology

[0002] The Five-hundred-meter Aperture Spherical radio Telescope (FAST) is a major national science and technology infrastructure project. Utilizing a natural karst depression in Guizhou Province as its site, it was built as the world's largest single-aperture radio telescope, enabling high-precision astronomical observations over a large sky area. Since passing national acceptance in January 2020, FAST has discovered over 1,000 pulsars, achieving a series of significant scientific results during its high-quality operation.

[0003] FAST has a massive structure, including: a ring beam (5400 tons), a cable net (6670 main cables and 2225 pull cables), reflector units (4450 pieces), actuator anchors (2225), six towers, structural foundations, and unstable rock slopes. During FAST's operation, it requires inspection and maintenance. The inspection includes: panel sub-units, back frame, joint bearings, node plates, targets and target seats, main cables and pull cables, bolts and pins, sliding layers and limiting gates, etc. After more than 5 years of operation, the following common problems have been found: panel deformation, member instability; ring beam corrosion, target failure, damage and replacement of pull cable sheaths, repair of main cable sheaths, main cable pin condition, motion interference, tower corrosion, and loose connecting bolts, etc.

[0004] The node structure in the reflector is complex, with multiple reflector unit back frames connected to the upper plate of the node, and their kinematic pairs are also different. This area is prone to problems such as corrosion, deformation of spare parts and rods, and motion interference. Although there are cable-stayed systems available for maintenance, they have disadvantages such as low efficiency, high cost, and small working area for reflector inspection, making them unsuitable for large-area, high-efficiency inspection work.

[0005] There are six feed towers, each with a length of 120 to 170 meters. They are built according to the terrain and have a complex structure. Manual inspection of the tower body and the loose connecting bolts is inefficient, risky and subject to many limitations.

[0006] Currently, structural inspections are mainly conducted manually, which has problems such as low efficiency, lack of video recording, safety hazards, and significant impact from weather.

[0007] In general, the following problems exist in the current inspection of the reflector and feed tower:

[0008] (1) Current inspection methods are mostly traditional methods such as cable hoisting and manual climbing, without video recording;

[0009] (2) Inspection is difficult under complex weather conditions, such as rain, snow, severe cold, freezing and other severe weather conditions.

[0010] (3) More than half of the components are located at high altitudes, with poor accessibility, complex terrain, low inspection efficiency, and high inspection risk.

[0011] (4) During inspections, only simple notes can be taken, which are difficult to review and read. Summary of the Invention

[0012] To address the problems existing in the prior art, the present invention aims to provide a FAST drone intelligent inspection method and system. This intelligent inspection method effectively assists personnel in carrying out structural inspection work, and under reasonable data collection conditions, effectively improves the detection accuracy of structural anomalies such as panel dents and metal corrosion. Another objective of the present invention is to provide a FAST drone intelligent inspection system that implements the above-mentioned intelligent inspection method.

[0013] To achieve the above objectives, the FAST unmanned aerial vehicle (UAV) intelligent inspection method of the present invention includes the following steps:

[0014] S1. Establish a drone platform: Based on the drone inspection reflector path, set up several operation platforms for launching and retrieving drones;

[0015] S2. Combine the data acquisition types from multiple sources to set up the drone flight plan;

[0016] S3. For FAST structural anomaly targets, acquire infrared and visible light images; use the characteristic linear mapping image registration method to map the infrared image corresponding to the visible light image to the visible light image scale to obtain the registered visible-infrared image pair;

[0017] S4. Label the visible light images of the abnormal targets as panel structure defects, corrosion, screw loss and rod loss, respectively. The labeled abnormal targets and the visible-infrared image pairs form a multi-source FAST structural abnormal target dataset.

[0018] S5. Preprocess the data in the abnormal target dataset to form the augmented dataset;

[0019] S6. Based on the dataset in S5, the abnormal targets in the FAST structure are effectively identified and verified according to the multi-source fusion abnormal target detection model.

[0020] Furthermore, the method for setting the path of the UAV inspection reflector is as follows: set several inspection circles on the reflector. Starting from the first circle at the center of the reflector, first inspect the node disks in a circular motion along the first circle, then expand to the second circle, and then expand the inspection circles outward in sequence, increasing the inspection path circle by circle.

[0021] Furthermore, when the inspection radius increases beyond the drone's range, an operating platform for the drone is set up on the outer FAST measurement base, and the drone is launched and retrieved on the operating platform.

[0022] Furthermore, based on the cable net partitioning, the reflective surface of FAST was divided into 5 identical areas, and drones conducted inspections of each of the 5 areas.

[0023] Furthermore, in S2, the UAV flight plan is set as follows: the UAV flight route is determined based on the main anomaly types and spatial distribution of the FAST structure; the UAV flight area is determined based on the sensor acquisition distance, flight altitude limit, and safe flight time limit; and then the UAV inspection system structural anomaly data acquisition flight plan is determined.

[0024] Furthermore, in step S3, the infrared image undergoes a linear transformation of grayscale values ​​and is combined with image information extracted from abnormal target features in the visible light image for geometric registration correction; the geometrically registered and corrected image is then combined with the visible light image for image mapping to finally obtain the registered infrared image.

[0025] Furthermore, in step S5, the preprocessing steps for the data in the abnormal target dataset are as follows:

[0026] S5.1 Based on the category of abnormal targets, the abnormal target dataset is randomly grouped into training set, test set and validation set in a ratio of 8:1:1;

[0027] S5.2 performs data augmentation operations on abnormal target data by performing image flipping, random offsetting of abnormal targets, image cropping, adding motion blur and white noise.

[0028] Furthermore, in S6, the multi-source fusion abnormal target detection model is specifically as follows:

[0029] S6.1 initializes the detection model parameters based on the dataset in S5;

[0030] S6.2 Input the expanded multi-source FAST structural anomaly target training set into the detection model to train the detection model;

[0031] S6.3 Input the amplified multi-source FAST structural anomaly target validation set for validation;

[0032] Once S6.4 achieves its intended purpose, an effective multi-source fusion abnormal target detection model is formed.

[0033] Furthermore, the training process of the multi-source fusion abnormal target detection model is as follows:

[0034] 1) Set initial training parameters;

[0035] 2) Randomly collect a smaller dataset from the original training set;

[0036] 3) Adjust the training rounds to conduct small-sample training and obtain preliminary detection results for classification and regression losses at the county level;

[0037] 4) Determine if there are any issues with the dataset design. If there are no issues, use all training sets for training. If there are issues, return to step 2) and randomly collect data from the dataset.

[0038] 5) Set Dropout to 0.5, start formal training using the initial training parameters, and set the batch size to 4;

[0039] 6) Determine whether there is overfitting or underfitting based on the loss curve, and then decide whether to increase the training accuracy;

[0040] 7) Analyze the loss curves of model classification and regression and the defect detection results, and then optimize the analysis results.

[0041] On the other hand, the present invention provides an intelligent inspection system for unmanned aerial vehicles (UAVs) based on the identification of structural anomalies in FAST, which is used to implement the above-mentioned intelligent inspection method for FAST UAVs.

[0042] This invention, based on the inspection needs of large astronomical equipment structures, is the first to apply drone technology to the inspection of astronomical equipment structures. By selecting appropriate image samples, it can effectively inspect four types of abnormal targets: panel anomalies, metal corrosion, loose screws, and broken rods. Attached Figure Description

[0043] Figure 1 A framework diagram of an intelligent UAV inspection method based on FAST structural anomaly target identification;

[0044] Figure 2 Framework diagram for spatial registration of multi-source data;

[0045] Figure 3 A diagram illustrating the data preprocessing framework;

[0046] Figure 4 This is a framework diagram of a multi-source fusion anomaly target detection model;

[0047] Figure 5A flowchart illustrating the training process of a multi-source fusion anomaly detection model;

[0048] Figure 6 This is a schematic diagram of the cable net zoning.

[0049] Figure 7 Schematic diagram of drone inspection path planning for reflector surfaces Figure 1 ;

[0050] Figure 8 Schematic diagram of drone inspection path planning for reflector surfaces Figure 2 ;

[0051] Figure 9 This is a schematic diagram illustrating the operating principle of GCANet;

[0052] Figure 10 This is a schematic diagram of a Gaussian transformer encoder network;

[0053] Figure 11 The images are the detection results; where Figure a is the actual labeled image of the panel abnormality indentation; and Figure b is the detection result image of the panel abnormality indentation in Figure a.

[0054] Figure 12 Figure 1 shows the detection result image of another panel, where Figure c is the actual labeled image of the abnormal depression in the other panel; and Figure d is the detection result image of the abnormal depression in the panel of Figure c. Detailed Implementation

[0055] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0057] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0058] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0059] like Figures 1 to 12 As shown, this invention discloses an intelligent UAV inspection method and system based on FAST structural anomaly target identification. Utilizing an optimized UAV and payload, the system conducts flight inspection tests on the FAST telescope structure. Employing image recognition technology, combined with a sample feature learning mechanism, deep learning algorithms, and training models, a highly reusable, scalable, and innovative image defect identification algorithm is established. This algorithm inspects for faults such as contamination, deformation, and jamming in the telescope structure, quickly identifying problems and potential hazards in the FAST structure, providing a basis for maintenance and repair, and enabling construction units to carry out timely maintenance and construction.

[0060] The intelligent inspection method for unmanned aerial vehicles (UAVs) based on FAST structural anomaly target identification provided by this invention includes the following steps:

[0061] S1. Establish a drone platform: Based on the drone inspection reflector path, set up several operation platforms for launching and retrieving drones;

[0062] S2. Combine the data acquisition types from multiple sources to set up the drone flight plan;

[0063] S3. For FAST structural anomaly targets, acquire infrared and visible light images; use the characteristic linear mapping image registration method to map the infrared image corresponding to the visible light image to the visible light image scale to obtain the registered visible-infrared image pair;

[0064] S4. Label the visible light images of the abnormal targets as panel structure defects, corrosion, screw loss and rod loss, respectively. The labeled abnormal targets and the visible-infrared image pairs form a multi-source FAST structural abnormal target dataset.

[0065] S5. Preprocess the data in the abnormal target dataset to form the augmented dataset;

[0066] S6. Based on the dataset in S5, the abnormal targets in the FAST structure are effectively identified and verified according to the multi-source fusion abnormal target detection model.

[0067] In step S2, the specific flight plan for the drone is as follows:

[0068] like Figure 1 The diagram shown is a framework diagram of the UAV flight plan. Based on the actual needs of FAST structural anomaly data collection, and taking into account factors such as the type and characteristics of the anomaly targets to be inspected, UAV flight parameters, anomaly target detection accuracy requirements, UAV flight duration and working period requirements, and data storage capacity, a suitable UAV is selected.

[0069] Appendix Figure 1 Note: Simultaneously, addressing the main structural anomalies of FAST, such as broken unit members, loose ring beam bolts, holes in unit panels, and structural corrosion, and considering the spatial distribution characteristics of the FAST structure—large span, high location, numerous components, and complex structure—and taking into account multi-source data acquisition types, local climate conditions, UAV flight altitude limitations, and safe flight time limits, a UAV inspection system flight path plan for structural anomaly data acquisition is formulated. Taking the flight acquisition of reflector units and cable net nodes as an example, the inspection path planning scheme for reflector node disks is as follows: Starting from the center of the reflector, the plan is to radially expand outwards. Beginning with the first ring, the system will first inspect one ring of node disks in a circular pattern, then extend radially outwards to the second ring, then to the third, fourth, and so on, with each ring increasing in a radial pattern. The distance for acquiring reflector data is approximately 1-2 meters, the flight altitude is generally about 2 meters above the reflector, and the flight area is generally above the reflector. The UAV will acquire node disk data at a 45° downward angle. The safe flight timeframe is generally 25 minutes before returning to base for battery replacement. The extended measuring base serves as the drone's takeoff and landing platform. When battery replacement or flight adjustments are needed, the measuring base can be used as a platform for drone deployment and retrieval. After each flight inspection, the drone returns to the operating platform via the shortest path for recovery, achieving data acquisition from the reflective surface. Multi-source data acquisition types include visible light video image data and infrared video image data.

[0070] S2.1 Determining the UAV Flight Path: Taking the flight data acquisition of the reflector unit and cable net nodes as an example, the following path planning scheme is proposed for the inspection of the reflector node disks: Starting from the center of the reflector, the path will radially expand outwards. Beginning with the first ring, the path will first inspect one ring of node disks in a circular pattern, then extend radially outwards to inspect the second ring of node disks, then to the third ring, the fourth ring, and so on, with each ring increasing in a radial inspection pattern. The measuring base extending from the reflector will serve as the UAV's take-off and landing platform. When the UAV battery needs to be replaced or flight adjustments need to be made, the measuring base can be used as the operating platform for launching and recovering the UAV. After each flight inspection, the UAV will return to the operating platform along the shortest path for retrieval, thus achieving reflector data acquisition.

[0071] In step S2.2, during sensor acquisition, visible light and infrared camera sensors are used to acquire video image data. The distance between the sensor acquisition drone and the current detection node is generally controlled at 1-2 meters. The flight altitude is generally about 2 meters above the reflective surface, and the flight area is generally above the reflective surface. The drone acquires node disk data by flying downwards at a 45° angle.

[0072] Step S2.3 Determine the UAV flight area: This invention is based on the development of this technology and is not a large-scale conventional structural inspection. Instead, it focuses on the research and development and implementation of intelligent UAV inspection technology for several types of structural damage, such as broken unit members and holes in panels. After the completion of this technology development, it will be applied on a large scale to the structural inspection work of FAST. Structural anomalies such as broken unit members, holes in panels, loose ring beam bolts, and structural corrosion are relatively common anomalies in telescope structures. This invention achieves a breakthrough in intelligent inspection technology for the above four types of problems.

[0073] Step S2.4 determines the abnormal data acquisition flight plan, which includes data acquisition of structural parts that may have structural anomalies such as unit member fracture, panel hole, ring beam bolt detachment, and structural corrosion. The data is collected in video and image format, and then intelligent identification is performed on the computer algorithm software to achieve intelligent identification of structural anomalies.

[0074] Step S3, as follows Figure 2The diagram shows the framework for multi-source data spatial registration. Multi-source data spatial registration is the foundation for structural anomaly detection in the FAST structure unmanned inspection system based on multi-source data fusion. To obtain more FAST structure texture and spectral information, the original information after infrared imaging correction will be used as much as possible for anomaly target information extraction. In addition, the dynamic range of visible grayscale in the original image data is narrow, so enhancement stretching and other work are required using a certain standard. After completing the above work, geometric registration should be performed on the data from each data source. During geometric registration correction, the image with low spatial resolution should be registered to the image with high spatial resolution, then classified and processed, and finally corrected to the visible light image actually used, in preparation for the next step of fusion and digital image mosaicking. In step S3, before detecting anomalies in the FAST structure based on multi-source fusion, since the acquired visible light image and the corresponding infrared image do not match perfectly, registration studies of the two-light images are required first. Feature-based registration methods focus on obtaining relatively stable features in the image, such as corners and edges, and then use these feature information for registration. Feature-based registration uses a distance function as a matching criterion to calculate and find the corresponding features. Distance criteria typically include Euclidean distance, Mahalanobis distance, and Hausdorff distance. To improve registration accuracy, a coarse-to-fine registration process is usually adopted: first, coarse registration compares the similarity of features using distance criteria to find corresponding features and establish a rough set of matching point pairs; second, fine registration uses algorithms to filter the set of registration point pairs to obtain a more accurate set of matching point pairs, and the RANSAC algorithm is used for fine registration.

[0075] S3.1 Infrared and visible light image data are acquired for areas in the FAST structure where there may be anomalous targets.

[0076] S3.2 uses a feature linear mapping image registration method to map the infrared image corresponding to the visible light image to the visible light image scale, obtaining a registered visible-infrared image pair. Specifically, the infrared image undergoes a linear transformation of grayscale values.

[0077] S3.3 Combine the image information extracted from the abnormal target features of the visible light image with geometric registration correction.

[0078] The image after S3.4 geometric registration and correction is combined with the visible light image again for image mapping, and finally the registered infrared image is obtained.

[0079] In step S4, the LabelImg data annotation tool is used to annotate the visible light images of the anomalous targets, which are labeled as panel structural defects, corrosion, screw loss and rod loss, respectively. The annotated anomalous target labels and the visible-infrared image pairs form a multi-source FAST structural anomalous target dataset.

[0080] Step S5, as follows Figure 3 The diagram shown illustrates the data preprocessing framework. The multi-source fusion anomaly detection model is the core tool for structural anomaly identification, achieving a higher anomaly detection rate. It is based on initializing network parameters using a multi-source fusion FAST defect detection network. The detection network model is trained using the amplified multi-source FAST structural anomaly target training set to achieve the desired anomaly detection rate. The multi-source FAST structural anomaly target dataset consists of one-to-one anomaly target labels and visible-infrared image pairs. To improve the detection accuracy and robustness of the anomaly target detection algorithm model, preprocessing of the anomaly target dataset is necessary. This includes:

[0081] S5.1 Based on the category of abnormal targets, the abnormal target dataset is randomly grouped into training set, test set and validation set in a ratio of 8:1:1;

[0082] S5.2 performs data augmentation operations on abnormal target data by performing image flipping, random offsetting of abnormal targets, image cropping, adding motion blur and white noise.

[0083] The entire dataset was randomly grouped according to the category of anomalous targets, into training, testing, and validation sets in an 8:1:1 ratio. Due to the reasonable structural design, good usage, and timely maintenance of the FAST telescope, the actual number of anomalous target images acquired was relatively small. Therefore, data augmentation operations were needed, including image flipping, random anomalous target offsetting, image cropping, and the addition of motion blur and white noise. Furthermore, the actual background environment was quite complex, so some negative samples were appropriately added to improve the detection accuracy of the model.

[0084] Step S6, as follows Figure 4 The diagram shown is a framework diagram of a multi-source fusion abnormal target detection model.

[0085] The multi-source fusion anomaly target detection model is as follows:

[0086] S6.1 initializes the detection model network parameters based on the dataset in S5;

[0087] S6.2 Input the amplified multi-source FAST structural anomaly target training set into the detection model to train the detection model; if the expected results are achieved, a multi-source fusion anomaly target detection network model is formed.

[0088] S6.3 Input the amplified multi-source FAST structural anomaly target validation set for validation;

[0089] S6.4 If the expected results are not achieved, adjust the network parameters and continue training the detection network model until the expected results are achieved, forming an effective multi-source fusion anomaly target detection model.

[0090] like Figure 5 The diagram shown is a flowchart of the training process for a multi-source fusion anomaly detection model. The training process for the multi-source fusion anomaly detection model is as follows:

[0091] 1) Set initial training parameters;

[0092] 2) Randomly collect a smaller dataset from the original training set;

[0093] 3) Adjust the training rounds to conduct small-sample training and obtain preliminary detection results for classification and regression losses at the county level;

[0094] 4) Determine if there are any issues with the dataset design. If there are no issues, use all training sets for training. If there are issues, return to step 2) and randomly collect data from the dataset.

[0095] 5) Set Dropout to 0.5, start formal training using the initial training parameters, and set the batch size to 4;

[0096] 6) Determine whether there is overfitting or underfitting based on the loss curve, and then decide whether to increase the training accuracy;

[0097] 7) Analyze the loss curves of model classification and regression and the defect detection results, and then optimize the analysis results.

[0098] This invention comprehensively considers factors such as the detection and application effects of inspection equipment, equipment technical indicators, and system integration. Based on the inspection business needs, actual operational conditions, airspace management regulations, and equipment procurement, the configuration of unmanned related operational equipment is optimized to reduce costs, increase efficiency, and maximize resource utilization. According to the distribution characteristics of the unmanned inspection route on the FAST structure (upper and lower reflector surfaces, feed towers, etc.), considering factors such as unmanned live-line operation performance, take-off and landing flexibility, flight stability, hovering, payload, and loading and unloading of inspection equipment, combined with the main characteristics of the inspection area, an optimization method for unmanned inspection plans is studied. Reasonable classification and optimization research are conducted to optimize inspection efficiency.

[0099] During the route inspection planning, since the center of the FAST structure is an irregular trough, the flight path of the UAV needs to be planned in combination with the structural characteristics. The FAST UAV structural inspection focuses on the FAST reflector panel.

[0100] The active reflector system includes a cable net, ring beam, actuator, reflector unit, windbreak wall, health monitoring and other components. It forms a 300m diameter instantaneous parabolic surface in the direction of the radio source, focusing the electromagnetic waves radiated by celestial bodies to the focal point for astronomical observation.

[0101] Based on the cable net's partitioning, it is divided into 5 identical areas, as shown in the diagram below. Figure 6 As shown, each node in the cable net connects to 6 main cables, 6 reflector elements, and 1 pull cable, transmitting the actuator's displacement control of the pull cable to the cable net nodes, as follows: Figure 6-10 As shown. The main cable net surface is usually a sphere with a radius of curvature of 300.4 meters. However, when the FAST telescope is working, the 300-meter diameter portion of the main cable net surface can be actively deformed from a neutral sphere into a paraboloid of revolution, and the position of the paraboloid can be continuously moved.

[0102] FAST UAV flight line inspection planning: Since the reflector unit and the cable net partition are consistent, both being 1 / 5 centrally symmetrical, that is, according to the cable net partition, the reflector is divided into 5 identical regions, such as... Figure 6 As shown, one-fifth of the reflective surface, Figure 7 Flight path scheme for UAV inspection of reflector node disks.

[0103] Considering the characteristics of drones, such as endurance, return along the original route, proximity alarm, and high efficiency of cruising at the same latitude, and combined with the drone's performance, a specific path planning scheme for node panel inspection is proposed: Starting from the center of the first ring of the reflector surface, the drone will first inspect one ring of the node panel in a circular pattern, then expand to the second ring, then the third, fourth, and so on, increasing the inspection path with each ring. When the number of inspection rings increases, a measuring base can be used as an operating platform for drone deployment and retrieval. A schematic diagram of the drone's reflector surface inspection path planning is shown below. Figure 7 , Figure 8 As shown.

[0104] Anomaly detection is performed using the deep learning network YOLOv5.

[0105] (I) Feature Extraction Network

[0106] CSPDarknet is a convolutional neural network that aggregates and forms image features at different fine-grained levels, serving as the feature extraction network for YOLOv5. CSPDarknet solves the problem of repetitive gradient information in the feature extraction network optimization of other large convolutional neural network frameworks by integrating gradient changes from beginning to end into the feature map. This reduces the number of model parameters, ensuring both inference speed and accuracy while reducing model size.

[0107] (II) Feature Enhancement Network

[0108] Feature augmentation networks pass image features to the prediction layer through a series of pre-designed network layers. They are primarily used to generate feature pyramids to enhance the model's ability to detect targets at different scaling scales, thus enabling the detection of the same target at various sizes and scales.

[0109] (III) Detection Network

[0110] The detection network predicts the category and location of objects based on image features. Primarily used in the final detection section, the network applies anchor boxes to the feature map and generates a final output vector containing class probabilities, object scores, and bounding boxes.

[0111] (iv) Loss Function Setting

[0112] For the FAST structural anomaly target detection task, a loss function needs to be set to adjust the model's convergence direction. Since there are many small targets in the detection task, GIoU Loss is used as supervision during training. Assume the coordinates of the predicted target's bounding box and the actual target's labeled bounding box are respectively... and in

[0113] B P B is the bounding box coordinate of the predicted target. g These are the coordinates of the actual target's bounding box. (x1, y1) and (x2, y2) are the coordinate points.

[0114] The specific process for calculating GIoU Loss is as follows:

[0115] (1) The areas of the actual target bounding box and the predicted target bounding box are calculated as follows, where A g A represents the area of ​​the actual target bounding box. p To predict the area of ​​the target bounding box.

[0116]

[0117]

[0118] A g A represents the area of ​​the actual target bounding box. p To predict the area of ​​the target bounding box, (x1, y1) and (x2, y2) are the coordinates of the points.

[0119] (2) Calculate the overlap area I between the predicted target bounding box and the actual target annotation box.

[0120]

[0121] I represents the overlap area between the predicted target bounding box and the actual target annotation box. (x1, y1) and (x2, y2) are the coordinates of the target bounding box and the actual target annotation box, respectively.

[0122] (3) Find the bounding box B that can contain the predicted target. P And the actual target annotation box Bg Minimum bounding box B c And calculate B c Area A c .

[0123]

[0124] B P It predicts the target bounding box. (B) g This is the actual target annotation box. (B) c It is the smallest bounding box, and its area is represented by A. c express.

[0125] (4) Calculate the intersection-union ratio (IoU) between the predicted target bounding box and the actual target labeled box.

[0126]

[0127] IoU represents the intersection-union ratio. A g A represents the area of ​​the actual target bounding box. p To predict the area of ​​the target bounding box.

[0128] (5) Calculate the global intersection-union ratio (GIoU) ​​between the predicted target bounding box and the actual target labeled box.

[0129]

[0130] GIoU represents the global intersection-union ratio. A g A represents the area of ​​the actual target bounding box. p To predict the area of ​​the target bounding box.

[0131] (6) Calculate the loss function L of GIoU Loss GIoU .

[0132] L GIoU =1-GIoU

[0133] IoU represents the intersection-over-union ratio. GIoU represents the global intersection-over-union ratio.

[0134] Infrared-visible fusion detection algorithm

[0135] To address the multi-source fusion problem, this invention proposes a cross-modal multi-source fusion detection algorithm, GCANet, based on a cross-attention mechanism. This part corresponds to which step S6?

[0136] (I) Overview of Methods

[0137] GCANet's network overview is as follows: Figure 9As shown, these are associated with feature extraction, multi-source data feature fusion, and FAST structural anomaly target detection tasks, respectively. Infrared-visible image pairs are used as input. First, the feature extraction network is fed into the network. This network consists of a deep convolutional network trained with parameter sharing, extracting visible light and near-infrared depth features separately. The extracted visible light and near-infrared depth features are then input into a Gaussian transformer encoder network based on an attention module. This network takes paired multi-source data features as input and enhances and fuses the salient features based on a learnable 2D Gaussian cross-attention mechanism, outputting a high-quality feature map containing significant local information and global distribution information of multi-source targets. The enhanced feature map is then fed into a classification network to obtain detection results with target categories, along with corresponding bounding boxes and confidence scores.

[0138] (II) Feature Extraction Network

[0139] The purpose of feature extraction networks is to extract features from visible light and infrared images separately through parameter sharing. This invention uses the ResNet50 network as the feature extraction network.

[0140] (III) Gaussian Transformer Encoder Network

[0141] The Gaussian transformer encoder network is a transformer model that enhances the representation of target saliency features by fusing deep features. To learn multi-source fused feature representations with lightweight computational and memory capabilities, this invention introduces a Gaussian cross-attention mechanism module to fuse target saliency depth features from infrared and visible light features, such as... Figure 10 As shown.

[0142] (iv) Classification and Positioning Network

[0143] For the classification and localization network, this invention designs a network consisting of two parallel branches: a classification branch and a regression branch. Different convolutional blocks are also set in the two branches. There are two blocks in the classification branch and four blocks in the regression branch. Each block contains a convolutional layer, a batch normalization layer, and a ReLU layer. Furthermore, this invention adds an implicit prediction of the object probability for each regression branch; the final classification result is multiplied by this implicit prediction.

[0144] (V) Loss Function

[0145] After training, GCANet can capture local and long-range dependencies, generate clearer content, and retain most of the visual information from multi-source data features. To more accurately detect anomalous targets in the FAST structure, this invention trains GCANet by minimizing the loss function, with the total loss L as follows:

[0146] L = L cls +λLreg

[0147] L consists of a weighted combination of classification loss and bounding box loss, where the classification loss is Focal loss and the bounding box loss is GIoU Loss.

[0148] The specific implementation of the FAST structural anomaly target detection experiment is as follows:

[0149] For the evaluation of abnormal inspections, appropriate image samples are selected with reference to actual sample anomalies. Recognition accuracy and running time are used as the main evaluation indicators. Based on the panel anomaly dent database, the effectiveness of the proposed FAST structural anomaly target detection method is experimentally verified for four types of abnormal targets: panel anomalies, metal corrosion, screw loosening, and rod breakage.

[0150] The specific evaluation metrics for FAST structural anomaly target detection are as follows:

[0151] To objectively evaluate the model's detection performance, two general evaluation metrics, recognition accuracy and single-frame detection time, are used as evaluation metrics for FAST structural anomaly target detection.

[0152] Recognition accuracy

[0153] The accuracy rate represents the proportion of correctly identified abnormal targets of a certain type out of all actually labeled abnormal targets of a certain building. Figure 11 It displays all possibilities in the detection results, where correctly predicted examples are positive examples and incorrectly predicted examples are negative examples. TP (True Positive Examples) represents the number of samples where both the prediction and the actual target are of this type of anomalous target. FP (False Positive Examples) represents the number of samples where the prediction is of this type of anomalous target but the actual target is not. TN (True Negative Examples) represents the number of samples where neither the prediction nor the actual target is of this type of anomalous target. FN (False Negative Examples) represents the number of samples where the prediction is not of this type of anomalous target but the actual target is.

[0154]

[0155] The formula for calculating recognition accuracy is as follows:

[0156]

[0157] Single frame detection time

[0158] Single-frame detection time is an important metric for evaluating a model's detection speed. It represents the time required for the model to detect each frame of an image; a shorter detection time indicates a faster detection speed. The formula for calculating single-frame detection time is as follows:

[0159]

[0160] Analysis of FAST Abnormal Target Detection Results

[0161] For anomaly detection and evaluation, this invention refers to actual sample anomalies, selects appropriate image samples, and uses recognition accuracy and running time as the main evaluation indicators. Based on the panel anomaly dent database, steel structure corrosion database, screw detachment database, and rod detachment database, the effectiveness of the proposed FAST structural anomaly target detection method for four types of anomalies—panel anomalies, metal corrosion, screw detachment, and rod fracture—is experimentally verified.

[0162] Panel abnormal dent database detection results

[0163] FAST panel anomaly data is characterized by small targets, high similarity, and significant differences between different shooting angles. Based on the YOLOv5 algorithm, a method specifically designed for small target detection was developed, such as data mosaic enhancement. Furthermore, to address the issue of significant differences between different shooting angles, a targeted solution was implemented in the flight plan design. During data acquisition, shooting at pitch angles of 15 degrees and -30 degrees yielded the best results.

[0164] Based on the characteristics of the panel anomaly database, the network parameters were set as follows: 150 training epochs, batch size of 8, initial learning rate of 0.01, final learning rate of 0.001, SGD (Stochastic Gradient Descent) optimization algorithm was used to iteratively update the weights during training, the model training momentum was 0.9, the IoU training threshold was 0.2, and image enhancement methods included mosaic enhancement, HSV-hue enhancement, HSV-saturation enhancement, HSV value enhancement, image translation, image scaling, and image flipping.

[0165] The detection results of the FAST structural anomaly target detection algorithm on the panel anomaly indentation database are shown in the table below.

[0166] Table 1. Detection results of the panel abnormality indentation database.

[0167]

[0168] The validation data consisted of 100 images, containing a total of 563 panel concave anomalies. The FAST structural anomaly detection model correctly detected 482 panel concaves, achieving an accuracy rate of 85.6%, with an average detection time of 0.07 seconds per image.

[0169] Examples of some detection image results are as follows: Figure 11 and Figure 12 As shown.

[0170] Figure 11and Figure 12 These are marks identified by intelligent recognition software based on the pits in the reflective surface unit panel. In other words, they are the areas identified by intelligent recognition algorithm software after pits or holes appear on the reflective surface unit panel.

[0171] The FAST unmanned aerial vehicle (UAV) intelligent inspection method and system provided by this invention have the following advantages:

[0172] (1) Remote control, high safety factor, no personal safety accidents will occur;

[0173] (2) It has a wide range of applications and can be used to inspect reflector surfaces, feed towers, surrounding mountains, etc.

[0174] (3) Less affected by terrain and weather;

[0175] (4) High detection accuracy, no blind spots, and can capture more details at close range;

[0176] (5) It can not only check external faults, but also detect hidden dangers that are difficult to be found by manual inspection, such as missing pins and cracked nuts through refined inspection.

[0177] (6) Low labor intensity, high work efficiency, and reduced risk of manual climbing.

[0178] This invention, based on the need for safe and efficient inspection of large astronomical equipment structures, is the first to apply UAV inspection technology to the inspection of astronomical equipment structures, significantly improving the inspection efficiency of the FAST telescope structure. This invention aims to explore the application of an intelligent UAV inspection system to the safe and efficient inspection of FAST structures (upper and lower parts of the reflector surface, feed tower, etc.), enabling efficient screening of faults such as contamination, deformation, corrosion, and jamming, achieving early detection and early maintenance, and realizing a high degree of harmony between detection and maintenance. This is of great significance for improving the reliability, robustness, and safe and reliable operation of FAST.

[0179] Any process or method described in the flowcharts of this invention or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, achievable on any computer-readable medium for use by an instruction execution system, apparatus, or device. The computer-readable medium can be any medium containing a program for storage, communication, propagation, or transmission for use by an execution system, apparatus, or device, including read-only memory, magnetic disks, or optical disks.

[0180] In the description of this specification, references to terms such as "embodiment," "example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. The illustrative expressions of the above terms in this specification do not necessarily refer to the same embodiment or example. Furthermore, those skilled in the art can combine or combine the different embodiments or examples described in this specification and their features therein without causing contradiction.

[0181] While embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and alterations to the above embodiments within the scope of the present invention.

Claims

1. An unmanned aerial vehicle intelligent inspection method based on FAST structure anomaly target identification, characterized in that, The method comprises the following steps: S1, establishing a UAV platform: according to the UAV inspection reflector path, a plurality of operation platforms for launching and recovering the UAV are arranged; S2, combining the multi-source data acquisition type, setting the UAV flight plan; S3, collecting infrared images and visible light images for the FAST structure abnormal target; Using a characteristic linear mapping image registration method, the infrared image corresponding to the visible light image is mapped to the visible light image scale to obtain a registered visible-infrared image pair; S4, labeling the visible light image of the abnormal target data, respectively labeled as panel structure defect, rust, screw falling and / or rod falling, and the labeled abnormal target label and the visible-infrared image pair constitute a multi-source FAST structure abnormal target data set; S5, preprocessing the data in the abnormal target data set to form an amplified data set; S6, based on the data set in S5, according to the multi-source fusion abnormal target detection model, the FAST structure abnormal target is effectively identified and verified; The multi-source fusion abnormal target detection model is GCANet network, which is associated with feature extraction, multi-source data feature fusion and FAST structure abnormal target detection task, and takes infrared-visible light image pair as input; it fuses visible light and infrared depth features through a Gaussian variable pressure encoder network based on cross attention mechanism; The Gaussian variable pressure encoder network takes paired multi-source data features as input, enhances and fuses the multi-source significant features based on a learnable two-dimensional Gaussian cross attention mechanism, and outputs a feature map containing significant local information and multi-source target global distribution information.

2. The unmanned aerial vehicle intelligent inspection method based on FAST structure anomaly target identification according to claim 1, characterized in that, The method for setting the UAV inspection reflector path is as follows: a plurality of inspection circles are arranged on the reflector from the inside to the outside, starting from the first circle at the center of the reflector, first circling the nodes along the first circle, then expanding to the second circle, and then expanding the inspection circle outward in turn, and the inspection path is increased circle by circle. 3.The unmanned aerial vehicle intelligent inspection method based on FAST structure anomaly target identification of claim 2, characterized in that, When the inspection circle radius increases to exceed the endurance mileage of the UAV, the operation platform of the UAV is arranged on the outer FAST measurement pier, and the UAV is launched and recovered on the operation platform.

4. The unmanned aerial vehicle intelligent inspection method based on FAST structure anomaly target identification according to claim 1, characterized in that, According to the cable net partition, the reflector of the FAST is divided into five identical areas, and the UAV inspects the five areas respectively.

5. The unmanned aerial vehicle intelligent inspection method based on FAST structure anomaly target identification according to claim 1, characterized in that, In S2, the UAV flight plan is set as follows: according to the main abnormal type of the FAST structure and the spatial distribution of the FAST structure, the UAV flight route is determined; according to the collection distance of the sensor, the flight height limit and the safe flight time limit, the UAV flight area is determined; and then the UAV inspection system structure abnormal data acquisition flight plan is determined.

6. The unmanned aerial vehicle intelligent inspection method based on FAST structure anomaly target identification according to claim 1, characterized in that, In S3, the infrared image is subjected to linear gray value transformation, and the image information after extracting the abnormal target features of the visible light image is combined for geometric registration correction; the geometrically registered image is combined with the visible light image again for image mapping, and finally the registered infrared image is obtained.

7. The unmanned aerial vehicle intelligent inspection method based on FAST structure anomaly target identification according to claim 1, characterized in that, In S5, the data in the abnormal target data set is preprocessed as follows: S5.1 According to the category of the abnormal target, the abnormal target data set is randomly grouped, and the training set, the test set and the verification set are divided according to the ratio of 8:1:1; S5.2, the image flipping, abnormal target random offset, image cropping, increase the form of motion blur and white noise, data augmentation operation is carried out on the abnormal target data.

8. The unmanned aerial vehicle intelligent inspection method based on FAST structure anomaly target identification according to claim 7, characterized in that, In the S6, the multi-source fusion abnormal target detection model is specifically: S6.1, based on the data set in S5, the detection model parameters are initialized; S6.2, the augmented multi-source FAST structure abnormal target training set is input into the detection model, and the detection model is trained; S6.3, the augmented multi-source FAST structure abnormal target verification set is input for verification; S6.4, after reaching the expectation, the effective multi-source fusion abnormal target detection model is formed. 9.The method of claim 8, wherein, The training process of the multi-source fusion abnormal target detection model is specifically: 1) set the initial training parameters; 2) randomly collect a small data set from the original training set; 3) adjust the training rounds for small sample training, obtain the classification and regression loss county level preliminary detection result; 4) judge whether there is a data set design problem, if there is no problem, put all the training set into training, if there is a problem, return to the data set random collection step 2); 5) set the Dropout to 0.5, start formal training using the initial training parameters, and set the batch size to 4; 6) according to the loss curve, judge whether there is overfitting and underfitting problem, and then determine whether to increase the training precision; 7) analyze the loss curve and defect detection result of the model classification and regression, and then optimize according to the analysis result.

10. An unmanned aerial vehicle intelligent inspection system based on FAST structure anomaly target identification, characterized in that, The inspection system is used to implement the FAST unmanned aerial vehicle intelligent inspection method according to any one of claims 1-9.

Images