An aircraft deformation scanning detection device and rapid detection method

Abstract

The present application relates to the technical field of aircraft surface deformation detection, and particularly relates to an aircraft deformation scanning detection device and a rapid detection method, which detects an aircraft globally through a UAV, compares the photographed pictures with a database for the first time, extracts characteristic values as search thresholds, obtains a preliminary result, filters out defect pictures, combines an improved cuckoo search algorithm with a SLAM algorithm, accurately plans a path suitable for secondary accurate detection of the UAV, makes the UAV detect locally at the fastest efficiency within a specified time, performs depth analysis on the secondarily acquired pictures, compares the result with standard component size parameters in a non-deformation aircraft template database, and obtains specific damage types and damage degrees. The present application reduces the overall time delay of the system and improves the utilization rate of network resources through cloud edge collaboration.

Description

Technical Field

[0001] This invention relates to the field of aircraft surface deformation detection technology, specifically to an aircraft deformation scanning detection device and a rapid detection method. Background Technology

[0002] Aircraft skin, fairings, and panels are prone to dents and deformations under various conditions, including but not limited to the following:

[0003] A. During takeoff and landing, the aircraft is struck by foreign objects such as birds;

[0004] B. During ground maintenance of the aircraft, improper operation, such as accidental collision of tools, may occur.

[0005] C. When an aircraft is overloaded or flies abnormally at low altitude, it is easy for the fuselage to deform during takeoff and landing.

[0006] D. Plastic instability caused by improper stress.

[0007] Dents and deformations in an aircraft can increase drag and affect its flight performance. In certain critical areas, such as stringers and flanges, dents and deformations are not permitted.

[0008] Currently, the inspection of dents and deformations on aircraft relies heavily on visual inspection by ground crew personnel. This inspection method has, but is not limited to, the following limitations:

[0009] A. The human eye has limited precision, making it difficult to detect some dents and deformations;

[0010] B. Inspectors are prone to fatigue, forgetting to perform inspections, etc.

[0011] With the development of multi-rotor drone technology, both the consumer and industrial drone markets have experienced rapid growth in recent years. The cost of using drones has decreased significantly, and more and more fields are beginning to use multi-rotor drones for inspection, a highly efficient and convenient method. Currently, drone visual SLAM collaborative mapping and navigation technology is developing rapidly, and the relatively simple environment of aircraft maintenance aprons makes drone inspection a suitable application of this technology.

[0012] The Cuckoo Search (CS) algorithm is based on the brood parasitism of cuckoos, which results in fewer parameters. This means it doesn't need to re-match a large number of parameters when solving specific problems, thus outperforming other emerging metaheuristic intelligent optimization algorithms in solving many optimization problems. Furthermore, the algorithm can be enhanced using Levy flights instead of simple isotropic random walks.

[0013] Studies have shown that this algorithm may be more efficient than genetic algorithms, PSO, and other algorithms.

[0014] However, the Cuckoo Search algorithm is currently only used to test standard test problems where the theoretical optimal solution is known. Its initial convergence speed is slightly lacking. When the algorithm is applied to practical engineering problems with objective functions that have high computational costs, the slow initial convergence speed will not be conducive to quickly locating the approximate area of ​​the global optimal solution. Therefore, it has certain shortcomings in the inspection of pits and deformations on the surface of aircraft.

[0015] Therefore, this application designs an improved Cuckoo Search algorithm (MCS), theoretically elucidating the algorithm improvement ideas and steps, and demonstrating the superiority of the improved algorithm through simulation experiments. During flight line maintenance, the improved Cuckoo Search algorithm and SLAM technology can be combined. By extracting features from the images collected during the first rapid inspection of the UAV, comparing the damage degree and maintenance time, several damage points that the UAV needs to scan in detail are connected to form a specific inspection path, which greatly improves the detection efficiency and allows for optimal detection. Summary of the Invention

[0016] The purpose of this invention is to fill the gap in existing technology by providing an aircraft deformation scanning and detection device and a rapid detection method. Through an improved Cuckoo Search algorithm (MCS), the invention theoretically elucidates the algorithm improvement ideas and steps, and demonstrates the superiority of the improved algorithm through simulation experiments. In line maintenance, the improved Cuckoo Search algorithm and SLAM technology can be combined. By extracting features from images collected during the initial rapid inspection of the UAV, comparing the degree of damage with maintenance time, several damage points requiring detailed scanning by the UAV can be linked together to form a specific inspection path, greatly improving detection efficiency and enabling optimal detection.

[0017] To achieve the above objectives, this invention provides an aircraft deformation scanning and detection device, including a detection system and a data processing and control box. The detection system includes a UAV detection system, a cloud receiving module, and a deformation analysis system. The UAV detection system includes a UAV, a high-definition camera, an industrial camera, a lens, and a camera controller. Under the control of the camera controller, the industrial camera collects structured light reflected from the fuselage through the lens. The deformation analysis system includes a laser and industrial camera synchronization system, a laser and industrial camera position detection module, a non-deformable aircraft template database, and a data comparison system. The high-definition camera, industrial camera, lens, and camera controller are all mounted on the UAV. The data processing and control box queries the non-deformable aircraft template database and performs comparisons through the data comparison system to determine whether the aircraft is deformed.

[0018] The drone transmits detection information to the gimbal, which then processes the information using the Cuckoo Search algorithm for positioning before transmitting it back to the drone for a second point-to-point scan.

[0019] The deformation analysis system first compares images captured by the drone with a database to extract feature values ​​as search thresholds and obtain preliminary results. Defective images are then filtered out and uploaded to the cloud. The Cuckoo algorithm is used to accurately detect the locations that need to be accurately detected a second time within a specific period. Instructions are then issued to perform in-depth analysis on the secondary images acquired by the drone. The results are compared with a standard database in the cloud to determine the specific type and degree of damage.

[0020] A rapid detection method for aircraft deformation scanning includes the following steps:

[0021] S1, using a drone to scan the entire surface of the machine;

[0022] S2, Image data acquisition and data augmentation for training;

[0023] S3: Image data analysis is performed in the cloud;

[0024] S4, Image Defect Feature Extraction and Comparison, UAV Secondary Scan Path Planning;

[0025] S5, the drone scans specific locations based on the calculation results;

[0026] S6, the drone transmits the detection results to the cloud for damage assessment;

[0027] S1 also includes:

[0028] S10, pre-programs flight paths according to aircraft type;

[0029] S11, trajectory calibration;

[0030] S11 also includes:

[0031] S11-1, Visual navigation is used if the correct location image information is captured in advance at each positioning point; S11-2, The real-time location image is compared and analyzed with the correct location image.

[0032] S11-3 feeds the comparison results back to the flight control system for position calibration;

[0033] S2 also includes:

[0034] S21, Image data acquisition for training;

[0035] S22, Defect labeling;

[0036] S23, Secondary defect path detection path planning;

[0037] S22 also includes:

[0038] S22-1, Open an image that needs to be calibrated;

[0039] S22-2, classify and annotate the defects of the calibrated images;

[0040] S22-3, determine whether the drone is carrying the corresponding detection probe;

[0041] S22-4, Locate the coordinates of the defect;

[0042] S23 also includes:

[0043] S23-1, Locate the coordinates of the defects in the calibration image;

[0044] S23-2 uses an algorithm to edit all coordinates into the most suitable automatic maintenance path based on maintenance time and actual drone accessibility;

[0045] There are three main methods of data augmentation in S2:

[0046] The original image is rotated by 90 degrees, 180 degrees, and 270 degrees, and then appropriately shrunk inward and expanded outward to generate a new image.

[0047] Use a 300*300 pixel sliding window to crop the image into several blocks;

[0048] Oversampling and detail duplication involve artificially duplicating flawed images and training the system multiple times with these flawed images.

[0049] There are three main methods of path planning in S4:

[0050] The step size can be variable during flight and sudden 90° turns can be made. Occasionally large step sizes can ensure that the search will not get trapped in local optima.

[0051] In CCS, chaos theory is incorporated into the Cuckoo Search technique. Chaos theory studies the behavior of highly sensitive systems, where small changes in the initial position can have a significant impact on the system's behavior. Chaos has the characteristics of non-repetition and ergodicity, which facilitates fast searching.

[0052] The concept of elites from genetic algorithms is introduced, and the best cuckoo is substituted into the next generation to construct a new and better solution.

[0053] The following is a way to construct a new, better solution:

[0054] A. Randomly generate n "bird's nest" locations in the search space, i.e., damage point locations x = (x1, x2, ..., xn).d ) T They are tested, and based on the test results, the initial globally optimal "bird's nest" location is selected and this "bird's nest" location is retained for the next generation;

[0055] B. Utilization Let p be the position of the i-th bird's nest in generation t; the probability that the drone can detect the damage and that the damage urgently needs repair is p. a p a ∈[0,1], in this case, the drone can either ignore the damage or mark the location of the damage; update the other "bird nest" locations, and then test the new set of "bird nest" locations obtained after the update, compare them with the previous generation of bird nest locations, select the location with the better test value among the corresponding "bird nest" locations and keep it for the next step of calculation;

[0056] and These are two different random sequences, H(u) is the Heaviside function, ε is a random number taken from a random distribution, and s is the step size;

[0057]

[0058] Essentially, it is a stochastic equation of a stochastic process. In general, it is a Markov chain. Its purpose is to avoid the system from getting trapped in local optima by using certain far-field stochastic processing to obtain the effective value of the new solution. Γ(λ) is the standard gamma function.

[0059] This indicates point-to-point multiplication, where α>0 is the step size scaling factor, which is related to the degree of interest in the problem. In most cases, α = o(L / 10) is taken, where L is the characteristic range of the interest in the problem. However, in some cases, α = o(L / 100) is more effective and can avoid flying too far.

[0060] A random number r∈(0,1) following a uniform distribution is generated. The probability p of the damage location exceeding the damage tolerance is set in the algorithm. a = Compare f(t) with the random number; if r > p a Then change randomly The value, conversely if r≤p a ,but The value remains unchanged;

[0061] C. After this process is completed, test the changed damage location. Based on the test results, compare them with the test values ​​of the set of damage locations obtained after the update in step B. Select the better damage location from the corresponding damage locations and retain it. In this way, the optimal global location of a set of globally optimal damage location sequences for the current model can be selected.

[0062] D. Judgment It is the globally optimal solution. The step is to check if the corresponding global optimum satisfies the algorithm termination condition set for the problem to be optimized. If not, return to step B to continue the calculation; if so, ... This refers to the globally optimal damage location gb, where g is an abbreviation for global.

[0063] This invention utilizes a collaborative approach between the cloud and edge to process terminal data, known as cloud-edge collaboration. While the cloud possesses powerful computing capabilities, its distance from the terminal results in data transmission latency, failing to meet real-time requirements. The edge, located in the industrial environment, offers faster data transmission, but its task processing capabilities are limited. The cloud-edge collaboration system leverages the advantages of both, assigning simple tasks from the terminal to the edge for processing, and delegating tasks more difficult to handle to the cloud. This reduces overall system latency and improves network resource utilization. Based on the characteristics of cloud-edge collaboration, this invention deploys corresponding deep learning networks on both the cloud and edge sides to collaboratively complete defect detection tasks.

[0064] Compared with existing technologies, the superiority of the improved algorithm is demonstrated through simulation experiments. During flight line maintenance, the improved cuckoo search algorithm and SLAM technology can be combined. By extracting features from the images collected during the first rapid inspection of the UAV, comparing the degree of damage and maintenance time, several damage points that need to be scanned in detail by the UAV can be connected to form a specific inspection path, which greatly improves the detection efficiency and allows for optimal detection. Attached Figure Description

[0065] Figure 1 This is a structural diagram of the UAV surface defect detection system according to an embodiment of the present invention;

[0066] Figure 2 This is a system flowchart of a preferred embodiment of the present invention; Detailed Implementation

[0067] The present invention will now be further described with reference to the accompanying drawings.

[0068] See Figures 1-2This invention discloses an aircraft deformation scanning detection device and a rapid detection method, including a cloud platform, an edge platform, and a terminal. The edge platform deploys a lightweight small neural network, while the cloud platform deploys a large neural network. The edge platform first detects the images collected by the terminal to obtain preliminary detection results and filters out defective images. Then, the defective images are uploaded to the cloud platform, where the large neural network accurately detects the location and type of defects. Finally, a suitable UAV detection path is planned using a search engine.

[0069] like Figure 1 As shown, based on a drone inspection platform and focusing on hangar application scenarios, a drone surround inspection system was developed and implemented. The main components include:

[0070] S1 Intelligent Inspection System for Aircraft Fuselage Structure Based on Machine Vision;

[0071] S2 differentiated inspection paths and UAV flight plans are generated in real time;

[0072] Full-surface fault detection and visual localization for S3 aircraft;

[0073] S4 aircraft surface defect assessment and intelligent maintenance decision-making.

[0074] Furthermore, the S2 program can be divided into:

[0075] S21 aircraft surface environment modeling. The original 3D aircraft surface environment data is rasterized using the aircraft results generated by machine vision, and the discrete aircraft surface depth data is interpolated using the surface fitting method to obtain uniformly distributed regular grid data, thereby representing the 3D seabed environment.

[0076] S22 Population Initialization. Population initialization is closely related to path planning performance. There are two key issues in initializing the population at the bird's nest location: the method for generating the initial population and the initial population size. This paper uses a method that leverages prior knowledge to generate the population within the path planning region for initialization.

[0077] S23 Bird's Nest Location Encoding. Before performing the path planning task, the bird's nest locations in the MCS algorithm need to be encoded. This transforms the abstract problem to be solved into a concrete mathematical expression. A reasonable encoding method can facilitate the MCS algorithm to perform quickly and efficiently.

[0078] S24 Fitness Function Design. In UAV path planning, the design of the fitness function plays a crucial role, as it reflects the relative merits of individuals within the population. An appropriate fitness function design is essential for the MCS algorithm to find the globally optimal path within the planning region. This patent's fitness function combines an optimization index and a cost function. The optimization index reflects the path planning objective, while the cost function reflects the degree of individual maladaptation to the aircraft detection environment. In summary, the evaluation function used in this paper mainly includes two performance indicators: path length and threat cost.

[0079] S25 Bird Nest Location Update Method. The UAV path planning process is inherently a search process; therefore, bird nest location updates are crucial to all UAV path planning problems, and their performance directly impacts the effectiveness of UAV path planning. The MCS algorithm is used to update and optimize the path population. By incorporating information exchange between bird nest locations and setting the Levy flight step size α as a variable, the algorithm gains a larger search range and faster optimization speed, thereby obtaining the globally optimal path within the path planning area.

[0080] S26 Termination Criteria Setting. A termination criterion needs to be set during the UAV path planning process based on the actual situation.

[0081] Furthermore, S4 decision-making can be divided into:

[0082] S41 is a deep learning network for aircraft defect detection, threat assessment and localization. It trains a deep learning network based on typical defects and completes the tasks of aircraft defect detection, threat assessment and localization on a computing platform.

[0083] After completing numerical measurements, manual reviews, and other tasks, S42 sends comprehensive evaluation data to senior engineers to assist in decisions on whether to continue monitoring or perform immediate repairs, in order to maximize aircraft utilization under airworthiness standards, reduce maintenance hours, lower maintenance costs, and achieve optimal maintenance control of surface defects.

[0084] The key to S43 fuselage surface fault detection lies in determining the fault location and type through video images acquired by UAVs. Based on the target detection principle in machine vision, this paper studies image feature extraction methods for fuselage surface fault detection, compares and analyzes classic handcrafted features such as SIFT and HOG with automatically acquired features from deep learning, and constructs a feature library composed of multiple features.

[0085] S44 compares with CNN networks to study intelligent feature selection strategies, automatically determining image features suitable for fuselage surface fault analysis based on different aircraft models and inspection task requirements;

[0086] The S45 research focuses on target detection algorithms for locating and identifying faults on the fuselage surface, enabling real-time detection of various types of fault targets.

[0087] Based on machine learning and deep learning, high-resolution image recognition of key / suspected areas is performed to identify defects on the aircraft surface, enabling trend tracking of potential faults. This process can be divided into the following steps:

[0088] Visible light images are used to identify aircraft models (aircraft registration number associated with aircraft model). The identification results are used for path planning of unmanned aerial vehicles (UAVs) during aircraft inspection. Because different aircraft models have different sizes and areas requiring focused inspection, the flight path and flight plan (such as dwell time at specific locations) of the UAV are determined based on the identified aircraft model. This project specifically focuses on the flight path planning research using the B737 as an example.

[0089] Deep learning methods have demonstrated superior performance in image recognition and natural language processing. When using CNNs for high-resolution image classification with a fixed-size window traversal, the images are first classified using features obtained from the convolutional neural network for qualitative or quantitative analysis. Then, a support vector machine is used to reclassify the misclassified categories caused by insufficient feature discriminative power in the first classification step. Information fusion oriented towards detection task objectives can achieve "convexification" of the detection object in the feature space for different task objectives, improving the robustness of subsequent detection, recognition, and localization algorithms.

[0090] Based on the fusion of multi-source information, a deep learning network for aircraft surface defect inspection, defect statistics, and threat assessment is constructed using CNN as the foundation. The deep learning network is trained based on typical defects and completes aircraft defect detection, threat assessment, and localization tasks on a computing platform.

[0091] The above are merely preferred embodiments of the present invention, intended only to aid in understanding the method and core ideas of this application. The scope of protection of the present invention is not limited to the above embodiments; all technical solutions falling within the scope of the present invention's concept are within its protection. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

[0092] This invention addresses the shortcomings of existing technologies where the Cuckoo Search algorithm is only used for standard test problems with theoretically known optimal solutions. Its initial convergence speed is somewhat lacking, hindering the rapid localization of the approximate global optimum when applied to computationally expensive practical engineering problems. This limitation makes it unsuitable for inspecting dents and deformations on aircraft surfaces. This invention utilizes a UAV for global aircraft inspection, comparing captured images with a database to extract feature values ​​as search thresholds and filter out defective images. By combining an improved Cuckoo Search algorithm with SLAM, a precise path is planned for secondary, accurate UAV inspection, enabling the UAV to perform local inspection efficiently within a specified time. Deep analysis of the acquired images is then performed, comparing the results with standard component size parameters in a database of undeformed aircraft templates to determine the specific damage type and severity. Furthermore, this invention reduces overall system latency and improves network resource utilization through cloud-edge collaboration.

Claims

1. A rapid detection method for aircraft deformation scanning, characterized in that, Includes the following steps: S1, using a drone to scan the entire surface of the machine; S2, Image data acquisition and data augmentation for training; S3, The image undergoes data analysis in the cloud; S4, the defect features extracted from the image are compared to perform secondary scanning path planning for the UAV; S5, the drone scans specific locations based on the calculation results; S6, the drone transmits the detection results to the cloud for damage assessment; S1 further includes: S10, pre-programs flight paths according to aircraft type; S11, trajectory calibration; S11 further includes: S11-1, Visual navigation, if the correct location image information is captured at each positioning point in advance; S11-2, compare and analyze the real-time location image with the correct location image; S11-3 feeds the comparison results back to the flight control system for position calibration; S2 further includes: S21, the training image data is collected; S22, Defect labeling; S23, Secondary defect path detection path planning; S22 further includes: S22-1, Open an image that needs to be calibrated; S22-2, classify and annotate the defects of the calibrated images; S22-3, determine whether the drone is carrying the corresponding detection probe; S22-4, Locate the coordinates of the defect; S23 further includes: S23-1, Locate the coordinates of the defects in the calibration image; S23-2 uses an algorithm to edit all coordinates into the most suitable automatic maintenance path based on maintenance time and actual drone accessibility; There are three main methods of data augmentation in S2: The original image is rotated by 90 degrees, 180 degrees, and 270 degrees, and then appropriately shrunk inward and expanded outward to generate a new image. Use a 300*300 pixel sliding window to crop the image into several blocks; Oversampling and detail duplication involve artificially duplicating defective images and then training the program with these defective images multiple times. There are three main methods of path planning in S4: The step size can be variable during flight and sudden 90° turns can be made. Occasionally large step sizes can ensure that the search will not get trapped in local optima. In CCS, chaos theory is incorporated into the Cuckoo Search technique. Chaos theory studies the behavior of highly sensitive systems, where small changes in the initial position can have a significant impact on the system's behavior. Chaos has the characteristics of non-repetition and ergodicity, which facilitates fast searching. The concept of elites from genetic algorithms is introduced, and the best cuckoo is substituted into the next generation to construct a new and better solution; The method for constructing a new, better solution is as follows: A. Randomly generate n "bird's nest" locations in the search space, i.e., damage point locations. They are tested, and based on the test results, the initial globally optimal "bird's nest" location is selected and this "bird's nest" location is retained for the next generation; B. Utilization Update the other "bird nest" locations, then test the updated set of new "bird nest" locations, compare them with the previous generation of bird nest locations, select the location with the better test value among the corresponding "bird nest" locations, and keep it for the next step of calculation; The and They are two different random sequences. It is the Heaviside function. It is a random number taken from a random distribution. It is the step size; ； ， ， ； The This represents point-to-point multiplication. This is the step size scaling factor, which is related to the degree of stake in the problem, and in most cases takes [value missing]. ,in The scope of the characteristics of the stakeholders in the issue, but in some cases, More effective and avoids flying too far; Randomly generate a number that follows a uniform distribution. The algorithm sets the probability of the damage location exceeding the damage tolerance. Compare with the random number, if Then change randomly The value, and conversely if ,but The value remains unchanged; C. After this process is completed, test the changed damage location. Based on the test results, compare them with the test values ​​of a set of damage locations obtained after the update in step B. Select the better damage location from the corresponding damage locations and keep it. In this way, a set of globally optimal damage location sequences for the current model can be selected. D. Determine the above Does the algorithm meet the termination condition set for the problem to be optimized? If not, return to step B to continue the calculation; if so, That is, the global optimal solution .

2. An apparatus for the rapid detection method of aircraft deformation scanning as described in claim 1, characterized in that, The system includes a detection system and a data processing and control box. The detection system comprises a UAV detection system, a cloud receiving module, and a deformation analysis system. The UAV detection system includes a UAV, a high-definition camera, an industrial camera, a lens, and a camera controller. The industrial camera, under the control of the camera controller, collects structured light reflected from the fuselage through the lens. The deformation analysis system includes a laser and industrial camera synchronization system, a laser and industrial camera position detection module, a non-deformable aircraft template database, and a data comparison system. The high-definition camera, the industrial camera, the lens, and the camera controller are all mounted on the UAV. The data processing and control box queries the non-deformable aircraft template database and performs comparisons through the data comparison system to determine whether the aircraft is deformed.

3. The aircraft deformation scanning and detection equipment according to claim 2, characterized in that, The drone transmits detection information to the gimbal, which is then processed by the cuckoo search algorithm for positioning and transmission to the drone for a second point-to-point scan.

4. The aircraft deformation scanning and detection equipment according to claim 2, characterized in that, The deformation analysis system compares the images captured by the UAV with the database for the first time, extracts feature values ​​as search thresholds, obtains preliminary results, filters out defective images, and then uploads the defective images to the cloud. The system uses the Cuckoo algorithm to accurately detect the locations that need to be accurately detected a second time within a specific period of time, issues instructions to the UAV to perform in-depth analysis on the secondary images, and compares the results with the standard database in the cloud to determine the specific type and degree of damage.

Images