Aircraft surface defect three-dimensional detection method and equipment based on image stitching

Abstract

The invention relates to an aircraft surface defect three-dimensional detection method and equipment based on image stitching, and the method comprises the steps: obtaining multi-angle image data of an aircraft surface through a camera, and carrying out the preprocessing; feature points are extracted from the preprocessed image, and feature matching is carried out; based on a feature matching result, splicing the plurality of images by using an image splicing algorithm to generate a panoramic image of the aircraft surface; according to the spliced panoramic image and the camera parameters, utilizing a three-dimensional reconstruction algorithm to recover a three-dimensional model of the aircraft surface; and performing defect three-dimensional detection on the reconstructed three-dimensional model, and labeling and recording defects. Compared with the prior art, the method has the advantages of realizing three-dimensional defect detection on the surface of the aircraft, and being accurate and efficient in detection and the like.

Description

Technical Field

[0001] This invention relates to the field of aircraft surface inspection technology, and in particular to a method and device for three-dimensional inspection of aircraft surface defects based on image stitching. Background Technology

[0002] With the development of the aviation industry, the demand for aircraft surface inspection is increasing. Traditional inspection methods mainly rely on manual splicing and visual inspection, which are cumbersome, inefficient, prone to missing defects, and relatively inaccurate. Although there are some image processing-based inspection methods in the existing technology, they often only provide two-dimensional inspection results and cannot intuitively display the three-dimensional morphology and defects of the aircraft surface.

[0003] Application No. CN201810517270.8 discloses an image stitching method for detecting micro-defects on the surface of large-aperture optical crystals. This method addresses the problem of time-consuming image acquisition and defect identification in current defect detection processes for large-aperture crystals. First, the surface area of ​​the large-aperture crystal element to be tested is scanned, and real-time image acquisition of the scanned area is performed using a detection microscope and a detection CCD. The size range of individual images and the size of overlapping areas are determined. Then, based on the coordinate system translation transformation method, the acquired images are stitched together and the coordinate transformation of defect points is achieved. The position of each defect point in each image in the global coordinate system is determined, and a defect database is established. This invention is applicable to image stitching for detecting micro-defects on the surface of optical crystals.

[0004] Therefore, there is a need for a method that can visualize three-dimensional aircraft surface inspection data based on image stitching technology.

[0005] A search revealed Chinese invention patent publication number CN113375594B, which discloses a digital inspection method for aircraft skin surfaces, comprising the following steps: Step 1. Initial measurement preparation: Calibrate the CCD camera and 3D scanner using a calibration plate to determine the measurement error; randomly affix markers to the surface of the skin part to facilitate subsequent 3D scanner image stitching; move the five-axis inspection platform above the skin part so that the skin part is completely within the scanning range of the five-axis inspection platform; Step 2. By adjusting the pose of the CCD camera mounted at the end of the five-axis inspection platform, acquire images of the skin part; correct the images according to the calibration results; extract the contour of the skin part; establish an external quadrilateral to standardize the contour; simultaneously, record the CCD camera pose information; establish a coordinate system between the part and the five-axis inspection platform; and determine the relative position of the skin part and the five-axis inspection platform. This existing patent suffers from problems such as complex data acquisition and processing, and the inability to perform 3D inspection and visualization.

[0006] How to achieve 3D detection of aircraft surfaces based on image stitching has become a technical problem that needs to be solved. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a three-dimensional detection method and device for aircraft surface defects based on image stitching.

[0008] The objective of this invention can be achieved through the following technical solutions:

[0009] According to one aspect of the present invention, a three-dimensional detection method for aircraft surface defects based on image stitching is provided, the method comprising the following steps:

[0010] Step S1: Acquire multi-angle image data of the aircraft surface using a camera and preprocess it;

[0011] Step S2: Extract feature points from the preprocessed image and perform feature matching;

[0012] Step S3: Based on the feature matching results, use an image stitching algorithm to stitch together multiple images to generate a panoramic image of the aircraft surface.

[0013] Step S4: Based on the stitched panoramic image and camera parameters, use a 3D reconstruction algorithm to restore the 3D model of the aircraft surface.

[0014] Step S5: Perform 3D defect detection on the reconstructed 3D model, and label and record the defects.

[0015] Preferably, step S2 includes: corner detection and extraction of descriptors for each feature point, comparison of descriptors of feature points in different images, feature matching based on distance similarity, establishment of correspondence between images, and verification and optimization of feature matching results.

[0016] More preferably, a fast feature matching algorithm is used for feature matching.

[0017] Preferably, the process of stitching multiple images together using an image stitching algorithm includes:

[0018] Step S31: Based on the feature matching results, select feature point pairs with a matching degree higher than the set threshold, use the random sample consensus algorithm to estimate the transformation model from the feature point pairs through iteration, and calculate the number of interior points under the transformation model to find the best set of matching point pairs.

[0019] Step S32: Based on the filtered matching point pairs, calculate the transformation matrix between the two images using the least squares method, and use the transformation matrix to transform the images so that the two images are aligned in spatial position.

[0020] Step S33: Based on the feature matching results and the transformation results in step S32, determine the overlapping areas between images, and then perform image fusion and global stitching to form a panoramic image of the aircraft surface.

[0021] Step S34 involves post-processing and optimizing the panoramic image, including color correction, removal of redundant areas, and image enhancement.

[0022] Preferably, the process of restoring the three-dimensional model of the aircraft surface using a three-dimensional reconstruction algorithm includes:

[0023] Step S41: Camera intrinsic parameter calibration and extrinsic parameter estimation;

[0024] Step S42: In the stitched panoramic image, feature point extraction and matching are performed again;

[0025] Step S43: Using camera parameters and feature point matching results, estimate the depth information of each feature point through triangulation and calculate the three-dimensional coordinates of the feature points in the camera coordinate system. Use the estimated three-dimensional coordinates as point cloud data to generate a preliminary three-dimensional point cloud of the aircraft surface.

[0026] Step S44: Optimize the preliminary 3D point cloud data using the iterative nearest point algorithm and perform filtering processing;

[0027] Step S45: Based on the data processed in step S44, a surface reconstruction algorithm is used to generate a three-dimensional model of the aircraft surface.

[0028] Step S46: Extract texture information corresponding to the 3D model from the image data obtained in step S1, and map the texture information onto the surface of the 3D model.

[0029] More preferably, step S45 further includes optimizing the generated 3D model, wherein optimization includes removing redundant vertices and triangular faces.

[0030] Preferably, the defect 3D detection of the reconstructed 3D model specifically involves comparing the reconstructed 3D model with a standard model. If the geometric difference between the two is greater than a set threshold, then a defective area is considered to exist.

[0031] More preferably, the geometric differences include differences in distance and volume.

[0032] Preferably, the method further includes visualizing the annotated 3D model, allowing for comprehensive observation of the aircraft's surface morphology and defects through interactive operations.

[0033] According to another aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described thereon.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1) This invention uses an image stitching algorithm to stitch together multiple images taken by a camera after feature matching to generate a panoramic image of the aircraft surface; based on the stitched panoramic image and camera parameters, a three-dimensional reconstruction algorithm is used to reconstruct the three-dimensional model of the aircraft surface; and the reconstructed three-dimensional model is used for defect three-dimensional detection, which improves the accuracy and efficiency of detection compared with traditional two-dimensional detection methods.

[0036] 2) This invention performs 3D reconstruction based on stitched panoramic images and camera parameters. The 3D reconstruction algorithm can accurately restore the 3D structure of the aircraft surface, including: feature point extraction and matching, using the estimated 3D coordinates as point cloud data, optimizing the point cloud data through the Iterative Closest Point (ICP) algorithm to improve the accuracy and consistency of the point cloud, smoothing the point cloud surface, optimizing the generated 3D model to improve the quality and efficiency of the mesh, making the 3D model clearer and more accurate, thereby improving the accuracy and efficiency of detection.

[0037] 3) This invention uses a fast feature matching algorithm to quickly find matching pairs among a large number of feature points. At the same time, it adopts an optimization strategy to improve the accuracy and stability of matching and speed up the detection efficiency.

[0038] 4) This invention improves the usability of the detection results by displaying the morphology and defects of the aircraft surface through three-dimensional visualization. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the three-dimensional detection method for aircraft surface defects in this invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0041] This embodiment relates to a three-dimensional detection method for aircraft surface defects based on image stitching, such as... Figure 1 This includes the following steps:

[0042] Step S1, Image Acquisition: Use a high-definition camera to take pictures of the aircraft surface from multiple angles to obtain multiple image data containing the aircraft surface.

[0043] Step S2, Image Preprocessing: The acquired image is preprocessed, including noise reduction, contrast enhancement, color correction and other operations, to improve image quality.

[0044] Step S3, Feature Extraction and Matching: Extract feature points from the preprocessed image and perform feature matching to establish the correspondence between different images.

[0045] Step S4, Image stitching: Based on the feature matching results, multiple images are stitched together using an image stitching algorithm to generate a panoramic image of the aircraft surface.

[0046] Step S5, 3D Reconstruction: Based on the stitched panoramic image and camera parameters, the 3D structure of the aircraft surface is restored using a 3D reconstruction algorithm.

[0047] Step S6, Defect Detection and Labeling: Perform defect detection on the reconstructed 3D model, and label and record the detected defects.

[0048] Step S7, 3D visualization: The labeled 3D model is visualized and displayed. You can observe the shape and defects of the aircraft surface from all angles by rotating, scaling, and translating.

[0049] Specifically, the detailed explanation of each step is as follows:

[0050] In step S1, a high-definition camera is used to take multiple photos of the aircraft surface from various angles to obtain multiple images containing the aircraft surface. Certain steps must be followed to ensure image quality and efficiency. These steps include:

[0051] Step S1-1, Preparation Stage:

[0052] Choose a suitable high-definition camera to ensure that its resolution, lens quality, and other features meet your shooting requirements.

[0053] Check the status of the camera battery and memory card to ensure that the battery is fully charged and there is enough storage space.

[0054] Prepare a stable tripod or other support equipment to ensure shooting stability.

[0055] Understand the characteristics of the aircraft's surface to determine the key angles and details that need to be photographed.

[0056] Step S1-2, set camera parameters:

[0057] Adjust camera parameters such as exposure and white balance according to the shooting environment and the lighting conditions on the aircraft surface.

[0058] Considering the possibility of needing to capture details of the aircraft's surface, set the camera to manual focus mode and adjust the focus as needed.

[0059] Turn off the flash to avoid interfering with or causing glare during shooting.

[0060] Steps S1-3: Take photos.

[0061] Use a tripod to stabilize the camera and ensure it remains stable and doesn't shake during shooting.

[0062] The aircraft surface was photographed from different angles, including the front, side, and top, to obtain comprehensive image data.

[0063] Pay attention to details when shooting, such as markings, coatings, and damage on the aircraft's surface, to ensure that the image is clear and complete.

[0064] During the shooting process, you can adjust the camera angle and height as needed to obtain the best shooting effect.

[0065] Steps S1-4, Inspection and Adjustment:

[0066] After shooting, promptly check the quality of the photos to ensure that each photo meets the requirements.

[0067] If you find that certain angles or details are not captured well, you can readjust the camera position and take more shots.

[0068] Steps S1-5: Organize and save:

[0069] Import the photos you took into your computer and then categorize and organize them.

[0070] Back up your photo data in case of accidental loss.

[0071] Step S2 involves preprocessing the acquired aircraft surface image, including noise reduction, contrast enhancement, and color correction, to improve image quality. Specific steps include:

[0072] Step S2-1, Denoising: First, it is necessary to identify and remove noise from the image.

[0073] Use filter techniques, such as median filters or Gaussian filters, to smooth images and reduce noise. These filters can help remove isolated pixels or small noise clumps from an image.

[0074] Depending on the type and level of noise in the image, the filter parameters may need to be adjusted to achieve the best denoising effect.

[0075] Step S2-2, Contrast Enhancement:

[0076] Contrast enhancement aims to improve the brightness and contrast of an image, making details in the image clearer.

[0077] Histogram equalization is used to enhance image contrast. This method improves contrast by redistributing pixel intensity values, making the image's histogram more uniform.

[0078] Further enhance contrast using contrast stretching or limit contrast adaptive histogram equalization (CLAHE).

[0079] Step S2-3, Color Correction:

[0080] The purpose of color correction is to adjust the colors of an image to make them more accurate and consistent.

[0081] First, check the overall tone and saturation of the image to ensure they match the actual situation or the expected effect.

[0082] Grayscale conversion: To simplify subsequent processing steps, an image can be converted to a grayscale image.

[0083] You can use a simple averaging method or a weighted averaging method to convert the image to grayscale, ensuring that the converted image still retains enough detail.

[0084] Step S3 involves extracting feature points from the preprocessed image and performing feature matching to establish correspondences between different images, including:

[0085] Step S3-1, Feature point extraction

[0086] Feature point extraction is a crucial step in image processing, used to identify salient and unique points from an image. These points typically correspond to corners, edges, or texture-rich regions in the image. These include:

[0087] Corner detection: Corners are the intersections of two or more edges in an image, exhibiting good stability from different viewpoints. The Harris corner detection algorithm identifies corners by calculating grayscale changes or edge information in local image regions, offering high efficiency and accuracy.

[0088] Feature descriptor extraction: For each detected feature point, a descriptor needs to be extracted to represent its features. Descriptors are typically a set of vectors that quantify information such as pixel intensity distribution, texture, and shape around the feature point. Using SURF (Speed-Up Robust Features), the descriptors not only possess uniqueness but also scale invariance and rotation invariance, maintaining consistency across different viewpoints.

[0089] Step S3-2: Feature matching is performed using the Fast Feature Matching Algorithm (FLANN).

[0090] Feature matching identifies the correspondences between feature points in different images by comparing their descriptors. This includes:

[0091] Descriptor comparison: For feature point descriptors in two images, distance metrics (such as Euclidean distance, cosine distance, etc.) can be used to compare their similarity. The closer the feature points are, the higher their similarity, and therefore the more likely they are to be associated.

[0092] Matching Algorithm: The FLANN (Fast Library for Approximate Nearest Neighbors) algorithm is used for matching. FLANN is a more efficient approximate nearest neighbor search algorithm that can quickly find matching pairs among a large number of feature points. The specific matching process is as follows:

[0093] First, matching is performed using a k-dimension tree; this includes the following steps:

[0094] Divide the k-dimensional space into nested half-spaces, where each node is a k-dimensional hyperrectangular region;

[0095] During the search, starting from the root node, the search determines which subtree to enter based on the relative position of the query point to the partitioning plane.

[0096] The data space is recursively divided into two subspaces until a certain stopping condition is met (such as the number of data points contained in each node reaching a threshold).

[0097] Secondly, Priority Search k-means Tree (PSKT) is used for matching verification:

[0098] Specifically, the search is performed by combining a k-means tree and a priority queue;

[0099] By calculating the distance between the query point and each cluster center, and sorting them according to the distance, the clusters with closer distances are searched first.

[0100] Calculate the distance, maintain a priority queue, and perform the search in the tree.

[0101] Finally, matching optimization: To improve the accuracy and stability of matching, some optimization strategies can be adopted, such as cross-validation. Cross-validation requires that the ratio of the distances between the nearest and second nearest neighbors of a feature point in another image is less than a certain threshold before the two feature points are considered to be a match.

[0102] Step S3-3: Establish the corresponding relationship

[0103] A series of matching feature point pairs can be obtained, which establish a clear correspondence between different images, providing a foundation for subsequent processes such as image stitching and 3D reconstruction.

[0104] Step S4: Based on the feature matching results, multiple images are stitched together to generate a panoramic image of the aircraft surface. Specific algorithm and steps are as follows:

[0105] Step S4-1: Feature matching result filtering and optimization

[0106] Filtering matching results: First, based on the feature matching results, filter out feature point pairs that have a high degree of matching and good stability.

[0107] False matches removal: To further improve matching accuracy, the RANSAC (Random Sample Consensus) algorithm is used to remove false matches. RANSAC estimates the transformation model from feature point pairs iteratively and calculates the number of interior points under this model, thereby finding the optimal set of matching point pairs.

[0108] Step S4-2, Image Transformation and Alignment

[0109] Calculate the transformation matrix: Based on the selected matching point pairs, calculate the transformation matrix between the two images using the least squares method. This transformation matrix typically includes transformation parameters such as rotation, translation, and scaling, used to align one image to another.

[0110] Image transformation: Based on the calculated transformation matrix, one image is transformed to align it spatially with another image.

[0111] Step S4-3, Image fusion and stitching

[0112] Identifying the overlapping region: Based on the results of feature matching and image transformation, the overlapping region between the two images is determined. This overlapping region is crucial for subsequent image fusion.

[0113] Image fusion: In overlapping areas, an image fusion algorithm is used to smoothly transition between the two images to avoid obvious stitching marks.

[0114] Global stitching: After merging the overlapping areas, the two images are stitched together to form a larger panoramic image.

[0115] Step S4-4, Post-processing and optimization

[0116] Color correction: Color correction is performed on the stitched panoramic image to ensure color consistency throughout the image. This is achieved through histogram matching.

[0117] Remove redundant areas: If there are redundant or overlapping areas in the stitched image, remove them by cropping or masking.

[0118] Image enhancement: This feature enhances the stitched panoramic image, improving its clarity and contrast to make it more suitable for subsequent analysis or presentation.

[0119] Step S5: Based on the stitched panoramic image and camera parameters, the 3D structure of the aircraft surface is restored using a 3D reconstruction algorithm. The specific steps are as follows:

[0120] Step S5-1, Camera Calibration

[0121] Intrinsic parameter calibration: First, the camera needs to be calibrated to obtain its internal parameters such as focal length, principal point coordinates, and distortion coefficients. This is usually done by taking pictures using a calibration board (such as a checkerboard calibration board) and calculating the parameters using a calibration algorithm (such as the Zhang Zhengyou calibration method).

[0122] Extrinsic parameter estimation: For each image, its position and orientation in the global coordinate system need to be estimated, i.e., extrinsic parameters (rotation matrix and translation vector). This can usually be solved by the correspondence with panoramic images and feature matching between adjacent images.

[0123] Step S5-2, Feature point extraction and matching

[0124] Feature point extraction: Feature points are extracted again from the stitched panoramic image. This step is for subsequent 3D point cloud generation and matching.

[0125] Feature point matching: For adjacent or overlapping images, feature points are matched. This is achieved by comparing the similarity of descriptors, ensuring the accuracy of the matching.

[0126] Step S5-3, 3D point cloud generation

[0127] Depth estimation: Using camera parameters and feature point matching results, the depth information of each feature point is estimated through triangulation. This involves calculating the three-dimensional coordinates of the feature point in the camera coordinate system.

[0128] Point cloud generation: The estimated 3D coordinates are used as point cloud data to generate a preliminary 3D point cloud of the aircraft surface.

[0129] Step S5-4, Point Cloud Optimization and Filtering

[0130] Point cloud optimization: By using the iterative nearest point (ICP) algorithm, point cloud data is optimized to improve the accuracy and consistency of the point cloud.

[0131] Filtering: Removes unnecessary data such as noise points and outliers, smooths the point cloud surface, and makes the 3D model clearer and more accurate.

[0132] Step S5-5, 3D surface reconstruction

[0133] Surface Reconstruction Algorithm: Using point cloud data, a surface reconstruction algorithm is employed to generate a 3D model of the aircraft surface. The surface reconstruction algorithm used is Poisson surface reconstruction.

[0134] Mesh optimization: The generated 3D model is optimized by removing redundant vertices and triangular faces to improve the quality and efficiency of the 3D model mesh.

[0135] Steps S5-6, Texture Mapping

[0136] Texture extraction: Extracting texture information corresponding to a 3D model from the original image.

[0137] Texture mapping: Mapping the extracted texture information onto the surface of a 3D model to make the model more realistic and vivid.

[0138] Through the above steps, using the stitched panoramic image and camera parameters, a 3D reconstruction algorithm is used to reconstruct the 3D structure of the aircraft surface. This 3D model can be used for subsequent analysis, measurement, visualization, and other applications, providing strong support for the inspection and maintenance of aircraft surfaces.

[0139] Step S6: Perform defect detection on the reconstructed 3D model, and label and record the detected defects. The specific algorithm and steps are as follows:

[0140] Step S6-1, 3D model preprocessing

[0141] Smoothing: First, the 3D model is smoothed to eliminate noise or minor unevenness that may be generated during the reconstruction process. This is achieved using a Gaussian filtering algorithm.

[0142] Simplification and optimization: In order to improve the efficiency of subsequent processing, the model can be simplified by removing redundant vertices or faces while maintaining the geometric features of the model.

[0143] Step S6-2, Defect Detection

[0144] Standard Model Comparison: The reconstructed 3D model is compared with a standard model. By calculating the geometric differences between the two (such as distance, volume, etc.), potential defect areas are detected.

[0145] Step S6-3, Defect Marking and Recording

[0146] Defect region localization: Once a defect is detected, its exact location in the 3D model needs to be determined. This is achieved by calculating the geometric information of the defect region, such as its boundaries and center point.

[0147] Annotation and visualization: Detected defects are annotated, such as by marking them on the 3D model using different colors or textures. Simultaneously, detailed defect information (such as location, size, and type) is recorded in a database for subsequent analysis and processing.

[0148] Report generation: Based on the recorded defect information, generate a detailed defect detection report, including statistical information such as the number, distribution, and severity of defects, as well as specific defect images or videos.

[0149] Step S7 involves visualizing the annotated 3D model, allowing users to observe the aircraft's surface morphology and defects from all angles through operations such as rotation, scaling, and translation. This is achieved through the following steps:

[0150] a. Select a visualization tool

[0151] b. Import the 3D model

[0152] Import the annotated 3D model into the selected visualization tool.

[0153] c. Add interactive features

[0154] Rotation: Allows users to rotate the 3D model using mouse or keyboard operations. This is achieved by listening for mouse drag events or keyboard arrow key events and updating the model's rotation matrix accordingly.

[0155] Zoom: Provides zoom functionality so users can adjust the distance and size of the model they are viewing. This is achieved through mouse wheel events or specific keyboard keys, by adjusting the camera position or the model's zoom level.

[0156] Translation: Allows users to translate the model for viewing from different angles. This is achieved by dragging the mouse or using specific keyboard keys, and the translation effect is achieved by updating the camera position or the model's translation matrix.

[0157] d. Defect visualization

[0158] Ensure that labeled defects are clearly displayed during visualization. Highlight defect areas using different colors, textures, or highlights so that users can easily identify them.

[0159] e. Optimize rendering performance

[0160] To improve rendering speed, texture compression and reducing the number of faces are used to optimize the model.

[0161] f. Export and Sharing

[0162] Once the visualization settings are complete, the results can be exported as videos, images, or interactive web applications for sharing with others or for further analysis.

[0163] Through comparative experiments and practical application verification, this invention demonstrates excellent performance in the three-dimensional visualization of aircraft surface inspection data. Compared with traditional two-dimensional inspection methods, this invention can more intuitively display the three-dimensional morphology and defects of the aircraft surface, improving the accuracy and efficiency of inspection.

[0164] The electronic device of this invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0165] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0166] The processing unit executes the various methods and processes described above, such as methods S1 to S7. For example, in some embodiments, methods S1 to S7 may be implemented as computer software programs tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of methods S1 to S7 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S7 by any other suitable means (e.g., by means of firmware).

[0167] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0168] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0169] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0170] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A three-dimensional detection method for aircraft surface defects based on image stitching, characterized in that, The method includes the following steps: Step S1: Acquire multi-angle image data of the aircraft surface using a camera and preprocess it; Step S2: Extract feature points from the preprocessed image and perform feature matching; Step S3: Based on the feature matching results, use an image stitching algorithm to stitch together multiple images to generate a panoramic image of the aircraft surface. Step S4: Based on the stitched panoramic image and camera parameters, use a 3D reconstruction algorithm to restore the 3D model of the aircraft surface. Step S5: Perform 3D defect detection on the reconstructed 3D model, and label and record the defects.

2. The method for three-dimensional detection of aircraft surface defects based on image stitching according to claim 1, characterized in that, The process of step S2 includes: corner detection and extraction of descriptors for each feature point, comparison of descriptors of feature points in different images, feature matching based on distance similarity, establishment of correspondence between images, and verification and optimization of feature matching results.

3. The method for three-dimensional detection of aircraft surface defects based on image stitching according to claim 2, characterized in that, The feature matching is performed using a fast feature matching algorithm.

4. The three-dimensional detection method for aircraft surface defects based on image stitching according to claim 1, characterized in that, The process of stitching multiple images together using an image stitching algorithm includes: Step S31: Based on the feature matching results, select feature point pairs with a matching degree higher than the set threshold, use the random sample consensus algorithm to estimate the transformation model from the feature point pairs through iteration, and calculate the number of interior points under the transformation model to find the best set of matching point pairs. Step S32: Based on the filtered matching point pairs, calculate the transformation matrix between the two images using the least squares method, and use the transformation matrix to transform the images so that the two images are aligned in spatial position. Step S33: Based on the feature matching results and the transformation results in step S32, determine the overlapping areas between images, and then perform image fusion and global stitching to form a panoramic image of the aircraft surface. Step S34 involves post-processing and optimizing the panoramic image, including color correction, removal of redundant areas, and image enhancement.

5. The three-dimensional detection method for aircraft surface defects based on image stitching according to claim 1, characterized in that, The process of restoring the three-dimensional model of the aircraft surface using a three-dimensional reconstruction algorithm includes: Step S41: Camera intrinsic parameter calibration and extrinsic parameter estimation; Step S42: In the stitched panoramic image, feature point extraction and matching are performed again; Step S43: Using camera parameters and feature point matching results, estimate the depth information of each feature point through triangulation and calculate the three-dimensional coordinates of the feature points in the camera coordinate system. Use the estimated three-dimensional coordinates as point cloud data to generate a preliminary three-dimensional point cloud of the aircraft surface. Step S44: Optimize the preliminary 3D point cloud data using the iterative nearest point algorithm and perform filtering processing; Step S45: Based on the data processed in step S44, a surface reconstruction algorithm is used to generate a three-dimensional model of the aircraft surface. Step S46: Extract texture information corresponding to the 3D model from the image data obtained in step S1, and map the texture information onto the surface of the 3D model.

6. The three-dimensional detection method for aircraft surface defects based on image stitching according to claim 5, characterized in that, Step S45 further includes optimizing the generated 3D model, wherein optimization includes removing redundant vertices and triangular faces.

7. The method for three-dimensional detection of aircraft surface defects based on image stitching according to claim 1, characterized in that, The specific method for performing 3D defect detection on the reconstructed 3D model is as follows: the reconstructed 3D model is compared with the standard model. If the geometric difference between the two is greater than a set threshold, it is considered that there is a defect area.

8. The three-dimensional detection method for aircraft surface defects based on image stitching according to claim 7, characterized in that, The geometric differences mentioned include differences in distance and volume.

9. The method for three-dimensional detection of aircraft surface defects based on image stitching according to claim 1, characterized in that, The method also includes visualizing the annotated 3D model, allowing for a comprehensive observation of the aircraft's surface morphology and defects through interactive operations.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 9.

Images