An aircraft fuselage defect in-situ automated detection system and method

Abstract

The application discloses an aircraft fuselage defect in-situ automatic detection system and method, and belongs to the field of aircraft maintenance and maintenance. The system comprises a visual system module, an ultrasonic system module, and a data processing and storage module. The visual system module is used for performing marker detection and ultrasonic probe position automatic detection, realizing alignment of an aircraft numerical model coordinate system and a camera coordinate system, and calculating a detailed position of the probe position on a target aircraft. The ultrasonic system module adopts a phased array C-scan ultrasonic device to realize detection of internal defects in a fuselage skin-lattice structure and obtain an ultrasonic detection result. The data processing and storage module is a computing device with a computing function and a certain amount of storage medium, and is used for realizing target detection and computing function of the visual system and the ultrasonic system. The application can realize automatic acquisition of position coordinates of detection data, normalized storage, and detection area visualization, and greatly reduces manual input.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of aircraft maintenance and repair, in particular to an aircraft fuselage defect in situ automatic detection system and method. BACKGROUND

[0002] Lightweight high-strength composite material load-bearing skin is one of the typical signs of advanced civil and military aircraft, but the layers and structures of the composite material skin are prone to skin delamination, skin and stringer debonding and other damage under impact load, fatigue cycle, and over-threshold load. Currently, the fuselage skin needs to be comprehensively non-destructively detected every few to several hundred flight hours to identify possible skin damage.

[0003] The commonly used detection method at present is to record the detection and record of aircraft defects by means of artificial detection + artificial judgment + artificial marking + artificial record. The operator pushes and scans the probe to detect and evaluate the detection results, marks the shape and area of the defect after finding the defect, and records the position, depth, area and other information of the defect in the record book in a handwritten manner.

[0004] However, the artificial recording method has certain disadvantages. First, the artificial recording method is time-consuming and labor-intensive and cannot adapt to the trend of automatic detection. Second, artificial detection inevitably has problems such as area omission, area repetition, and inaccurate defect position interpretation. In addition, the information of defect position, defect area, and defect depth recorded by artificial recording has the problem of non-unique reference and poor standardization. Finally, paper defect record data needs to be processed twice to be saved as digital data, which is not convenient for statistical analysis of defects and detection conditions.

[0005] Therefore, the skilled in the art provides an aircraft fuselage defect in situ automatic detection system and method to solve the problems raised in the background art. SUMMARY

[0006] The purpose of the present application is to provide an aircraft fuselage defect in situ automatic detection system and method, which can realize automatic acquisition of position coordinates of detection data and standardized storage, as well as detection area visualization, and can greatly reduce manual input to solve the problems raised in the background art.

[0007] To achieve the above purpose, the present application provides the following technical solutions:

[0008] An aircraft fuselage defect in situ automatic detection system, comprising a vision system module, an ultrasonic system module, and a data processing and storage module;

[0009] The visual system module is used for marker detection and automatic detection of the ultrasonic probe position, realizes alignment of the aircraft digital model coordinate system and the camera coordinate system, and calculates the detailed position of the probe position on the target aircraft;

[0010] The ultrasonic system module adopts a phased array C-scan ultrasonic device to realize detection of internal defects in the aircraft skin-lattice structure and obtain an ultrasonic detection result.

[0011] The data processing and storage module is a computing device with a computing function and a certain amount of storage medium, and is used for realizing target detection and computing function of the visual system and the ultrasonic system, and providing a database for storing all damage items.

[0012] As a further scheme of the present application, the visual system module comprises one or more networked cameras and a plurality of coded markers attached to specific positions of the aircraft body, wherein the bearing platform of the networked camera comprises a UAV platform and an external fixed support, and each networked camera needs to be calibrated to determine an intrinsic matrix.

[0013] As a further scheme of the present application, the ultrasonic system module comprises one or more networked ultrasonic nondestructive testing devices for detecting delamination and debonding damage points of the aircraft skin and lattice structure.

[0014] As a further scheme of the present application, the data processing and storage module comprises a networked host server and a display, wherein the networked host server is used for providing storage and computing services, and the display is used for presenting the results of detection progress and defect statistics.

[0015] As a further scheme of the present application, the data processing and storage module needs to store a high-fidelity three-dimensional 1:1 digital model of the aircraft being repaired, which stores surface key points and mesh information of the aircraft, and all key points have unique coordinates in the model coordinate system.

[0016] As a further scheme of the present application, the specific steps of the visual system module for aligning the aircraft digital model coordinate system and the camera coordinate system are as follows:

[0017] S101: Code markers are attached to the preset positions on the aircraft body surface, and the positions and distribution meet two requirements: the first requirement is that each networked camera can clearly capture at least three code markers at the same time; and the second requirement is that the code markers need to be accurately corresponded to the key points of the known coordinates on one surface of the digital model.

[0018] S102: For the camera s The projection transformation corresponding to the known intrinsic parameters is represented as , take a photo at one of the poses;

[0019] S103: Run the marker detection program to ensure that it can clearly detect n a visual positioning marker point, whose coordinates are , in the imaging pixel position coordinates on the camera s ; ;

[0020] S104: Run the space transformation matrix solving program, input the camera intrinsic parameters, the point pair, and solve the orthogonal transformation matrix R of the camera s coordinate system to the model coordinate system and the translation vector T, which parameterizes the affine transformation parameters, denoted as .

[0021] As a further scheme of the present application: the specific steps for the visual system module to obtain the position of the defect are:

[0022] S201: According to the intrinsic and extrinsic parameters of the camera imaging, solve the key points of the digital model in the imaging system under the curved surface imaging position, according to

[0023] ;

[0024] S202: When using the probe to detect, ensure that the probe at least clearly appears in one camera s , detect the pixel position where the probe is located through the trained target detection neural network algorithm ;

[0025] S203: According to the sorting method, find the three imaging points closest to x s in the Euclidean distance index ; ;

[0026] S204: Solve the barycentric method parameters:

[0027] ;

[0028] S205: Calculate the real coordinates of the probe in the model coordinate system:

[0029] .

[0030] As a further scheme of the present application: the specific process for the digital model to present the detection progress result is:

[0031] S301: Run steps S101~S104, S201~S205 frame by frame to obtain the running trajectory point sequence of the probe on the surface of the aircraft, and connect them into a curve;

[0032] S302: calculating the probe scanning area according to the actual width of the probe;

[0033] S303: filling the area scanned by all the probes in a single detection with a certain color or a map as the detection result picture of the ultrasound system module; the area not scanned by the probe is not filled with color.

[0034] As a further scheme of the present application, the specific steps of adding the nondestructive testing data entry in the data processing and storage module are:

[0035] S401: obtaining the C-scan picture from the ultrasound system module, running the defect target automatic detection program, if a defect is detected, obtaining the position, area, depth, time range of the defect, and running S402-S404, otherwise waiting for the next ultrasound system input result;

[0036] S402: for the push-scan track of one probe, obtaining the track of the probe in the push-scan process and the coverage area in the model coordinate system through steps S101-S104 and S201-S205;

[0037] S403: aligning the time range output by S401 with the position of the probe in the time range in S402, obtaining the model coordinate system position corresponding to the probe when the defect is detected;

[0038] S404: combining the defect position in the model coordinate system obtained by S403 with the defect depth, area, shape information obtained by S401 to form a defect detection record, and storing the record in the database.

[0039] The present application also discloses an aircraft fuselage defect in-situ automatic detection method, which adopts an aircraft fuselage defect in-situ automatic detection system and comprises the following steps:

[0040] The marker detection and ultrasonic probe position automatic detection are performed through the vision system module, the camera coordinate system and the aircraft model coordinate system are aligned, and the detailed position of the probe position on the target aircraft is calculated;

[0041] The detection of the internal defect in the aircraft skin-grid structure is realized through the ultrasound system module, and the ultrasonic detection result is obtained;

[0042] The target detection and calculation functions of the vision system and the ultrasound system are realized through the data processing and storage module, and the database is provided, which is responsible for storing all the damage entries.

[0043] Compared with the prior art, the present application has the following beneficial effects:

[0044] 1、The application can realize the functions of automatic acquisition of position coordinates and standardized storage of detection data, and visualization of detection area, and has high overall automation degree, which can greatly reduce manual input.

[0045] 2、The application can effectively avoid problems such as area omission, area repetition, and inaccurate defect position interpretation, and improve detection accuracy and efficiency through in-situ automatic detection of fuselage defects.

[0046] 3、Compared with traditional manual detection, the in-situ automatic detection of fuselage defects of the application can effectively avoid problems such as non-unique reference and poor standardization.

[0047] 4、The application can directly save defect record data without the need for secondary processing to save as digital data, which is convenient for statistical analysis of defects and detection conditions. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 It is a structural block diagram of a fuselage defect in-situ automatic detection system.

[0049] Figure 2 It is a detection flowchart of a fuselage defect in-situ automatic detection system.

[0050] In the figure: 110, visual system module; 111, networked camera; 112, unmanned aerial vehicle platform; 113, first data transmission module; 114, coded marker; 120, ultrasonic system module; 121, probe; 122, damage point; 123, second data transmission module; 130, data processing and storage module; 131, networked host server; 132, digital model; 133, non-destructive testing data entry; 134, model coordinate system; 140, target aircraft. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0052] As mentioned in the background of the present application, it is found through research that the commonly used defect detection method is to record the detection and record of the defects of the aircraft by means of artificial detection + artificial discrimination + artificial marking + artificial record, but the artificial record method has certain disadvantages, first, the artificial record method is time-consuming and laborious, and cannot adapt to the trend of automatic detection; second, artificial detection inevitably has problems such as area omission, area repetition, and inaccurate defect position interpretation; in addition, the information such as defect position, defect area, and defect depth recorded by artificial record has the problem of non-unique reference and poor standardization; finally, the paper defect record data needs to be processed twice to be saved as digital data, which is not convenient for statistical analysis of defects and detection conditions.

[0053] In order to solve the above defects, the present application discloses an aircraft fuselage defect in-situ automatic detection system and method, which can realize the functions of automatic acquisition of position coordinates of detection data and standardized storage, and visualization of detection area, and can greatly reduce manual input.

[0054] The scheme of the present application will be described in detail below in combination with the drawings.

[0055] Please refer to Figure 1 In the embodiment of the present application, an aircraft fuselage defect in-situ automatic detection system comprises a vision system module 110, an ultrasonic system module 120, and a data processing and storage module 130; the vision system module 110 is used for marker detection and ultrasonic probe position automatic detection, realizes alignment of the aircraft numerical model coordinate system and the camera coordinate system, and calculates the detailed position of the probe 121 on the target aircraft 140; the ultrasonic system module 120 adopts a phased array C-scan ultrasonic device to realize the detection of internal defects in the fuselage skin-grid structure and obtain the ultrasonic detection result; the data processing and storage module 130 is a computing device with computing function and certain storage medium, which is used to realize the target detection and computing function of the vision system and the ultrasonic system, and provide a database for storing all damage items. The present application can realize the functions of automatic acquisition of position coordinates of detection data and standardized storage, and visualization of detection area, and has high overall automation degree, which can greatly reduce manual input.

[0056] In the embodiment, the visual system module 110 includes one or more networked cameras 111, and several coded markers 114 attached to specific locations of the fuselage, wherein the bearing platform of the networked cameras 111 includes the UAV platform 112 and a fixed support outside the world, and each networked camera 111 needs to be calibrated to determine the intrinsic (projection parameter) matrix. The intrinsic matrix is a parameter matrix describing the internal properties of the camera, which establishes the transformation relationship between the three-dimensional camera coordinates and the two-dimensional homogeneous image coordinates. These parameters are only determined by the physical properties of the camera itself, such as focal length, principal point coordinates (optical center), etc., and are independent of the external environment. Therefore, once the camera is manufactured and calibrated, its intrinsic matrix is usually kept unchanged during the use of the camera. The visual system module 110 sends the detection data to the data processing and storage module 130 through the first data transmission module 113.

[0057] In the embodiment, the ultrasonic system module 120 includes one or more networked ultrasonic non-destructive testing devices for detecting delamination and debonding damage points 122 of the aircraft skin and grid structure, and the networked ultrasonic non-destructive testing devices have probes 121.

[0058] In the embodiment, the data processing and storage module 130 includes a networked host server 131 and a display, wherein the networked host server 131 is used to provide storage and computing services, and the display is used to present the results of detection progress and defect statistics.

[0059] In the embodiment, the data processing and storage module 130 needs to store a high-fidelity three-dimensional 1:1 digital model 132 of the aircraft being repaired, which stores the surface key points and mesh information of the aircraft surface, and all key points have unique coordinates in the model coordinate system 134. The surface key points and mesh information of the aircraft surface play a crucial role in the design, manufacture, analysis and maintenance of the aircraft, wherein the surface key points of the aircraft surface usually refer to those points that have important influence on the aerodynamic performance, structural strength and appearance modeling of the aircraft, and the mesh information of the aircraft surface refers to the information required when the aircraft surface is divided into a series of small units (such as triangles, quadrilaterals, etc.) for numerical analysis (such as computational fluid dynamics analysis, finite element analysis, etc.).

[0060] In the embodiment, the specific steps of aligning the aircraft digital model coordinate system and the camera coordinate system by the visual system module 110 are as follows: S101: attaching coded markers 114 at predetermined positions on the surface of the fuselage, which meet two requirements: the first requirement is that each networked camera 111 can clearly capture at least three coded markers 114 at the same time; the second requirement is that the coded markers 114 need to accurately correspond to the key points of the known coordinates on one surface of the digital model 132; S102: for the camera sThe projection transformation corresponding to its known intrinsic parameters is expressed as: S103: Take a photo in one of the poses; S104: Run the marker detection program to ensure that it can be clearly detected within the field of view. n A visual positioning marker, with coordinates as follows: In the camera s The image pixel position coordinates on are S104: Run the space transformation matrix solver, input camera intrinsic parameters and point pairs, and solve for the camera... s The orthogonal transformation matrix R from the coordinate system to the model coordinate system 134 and the translation vector T (camera extrinsic parameter) form the affine transformation parameters, denoted as... In the process of aligning the aircraft's digital model coordinate system and the camera coordinate system in the vision system module 110, the marker detection program is a crucial step. Its main function is to accurately identify and locate pre-defined coded markers 114 in the captured photographs, thereby obtaining the pixel position coordinates of these markers on the photographs. This coordinate information is the foundation for subsequent calculations of camera extrinsic parameters and the realization of coordinate system alignment. Affine transformation is a method of geometrically transforming graphics, including translation, scaling, rotation, and skewing. It can perform various linear transformations and displacements on graphics while maintaining their "flatness" and "parallelism." The parameters of an affine transformation are typically used to define the specific form of this transformation.

[0061] In this embodiment, the specific steps for the vision system module 110 to obtain the location of the defect are as follows:

[0062] S201: Solve for each keypoint of digital model 132 based on the camera's intrinsic parameters (intrinsic parameters are the camera's own properties, independent of the scene, such as focal length, principal point, distortion coefficients, and image size) and extrinsic parameters (extrinsic parameters describe the camera's position and orientation relative to the world coordinate system, such as rotation matrix and translation vector). At the curved surface imaging position in the imaging system, according to

[0063] ;

[0064] S202: When using probe 121 for detection, ensure that probe 121 is clearly visible in at least one camera. s In the process, the pixel position of probe 121 is detected by a trained target detection neural network algorithm. ;

[0065] S203: According to the sorting method, and according to the Euclidean distance index (Euclidean distance is a commonly used method for calculating the distance between two points; mathematically, it is represented as the straight-line distance between the two points), find... In and x s The three closest imaging points ;

[0066] S204: Solve the barycentric method parameters:

[0067] ;

[0068] S205: Calculate the real coordinates of the probe 121 in the model coordinate system 134:

[0069] .

[0070] In this embodiment, the specific process of the digital model 132 presenting the detection progress result is as follows:

[0071] S301: Run steps S101-S104 and S201-S205 frame by frame to obtain the running track point sequence of the probe 121 on the aircraft surface and connect them into a curve;

[0072] S302: Calculate the scanning area of the probe 121 according to the actual width of the probe 121;

[0073] S303: Use the blank digital model 132 to fill the area scanned by all the probes 121 in a single detection with a certain color or map the detection result picture of the ultrasonic system module 120; the area not scanned by the probe 121 is not filled with color.

[0074] In this embodiment, the specific steps of adding the nondestructive testing data entry 133 in the data processing and storage module 130 are as follows:

[0075] S401: Obtain the C-scan picture from the ultrasonic system module 120, run the defect target automatic detection program, if a defect is detected, obtain the position, area, depth, time range of the defect, and run S402-S404, otherwise wait for the next ultrasonic system input result;

[0076] S402: For the push-scan track of the probe 121, obtain the track of the probe 121 in the push-scan process and the coverage area thereof in the model coordinate system 134 by steps S101-S104 and S201-S205;

[0077] S403: Align the time range output by S401 with the position of the probe 121 in S402 to obtain the position of the probe 121 in the model coordinate system 134 corresponding to the detected defect;

[0078] S404: Form a defect detection record by combining the position of the defect in the model coordinate system 134 obtained by S403 with the defect depth, area, shape information obtained by S401, and store it in the database.

[0079] In the embodiment, the visual system module 110, the ultrasonic system module 120, and the data processing and storage module 130 need to be networked through 5G, WiFi or other local or global networks, and data transceiving sharing can be achieved; after the hardware part of the detection system is built based on the above requirements, the following steps are combined Figure 2 Introduce the data analysis and automatic detection process at the algorithm level.

[0080] The application also discloses an in-situ automatic detection method for aircraft fuselage defects, adopts an in-situ automatic detection system for aircraft fuselage defects, and comprises the following steps: performing marker detection and ultrasonic probe position automatic detection through the visual system module 110, realizing alignment of an aircraft digital model coordinate system and a camera coordinate system, and calculating a detailed position of the probe 121 on the target aircraft 140; performing detection on internal defects in a fuselage skin-lattice structure through the ultrasonic system module 120, and obtaining an ultrasonic detection result; and realizing target detection and calculation functions of the visual system and the ultrasonic system through the data processing and storage module 130, and providing a database for storing all damage items.

[0081] The application can realize automatic acquisition of position coordinates of detection data, standardized storage, and detection area visualization, has high overall automation, and can greatly reduce manual input. The in-situ automatic detection of aircraft fuselage defects can effectively avoid problems such as area omission, area repetition, and inaccurate defect position interpretation, and improve detection accuracy and efficiency. Compared with traditional manual detection, the in-situ automatic detection of aircraft fuselage defects can effectively avoid problems such as non-unique reference and poor standardization. The defect record data can be directly saved without being processed again to be saved as digital data, and the defect and detection condition can be conveniently analyzed.

[0082] Obviously, those skilled in the art can make various modifications and changes to the application without departing from the spirit and scope of the application. Thus, if these modifications and changes of the application belong to the scope of the claims of the application and equivalent technologies thereof, the application also intends to include these modifications and changes.

[0083] The above is only a preferred specific embodiment of the application, but the protection scope of the application is not limited to this. Any skilled person in the art can make equivalent substitutions or changes to the technical solutions and inventive concepts of the application within the technical range disclosed by the application, and all of them should be covered within the protection scope of the application.

Claims

1. An automated in-situ detection system for defects in an aircraft fuselage, characterized in that, The visual system module, the ultrasonic system module, and the data processing and storage module are included. The visual system module is used for marker detection and automatic detection of the ultrasonic probe position, aligning the aircraft digital model coordinate system and the camera coordinate system, and calculating the detailed position of the probe on the target aircraft. The ultrasonic system module adopts a phased array C-scan ultrasonic device to detect internal defects in the aircraft skin-lattice structure and obtain ultrasonic detection results. The data processing and storage module is a computing device with computing functions and a certain amount of storage medium, which is responsible for target detection and computing functions of the visual system and the ultrasonic system, and provides a database for storing all damage entries. The visual system module includes one or more networked cameras and several coded markers attached to specific positions of the aircraft body. The ultrasonic system module includes one or more networked ultrasonic non-destructive testing devices for detecting delamination and debonding damage points of the aircraft skin and lattice structure. The data processing and storage module includes a networked host server and a display, wherein the networked host server is used to provide storage and computing services, and the display is used to present the detection progress and defect statistics results. The data processing and storage module needs to store a high-fidelity three-dimensional 1:1 digital model of the aircraft being repaired, which stores the surface key points and mesh information of the aircraft, and all key points have unique coordinates in the model coordinate system. The specific steps of the visual system module for aligning the aircraft digital model coordinate system and the camera coordinate system are as follows: S101: Apply coded markers at predetermined positions on the aircraft body surface, which meet two requirements: the first requirement is that each networked camera can clearly capture at least three coded markers at the same time; the second requirement is that the coded markers need to accurately correspond to the known coordinates of the key points on the digital model. S102: For the camera of interest s The projection transformation whose known intrinsic parameters are denoted as At one of the poses, a photo is taken; S103: Run the marker detection program to ensure that it can clearly detect n a visual positioning marker point with coordinates , whose imaging pixel position coordinates on the camera s are ; S104: Run the space transformation matrix solving program, input camera intrinsic parameters, point pairs, solve the camera s The orthogonal transformation matrix R and the translation vector T from the camera coordinate system to the model coordinate system, which parameterizes the affine transformation parameters, are denoted as ; The specific steps of the visual system module for obtaining the position of the defect are as follows: S201: Solve the key points of the digital model according to the internal and external parameters of camera imaging Under the imaging system, the curved surface imaging position is ； S202: When detecting using the probe, ensure that the probe is at least clearly visible in one camera s The pixel position where the probe is located is detected by the trained target detection neural network algorithm ; S203: find the three imaging points closest to x in the middle of the three imaging points s ;​ S204: Solve the gravity method parameters: ； S205: Calculate the real coordinates of the probe in the model coordinate system: 。 2. An automated in-situ detection system for defects in an aircraft fuselage as defined in claim 1, wherein, The specific process of the digital model for presenting the detection progress result is as follows: S301: Run steps S101~S104, S201~S205 frame by frame to obtain the running trajectory point sequence of the probe on the aircraft surface and connect them into a curve; S302: Calculate the probe scanning area according to the actual width of the probe; S303: Use the blank digital model to fill the area scanned by all probes in a single detection with a certain color or map the detection result picture of the ultrasonic system module; the area not scanned by the probe is not filled with color.

3. An automated in-situ detection system for defects in an aircraft fuselage as defined in claim 2, wherein, The specific steps of adding non-destructive testing data entries in the data processing and storage module are as follows: S401: Obtain C-scan picture from ultrasonic system module, run defect target automatic detection program, if defect is detected, get position, area, depth, time range of defect, and run S402-S404, otherwise wait for next ultrasonic system input result; S402: For the push-scan track of the probe, obtain the track of the probe in the push-scan process and the coverage area thereof in the model coordinate system through steps S101-S104 and S201-S205; S403: Align the time range output by S401 with the position of the probe in the time range in S402, and obtain the position of the probe in the model coordinate system corresponding to the detected defect; S404: Form a defect detection record by combining the position of the defect in the model coordinate system obtained by S403 with the depth, area and shape information of the defect obtained by S401, and store the record in a database.

4. A method for automated in-situ detection of defects in an aircraft fuselage, characterized in that The aircraft fuselage defect in-situ automatic detection system comprises the following steps: Marker detection and ultrasonic probe position automatic detection are performed through a visual system module, the camera coordinate system and the aircraft model coordinate system are aligned, and the detailed position of the probe on the target aircraft is calculated; The detection of internal defects in the aircraft skin-lattice structure is realized through an ultrasonic system module, and ultrasonic detection results are obtained; The target detection and calculation functions of the visual system and the ultrasonic system are realized through a data processing and storage module, and a database is provided to store all damage entries.

Images