A method and device for detecting aircraft skin parts

Abstract

The present invention discloses a method and device for detecting aircraft skin parts, wherein the detection method comprises: obtaining an image of the surface of an aircraft skin part to be detected by a detection mechanism; preprocessing the image to obtain a preprocessed sample; converting the preprocessed sample into a two-dimensional spectrum diagram, eliminating high-frequency signals containing texture information, and then converting the processed spectrum diagram into a spatial domain image; performing binarization and morphological operations on the texture filtered image to determine whether there are fiber tearing defect features; using the test sample to train a deep learning model until it converges; inputting the test sample into the converged deep learning model to determine whether there are imprint defects in the image; the present invention realizes automatic online detection of surface defects by marking imprint defects on aircraft skin parts; and detecting whether there is tearing by converting the preprocessed image into a two-dimensional spectrum diagram.

Description

Technical Field

[0001] The invention relates to the field of aircraft skin parts detection, and in particular to a method and device for detecting aircraft skin parts. Background Art

[0002] In recent years, with the energy saving and weight reduction of aviation materials, high-performance composite materials have been widely used in aircraft skins, such as radar antenna covers, radio antenna covers, ailerons, some wings, horizontal tails and vertical tails, and some fuselage skins are mostly made of composite materials. Most of the composite materials used for aircraft skins are epoxy carbon fiber composite materials, which are resistant to high temperatures, friction, electricity, heat and corrosion, and have high strength along the fiber axis. Composite materials made of carbon fiber and epoxy resin have low relative density, good rigidity and high strength, and have been widely used in aircraft structural materials.

[0003] In the manufacturing process of aircraft carbon fiber composite skin, the damage is mainly caused by two links, namely the molding process and the assembly process. (1) Defects caused by the molding process: In the manufacturing process of composite structure, under certain process conditions, the matrix resin and the reinforcing fabric fiber are evenly mixed and well combined, and the fiber and resin, and the plies directly reach the expected interface bonding strength. The fiber itself will not cause significant changes in the quality and performance of the composite material after the section, unless the deviation of the curing conditions damages the original performance of the fiber. Under given process conditions such as temperature and pressure, the viscosity of the matrix resin will change significantly with the change of curing temperature. When the curing parameters are not optimized reasonably or fluctuate or deviate, it may affect the resin fluidity and permeability and the quality of the bonding interface between the fiber and the resin, and the plies and the plies. In severe cases, defects such as delamination and pores will occur. (2) Defects caused by the assembly process: The assembly process refers to the connection between the rib and the skin, or the connection between the rib and the skin, which is usually connected by drilling or riveting. However, this method will cause damage, and the damage generation mechanism is different. Drilling or scratching will cause the skin and the drill bit to contact and produce thermal fatigue and stress fatigue. The riveted skin is damaged due to impact. The scratching process causes the carbon fiber composite skin to go through a thermal cycle from room temperature to high temperature to room temperature. The thermal stress generated in the process causes the glue layer to crack. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a fully automatic and efficient aircraft skin parts detection method and device capable of simultaneously detecting surface tearing and imprint defects of aircraft skin parts.

[0005] In order to achieve the above object, the present invention provides a method for detecting aircraft skin parts, comprising:

[0006] Placing the aircraft skin part to be tested on the testing mechanism, and obtaining an image of the surface of the aircraft skin part to be tested through the testing mechanism;

[0007] Preprocessing the image to obtain a preprocessed sample; wherein the preprocessed sample includes a training sample and a test sample;

[0008] Converting the preprocessed samples into a two-dimensional spectrum map, eliminating high-frequency signals containing texture information, and then converting the processed spectrum map into a spatial domain image;

[0009] Performing binarization and morphological operations on the texture-filtered image to determine whether there are fiber tearing defect features;

[0010] Using the test samples to train the deep learning model until it converges;

[0011] The test samples are input into the converged deep learning model, and the input image is forward operated through the trained deep learning model to determine whether there are print defects in the image and transmit the defect information marked on the original image to the production line staff via wireless transmission.

[0012] As a preferred technical solution, the detection mechanism includes:

[0013] The robot ground rail is connected to the ABB manipulator, and the embedded ARM board and wireless transmission module that constitute the image processing system are integrated at its rear to change the base position of the manipulator;

[0014] ABB manipulator, mounted on the robot rail, is used to clamp and move the image acquisition system to collect surface images of aircraft skin parts;

[0015] The image processing system consists of an ARM board and a wireless transmission module. The ARM board integrates an image processing program of a deep learning model to detect and mark surface defects of parts in images captured by industrial cameras. The wireless transmission module then transmits images with imprint defects to the industrial computer database.

[0016] Parts installation platform, which is fixed to the ground and used to install composite parts to be tested;

[0017] The industrial camera group, consisting of 14 industrial cameras, is installed at the clamping position of the manipulator and is used to collect surface images of aircraft skin parts;

[0018] The ring light source group consists of fourteen ring light sources and is used to illuminate the surface of parts.

[0019] As a preferred technical solution, placing the aircraft skin part to be tested on a testing mechanism, and obtaining an image of the surface of the aircraft skin part to be tested through the testing mechanism, further comprising:

[0020] The spatial position of the image acquisition system and the aircraft skin part to be tested is adjusted by translating and rotating the manipulator on the robot rail. When the acquired image has the highest clarity, the movement is stopped and the acquired image is uploaded.

[0021] As a preferred technical solution, after collecting the surface image of the aircraft skin part at the current position, the ARM board drives the manipulator to move laterally on the robot rail to collect the surface image of the part at the next position until the detection work is completed.

[0022] As a preferred technical solution, training the deep learning model with the training samples until the deep learning model converges further includes:

[0023] The training samples are made into a VOC format data set, the real boundary boxes of scratches in the image are manually annotated, and the xml file required by the deep learning model is automatically generated; the xml file in the VOC format is converted into a txt file required by the deep learning model;

[0024] Determine the training parameters of the deep learning model, wherein the training parameters include a learning rate of 0.001, a batch size of 32, and an iteration number of 500 times;

[0025] Input the training samples into the deep learning model for training; observe the loss curve during the training process to determine whether the deep learning model converges; if it converges, stop training; if not, continue training.

[0026] On the other hand, the present invention also provides a detection device for aircraft skin parts, comprising:

[0027] An acquisition unit, used for placing the aircraft skin part to be tested on the detection mechanism, and acquiring an image of the surface of the aircraft skin part to be tested through the detection mechanism;

[0028] A preprocessing unit, which preprocesses the image to obtain a preprocessed sample; wherein the preprocessed sample includes a training sample and a test sample;

[0029] A conversion unit, used to convert the preprocessed samples into a two-dimensional spectrum map, eliminate high-frequency signals containing texture information, and then convert the processed spectrum map into a spatial domain image;

[0030] A first determination unit is used to perform a binarization operation and a morphological operation on the texture-filtered image to determine whether there is a fiber tear defect feature;

[0031] A training unit, used to train the deep learning model using the test sample until it converges;

[0032] The second determination unit is used to input the test sample into the converged deep learning model, perform forward operation on the input image through the trained deep learning model, so as to determine whether there is a print defect in the image and transmit the defect information marked on the original image to the production line staff via wireless transmission.

[0033] The beneficial effects of the present invention compared to the prior art are as follows: the present invention can complete the acquisition of surface images of large-size composite parts by cooperating with the detection platform and the manipulator, and then train the deep learning model to mark the imprint defects on the aircraft skin parts, thereby realizing automatic online detection of surface defects; by converting the preprocessed image into a two-dimensional spectrum diagram, and performing binarization and morphological operations, it is possible to detect whether there is tearing. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of a method for detecting aircraft skin parts provided by the present invention;

[0035] Figure 2 It is a structural diagram of the detection mechanism provided by the present invention;

[0036] Figure 3 The present invention is a structural diagram of a detection device for aircraft skin parts. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Reference Figure 1 This embodiment provides a method for detecting aircraft skin parts, comprising the following steps:

[0039] S10: placing the aircraft skin part to be tested on the testing mechanism, and obtaining an image of the surface of the aircraft skin part to be tested through the testing mechanism;

[0040] Specifically, Figure 2 As shown, the detection mechanism includes a robot ground rail 101, which is connected to the ABB manipulator, and an embedded ARM board and a wireless transmission module constituting an image processing system are integrated at the rear thereof to change the base position of the manipulator;

[0041] ABB manipulator 102, mounted on the robot ground rail, is used to clamp and move the image acquisition system to collect surface images of composite parts;

[0042] The image processing system 103 is composed of an ARM board and a wireless transmission module. The ARM board integrates an image processing program of a deep learning model to detect and mark surface defects of parts in images captured by an industrial camera. The wireless transmission module then transmits images with imprint defects to an industrial computer database.

[0043] Composite material part 104, as a part to be tested, for the camera group to collect surface images;

[0044] Parts installation platform 105, which is fixed to the ground and used to install composite material parts to be tested;

[0045] Industrial camera group 106, consisting of fourteen industrial cameras, is installed at the clamping position of the manipulator and is used to collect surface images of composite parts;

[0046] The annular light source group 107 is composed of fourteen annular light sources and is used to illuminate the surface of the parts to avoid reflection.

[0047] When collecting images, first initialize the hardware equipment of the detection mechanism, return the position and angle of the ABB manipulator to zero, complete the initial inspection of the camera and light source, and install the aircraft skin parts to be tested; then set the system acquisition module parameters according to the current part size, detection accuracy, etc., and adjust the ABB manipulator to the initial detection position; then, complete the on-machine calibration of the detection system under the current system setting parameters and detection state; finally, after other components are installed, the manipulator is equipped with an image acquisition module to complete the horizontal image acquisition of the part surface;

[0048] S20: preprocessing the image to obtain a preprocessed sample; wherein the preprocessed sample includes a training sample and a test sample;

[0049] Specifically, the captured image is subjected to image preprocessing operations such as filtering, denoising, and graying. In this embodiment, a Gaussian filter algorithm is used to perform image denoising. In essence, Gaussian filtering is to perform a convolution operation on the grayscale image I and a Gaussian kernel:

[0050] I σ =I*G σ

[0051] Among them, * represents the convolution operation; G σ is a two-dimensional Gaussian kernel with standard deviation σ, defined as:

[0052]

[0053] Among them, σ is the standard deviation; (x, y) is the point coordinate, which can be considered as an integer in image processing.

[0054] Typically, the size of the Gaussian kernel is 3×3, 5×5, or 7×7.

[0055] S30: converting the preprocessed samples into a two-dimensional spectrum map, eliminating high-frequency signals containing texture information, and then converting the processed spectrum map into a spatial domain image;

[0056] Specifically, in this embodiment, a two-dimensional image Fourier transform is used to transform an image from a spatial domain to a frequency domain. The transformation formula is as follows:

[0057]

[0058] Among them, f(i,j) is the grayscale image; F(k,l) is the frequency spectrum of image f(i,j); M is the number of rows of the image; N is the number of columns of the image.

[0059] S40: performing binarization operation and morphological operation on the texture filtered image to determine whether there is a fiber tearing defect feature;

[0060] It should be noted here that since binarization operations and morphological operations are conventional technical means in the field of image processing, they will not be elaborated here.

[0061] S50: Using the test sample to train the deep learning model until it converges;

[0062] Specifically, the training samples are first made into a VOC format data set, the real boundary boxes of scratches in the image are manually annotated, and the xml file required by the deep learning model is automatically generated; the VOC format xml file is converted into a txt file required by the deep learning model;

[0063] Then, the training parameters of the degree learning model are determined, wherein the training parameters include a learning rate of 0.001, a batch size of 32, and an iteration number of 500 times;

[0064] Finally, the training samples are input into the deep learning model for training; the loss curve during the training process is observed to determine whether the deep learning model converges; if it converges, the training is stopped; if not, the training continues.

[0065] S60: Input the test sample into the converged deep learning model, perform forward operation on the input image through the trained deep learning model, so as to determine whether there is a print defect in the image and transmit the defect information marked on the original image to the production line staff via wireless transmission.

[0066] Through the above method steps, the surface tearing and imprint defects of aircraft skin parts can be automatically detected for products, which can effectively improve the accuracy of detection and can also reduce production costs and improve product quality. In addition, the image processing system uses an embedded ARM board and a wireless transmission module as hardware devices, and adopts a deep learning detection method to realize automatic detection and annotation of imprint defects on the surface of composite parts.

[0067] On the other hand, the present invention also provides a detection device for aircraft skin parts, comprising:

[0068] The acquisition unit 10 is used to place the aircraft skin part to be tested on the detection mechanism, and obtain the image of the surface of the aircraft skin part to be tested through the detection mechanism; it should be noted here that since the specific acquisition method and process have been explained in detail in step S10 of the above-mentioned aircraft skin part detection method, they will not be repeated here.

[0069] The preprocessing unit 20 preprocesses the image to obtain a preprocessed sample; wherein the preprocessed sample includes a training sample and a test sample; it should be noted that since the specific preprocessing method and process have been detailed in step S20 of the above-mentioned aircraft skin parts detection method, they will not be repeated here.

[0070] The conversion unit 30 is used to convert the preprocessed samples into a two-dimensional spectrum diagram, eliminate the high-frequency signals containing texture information, and then convert the processed spectrum diagram into a spatial domain image; it should be noted here that since the specific conversion method and process have been explained in detail in step S30 of the above-mentioned aircraft skin parts detection method, they will not be repeated here.

[0071] The first determination unit 40 is used to perform binarization and morphological operations on the texture filtered image to determine whether there are fiber tearing defect characteristics; it should be noted here that since the specific determination method and process have been explained in detail in step S40 of the above-mentioned aircraft skin parts detection method, they will not be repeated here.

[0072] The training unit 50 is used to train the deep learning model using the test samples until it converges. It should be noted that since the specific training method and process have been explained in detail in step S50 of the above-mentioned aircraft skin parts detection method, they will not be repeated here.

[0073] The second determination unit 60 is used to input the test sample into the converged deep learning model, and perform forward operation on the input image through the trained deep learning model to determine whether there is an imprint defect in the image and transmit the defect information marked on the original image to the production line staff via wireless transmission; it should be noted here that since the specific determination method and process have been explained in detail in step S60 of the above-mentioned aircraft skin parts detection method, they will not be repeated here.

[0074] In addition, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium may store a program, which, when executed, includes part or all of the steps of any aircraft skin parts detection method recorded in the above method embodiment.

[0075] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0076] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0077] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0078] The above describes an exemplary flow chart of a method for implementing an aircraft skin part detection method according to an embodiment of the present invention with reference to the accompanying drawings. It should be noted that the large amount of details included in the above description are only exemplary illustrations of the present invention, rather than limitations of the present invention. In other embodiments of the present invention, the method may have more, fewer or different steps, and the order, inclusion, function, etc. of the steps may be different from those described and illustrated.

Claims

1. A method for detecting aircraft skin parts, characterized in that: include: Placing the aircraft skin part to be tested on the testing mechanism, and obtaining an image of the surface of the aircraft skin part to be tested through the testing mechanism; Preprocessing the image to obtain a preprocessed sample; wherein the preprocessed sample includes a training sample and a test sample; Converting the preprocessed samples into a two-dimensional spectrum map, eliminating high-frequency signals containing texture information, and then converting the processed spectrum map into a spatial domain image; Performing binarization and morphological operations on the texture-filtered image to determine whether there are fiber tearing defect features; Using the test samples to train the deep learning model until it converges; The test samples are input into the converged deep learning model, and the trained deep learning model is used to perform forward operations on the input images to determine whether there are print defects in the image and transmit the defect information marked on the original image to the production line staff via wireless transmission; The detection mechanism includes: The robot ground rail is connected to the manipulator, and the embedded ARM board and wireless transmission module constituting the image processing system are integrated at its rear to change the base position of the manipulator; The manipulator, installed on the robot's ground rail, is used to clamp and move the image acquisition system to collect surface images of aircraft skin parts; The image processing system consists of an ARM board and a wireless transmission module. The ARM board integrates an image processing program of a deep learning model to detect and mark surface defects of parts in images captured by industrial cameras. The wireless transmission module then transmits images with imprint defects to the industrial computer database. Parts installation platform, which is fixed to the ground and used to install composite parts to be tested; The industrial camera group, consisting of 14 industrial cameras, is installed at the clamping position of the manipulator and is used to collect surface images of aircraft skin parts; The ring light source group consists of fourteen ring light sources and is used to illuminate the surface of parts.

2. The detection method according to claim 1, characterized in that: Placing the aircraft skin part to be tested on a testing mechanism, and obtaining an image of the surface of the aircraft skin part to be tested through the testing mechanism, further comprising: The spatial position of the image acquisition system and the aircraft skin part to be tested is adjusted by translating and rotating the manipulator on the robot rail. When the acquired image has the highest clarity, the movement is stopped and the acquired image is uploaded.

3. The detection method according to claim 2, characterized in that: After collecting the surface image of the aircraft skin part at the current position, the ARM board drives the manipulator to move laterally on the robot ground rail to collect the surface image of the part at the next position until the detection work is completed.

4. The detection method according to claim 1, characterized in that: Training the deep learning model with the training samples until the deep learning model converges, further comprising: The training samples are made into a VOC format data set, the real boundary boxes of scratches in the image are manually annotated, and the xml file required by the deep learning model is automatically generated; the xml file in the VOC format is converted into a txt file required by the deep learning model; Determine the training parameters of the deep learning model, wherein the training parameters include a learning rate of 0.001, a batch size of 32, and an iteration number of 500 times; Input the training samples into the deep learning model for training; observe the loss curve during the training process to determine whether the deep learning model converges; if it converges, stop training; if not, continue training.

5. A detection device for aircraft skin parts, characterized in that: include: The acquisition unit is used to place the aircraft skin part to be tested on the detection mechanism, and obtain the image of the surface of the aircraft skin part to be tested through the detection mechanism; the detection mechanism includes: The robot ground rail is connected to the manipulator, and the embedded ARM board and wireless transmission module constituting the image processing system are integrated at its rear to change the base position of the manipulator; The manipulator, installed on the robot's ground rail, is used to clamp and move the image acquisition system to collect surface images of aircraft skin parts; The image processing system consists of an ARM board and a wireless transmission module. The ARM board integrates an image processing program of a deep learning model to detect and mark surface defects of parts in images captured by industrial cameras. The wireless transmission module then transmits images with imprint defects to the industrial computer database. Parts installation platform, which is fixed to the ground and used to install composite parts to be tested; The industrial camera group, consisting of 14 industrial cameras, is installed at the clamping position of the manipulator and is used to collect surface images of aircraft skin parts; The ring light source group consists of fourteen ring light sources and is used to illuminate the surface of parts; A preprocessing unit, which preprocesses the image to obtain a preprocessed sample; wherein the preprocessed sample includes a training sample and a test sample; A conversion unit, used to convert the preprocessed samples into a two-dimensional spectrum map, eliminate high-frequency signals containing texture information, and then convert the processed spectrum map into a spatial domain image; A first determination unit is used to perform a binarization operation and a morphological operation on the texture-filtered image to determine whether there is a fiber tear defect feature; A training unit, used to train the deep learning model using the test sample until it converges; The second determination unit is used to input the test sample into the converged deep learning model, perform forward operation on the input image through the trained deep learning model, so as to determine whether there is a print defect in the image and transmit the defect information marked on the original image to the production line staff via wireless transmission.

6. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of a method for detecting aircraft skin parts according to any one of claims 1 to 4.

Images