Aircraft skin scratch detection method based on Ghostnet architecture neural network model
By applying the neural network model of the Ghostnet architecture in the detection of aircraft skin scratches and combining computer vision technology, automatic detection of aircraft skin scratches is realized, solving the problems of high error rate and low efficiency of manual detection in the existing technology, and significantly improving detection efficiency and safety.
Patent Information
- Application Number
- CN202411737912.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-04-25
Smart Images
Figure CN119887623A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to an aircraft skin scratch detection method based on a Ghostnet architecture neural network model. Background Art
[0002] Aircraft skin is an important part of the aircraft structure. It surrounds the aircraft skeleton structure, forms the aircraft's aerodynamic shape, and plays a role in bearing and transmitting aerodynamic loads. Therefore, the skin material must not only have high strength and good plasticity, but also a smooth surface without scratches. However, the pressure increase and decrease process experienced by the aircraft during the ascent and descent process will cause the cabin's reinforcement ribs and skin to bear alternating loads, which in turn causes material fatigue around the rivets and eventually forms tiny cracks.
[0003] Therefore, aircraft skin scratch detection is an important engineering maintenance task, the purpose of which is to detect and deal with cracks before they extend to a critical length, in order to prevent structural failure and potential flight safety accidents. Common detection methods at this stage include visual inspection, non-destructive testing (such as ultrasonic testing, X-ray testing), etc., which are too dependent on manpower, and have low accuracy and efficiency in selecting scratches, and cannot meet the requirements of actual application standards.
[0004] Using convolutional neural networks to detect skin scratches can effectively extract surface features and use artificial intelligence for comparative analysis, thereby improving the efficiency of manufacturing skin scratch detection, controlling aircraft production quality, and avoiding accidents caused by skin scratches. Summary of the invention
[0005] The existing manual methods for checking aircraft skin scratches have problems such as high error rate, low efficiency and high labor cost. The Ghostnet model can use computer vision technology to realize automatic detection of aircraft skin scratches, greatly improving product inspection efficiency and safety.
[0006] The technical solution of the present invention:
[0007] In a first aspect, the present invention provides an aircraft skin scratch detection method based on a Ghostnet neural network model, the method comprising:
[0008] Step 1: Acquire the collected aircraft skin video, use the aircraft skin video as training data, wherein the training data includes various known aircraft skin scratch data and scratch-free data, and divide the aircraft skin video into frames into pictures;
[0009] Step 2: Preprocess all images and extract the directional gradient histogram dataset:
[0010] Step 3: training the improved neural network model according to the directional gradient histogram data set, and generating a perceptual hash value through perceptual hash construction;
[0011] Step 4: Obtain the video of the aircraft skin scratches to be detected, pre-process the video of the aircraft skin scratches to be detected, import it into the improved neural network model in step 3, and output the scratch detection result by comparing the Hamming distance.
[0012] Furthermore, in step 2, the image is preprocessed, specifically:
[0013] The image is subjected to wavelet transformation, and then high-pass filtering is performed by removing low-frequency components. Discrete wavelet transform DWT is used here, and the first-level decomposition formula for the image signal transformation is as follows:
[0014]
[0015] Where n represents the number of decomposition levels, D(n) is the high-frequency coefficient, f(k) is the discrete image input signal, and g(k) is the high-pass filter coefficient of the wavelet analysis filter.
[0016] Furthermore, in step 2, the image feature dataset is extracted, specifically:
[0017] The preprocessed image signal is calculated and the gradient direction of the local area of the image is counted using the directional gradient histogram to obtain the directional gradient histogram dataset G;
[0018] Use the convolution kernel to perform convolution operations on each pixel of the image to obtain the gradients in the horizontal and vertical directions. The formula is as follows:
[0019] Where D(n) is the high frequency coefficient after processing, W x represents the horizontal convolution kernel, W y Represents the convolution kernel in the vertical direction;
[0020] Combine these two gradients to get the gradient magnitude G o And the gradient direction θ, we get the directional gradient histogram dataset G.
[0021] Furthermore, the gradient amplitude G o And the gradient direction θ is specifically:
[0022]
[0023]
[0024] Furthermore, in step three, the improved neural network model is trained according to the image feature data set, specifically:
[0025] Step 31, obtain the directional gradient histogram dataset G, and obtain the basic feature map Y through conventional convolution:
[0026] Y=G*f
[0027] Among them, the directional gradient histogram dataset a is the data width, b is the data length, c is the number of data channels, and the convolution kernel c is the number of input channels, w is the size of the convolution kernel, u is the number of output channels, and the output feature map
[0028] Step 32, each channel feature map of the intrinsic feature map Y is converted into Operation generates Ghost feature map
[0029] Among them, y ir is the mth feature map in the basic feature map set Y, which is obtained by The linear change made to the nth Ghost feature map generated by the operation, where the operation refers to generating e Ghost feature maps through linear operations on each feature map in the corresponding basic feature map set;
[0030] Step 33: combine the intrinsic feature map Y obtained in step 31 and the Ghost feature map obtained in step 32 The final result X is obtained by splicing.
[0031] Furthermore, in step three, a perceptual hash value is generated by perceptual hash construction, specifically:
[0032] The perceptual hashing algorithm reduces the frequency through discrete cosine transform to achieve image compression. The formula of one-dimensional DCT transform is as follows:
[0033]
[0034]
[0035] Among them, N=u+e, N is the number of points of the original signal, X(g) is the final signal after splicing the Ghost feature map, g=0, 1, ..., N-1, each discrete cosine coefficient F(u) is the result of a linear combination of N inputs X(g) and N cosine functions, which is the surface feature signal extracted in detecting skin scratches, c(u) is used as a compensation coefficient, u=0, 1, ..., N-1.
[0036] Furthermore, step four is specifically as follows:
[0037] After preprocessing the aircraft skin scratch video to be detected, input it into the trained network model for detection to obtain the final result and name it x i ;
[0038] Select a set of ideal data sets without scratches on the surface, use the same image preprocessing method, input the data into the trained network model for detection, and obtain the final result and name it y i ;
[0039] The skin scratch detection result is output by comparing the Hamming distance dH, and the corresponding mathematical expression is:
[0040] d || (x, y) = {i|1≤i≤l, (x i ,y i )≠(0,0)}
[0041] Among them, i represents the i-th feature map in a dataset of length 1, x i is the result obtained by the user input data, y i It is the data of the aircraft skin without scratches;
[0042] When H When it is less than or equal to the preset judgment threshold, it is determined that there are no scratches on the aircraft skin surface.
[0043] Furthermore, the preset decision threshold is 3.
[0044] On the other hand, the present invention also provides an aircraft skin scratch detection device based on the GhostNet neural network model, the device is used to implement the scratch detection method, the device includes: an acquisition unit, an image preprocessing unit, a detection model building unit, a perceptual hash construction unit, and a user detection unit;
[0045] An acquisition unit is used to acquire the collected aircraft skin video, use the aircraft skin video as training data, wherein the training data includes various known aircraft skin scratch data and scratch-free data, and divide the aircraft skin video into frames into pictures;
[0046] Image preprocessing unit, used to preprocess all images and extract directional gradient histogram dataset:
[0047] A detection model building unit, used for training an improved neural network model according to the directional gradient histogram data set, and generating a perceptual hash value through a perceptual hash construction;
[0048] The user detection unit is used to obtain the video of the aircraft skin scratches to be detected, pre-process the video of the aircraft skin scratches to be detected, import it into the improved neural network model in the detection model building unit, and output the scratch detection result by comparing the Hamming distance.
[0049] The technical solution of the present invention proposes a method for detecting scratches on aircraft skin based on the Ghostnet architecture, aiming to transform the traditional manual detection method into efficient artificial intelligence detection. The phantom convolution model is adopted, and its lightweight and efficient characteristics are utilized to generate more feature maps through a small number of operations, significantly reducing model parameters and calculations. The detection method includes four modules: image preprocessing, establishment of detection model, perceptual hash construction and user detection. In the image preprocessing stage, features are extracted by methods such as wavelet transform and gradient histogram. The detection model uses the phantom model to construct a lightweight neural network to realize automatic detection of scratches. The perceptual hash construction is used to quickly compare and identify images. In the user detection stage, the scratch detection results are output through the trained model. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 This is the workflow diagram of the aircraft skin scratch detection method;
[0051] Figure 2 Phantom model structure diagram. DETAILED DESCRIPTION
[0052] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.
[0053] The present invention is an aircraft skin scratch detection method based on the Ghostnet neural network model. In practical application, the method is as follows: after dividing the video into frames, the pictures are preprocessed, the picture features are extracted, and the improved neural network model is trained using the picture data set, the model parameters are continuously optimized, the model detection performance is improved, and the perceptual hash value is generated through the perceptual hash construction to achieve rapid comparison and recognition of the image. In the application process, the user preprocesses the detection video and image data, imports them into the pre-trained model for training, and finally outputs the scratch detection result by comparing the Hamming distance.
[0054] like Figure 1 As shown, the aircraft skin scratch detection method provided by the present invention consists of four major modules, namely image preprocessing, building a detection model, perceptual hash construction, and user detection.
[0055] (1) Image preprocessing
[0056] The captured video is divided into frames, the signal f is subjected to wavelet transform, and then high-pass filtering is obtained by removing the low-frequency components. Discrete Wavelet Transformation (DWT) is used here, and the first-level decomposition formula for signal transformation is as follows:
[0057]
[0058] Where n represents the number of decomposition levels, D(n) is the high frequency coefficient, f(k) is the discrete input signal, and g(k) is the high pass filter coefficient of the wavelet analysis filter.
[0059] The processed signal is used to calculate and count the gradient direction of the local area of the image using the Histogram of Oriented Gradient (HOG) to obtain the data set G. During the calculation, the convolution kernel is used to perform a convolution operation on each pixel of the image to obtain the gradient in the horizontal and vertical directions. The formula is as follows:
[0060]
[0061] Where D(n) is the processed high-frequency coefficient, and W represents the convolution kernel in the horizontal and vertical directions.
[0062] In order to make the image clearer, the two gradients are merged to obtain the gradient magnitude G and the gradient direction θ.
[0063]
[0064] (2) Establishing a detection model
[0065] By introducing the ghost model (Ghost Module), a small number of operations are used to generate more feature maps, and the ghost model structure (Ghost bottleneck) is obtained by stacking the Ghost modules, and then a lightweight neural network GhostNet is built to reduce the amount of calculation and improve the efficiency of the algorithm. The supervised learning method is used to automatically extract the potential features and information in the image data. The corresponding neural network model is selected according to the task requirements, and the new model is applied to the skin scratch detection for training. The parameters are optimized and adjusted according to the detection results to realize the model application.
[0066] The core idea of this model is to reduce the number of parameters and the amount of computation by using a main ghost convolution kernel and several auxiliary ghost convolution kernels. The main ghost convolution kernel is used to extract the main feature information, while the auxiliary ghost convolution kernel is used to learn details and auxiliary information. Compared with traditional convolution operations, GhostNet can significantly reduce the size of the model and the use of computing resources while maintaining high performance. After s transformations, the amount of computation of ordinary convolution operations is s compared to the amount of computation of the Ghost module. Therefore, phantom convolution can significantly reduce the amount of computation.
[0067] Figure 2 This is a schematic diagram of the phantom model structure. The working steps of this module are as follows:
[0068] Step 1: First, introduce the previously calculated directional gradient histogram dataset G, and obtain the intrinsic feature maps Y through conventional convolution. The amount of calculation for this part is approximately equal to (ignoring the bias term):
[0069] Y=G*f *
[0070] Among them, the feature map (Size parameters are a: width, b: length, c: number of channels), convolution kernel (The size parameters are c: number of input channels, w: convolution kernel size, and u: number of output channels), the output feature map is
[0071] Step 2: Use ∨ to represent each channel feature map. Operation to generate Ghost feature map The GhostModule is divided into two steps to obtain the same number of feature maps as ordinary convolution. The first step is to use a small number of convolutions to reduce the amount of calculation by half; the second step is cheap operations, which generates redundant feature maps through operations with lower computational complexity, and convolutions are performed on feature maps one by one (Depth-wise convolutional).
[0072]
[0073] Among them, y r is the mth feature map in the basic feature map set Y, which is obtained by The linear change made to the nth Ghost feature map generated by the operation. Here, the operation refers to generating e Ghost feature maps through linear operations for each feature map in the corresponding basic feature map set. The last linear operation is an identity mapping to retain the inherent features.
[0074] Step 3: Combine the intrinsic feature map Y obtained in the first step and the Ghost feature map obtained in the second step The final result X (OutPut) is obtained by splicing (identity connection).
[0075] (3) Perceptual Hash Construction
[0076] While calculating the grayscale mean of the image, the pixel values are compared and grouped and marked, and finally the perceptual hash value is constructed. This process realizes the rapid comparison and recognition function between images. The perceptual hash algorithm uses DCT (discrete cosine transform) to reduce the frequency and realize image compression. The formula of one-dimensional DCT transform is as follows:
[0077]
[0078]
[0079] Where N = u + e, N is the number of points in the original signal, X(g) is the final signal after the Ghost feature map is spliced, g = 0, 1, ..., N-1. Each discrete cosine coefficient F(u) is the result of a linear combination of N inputs X(g) and N cosine functions, c(u) is the compensation coefficient, u = 0, 1, ..., N-1.
[0080] The initial signal f is processed through three modules: image preprocessing, detection model establishment, and perceptual hash construction, and then the final result F(u) is output. F(u) is the surface feature signal extracted when detecting skin scratches.
[0081] (4) User Detection
[0082] The data set to be tested is input into the trained network model for testing to obtain the final result and named x i In order to detect skin scratches, a set of ideal data sets without scratches on the surface is selected, and the same image preprocessing method is used to input the data into the trained network model for detection to obtain the final result and named y i .
[0083] Hamming distance is a distance measurement method used in data transmission error control coding. It indicates the number of different corresponding bits of two (same length) strings. The skin scratch detection result is output by comparing the Hamming distance dH. The corresponding mathematical expression is:
[0084] d || (x, y) = {i|1≤i≤l.(x i ,y i )≠(0,0)}
[0085] Among them, i represents the i-th feature map in a dataset of length l, x i is the result obtained by the user input data, y i is the aircraft skin data without scratches. The present invention sets the judgment limit to 3, that is, d H When ≤3, the model determines that there are no scratches on the aircraft skin surface.
[0086] The model has the advantages of light weight, high efficiency, and portability when applied to aircraft skin detection. GhostNet greatly reduces the number of parameters and the amount of calculation by introducing the phantom model, and can significantly reduce the size of the model and the use of computing resources while maintaining high performance. The experimental results show that compared with the existing methods, the neural network model used in the present invention greatly reduces the number of parameters and the amount of calculation, which is conducive to deployment in embedded devices. Compared with the manual detection of skin scratches, the recognition efficiency and accuracy are improved. The experimental results show that this method greatly reduces the number of parameters and the amount of calculation while maintaining high detection performance, which is conducive to deployment in embedded devices. Compared with the existing methods, the recognition efficiency and accuracy are improved, providing a new solution for aircraft skin scratch detection.
Claims
1. A method for detecting aircraft skin scratches based on a Ghostnet neural network model, characterized in that: The method comprises: Step 1: Acquire the collected aircraft skin video, use the aircraft skin video as training data, wherein the training data includes various known aircraft skin scratch data and scratch-free data, and divide the aircraft skin video into frames into pictures; Step 2: Preprocess all images and extract the directional gradient histogram dataset: Step 3: training the improved neural network model according to the directional gradient histogram data set, and generating a perceptual hash value through perceptual hash construction; Step 4: Obtain the video of the aircraft skin scratches to be detected, pre-process the video of the aircraft skin scratches to be detected, import it into the improved neural network model in step 3, and output the scratch detection result by comparing the Hamming distance.
2. According to claim 1, a method for detecting aircraft skin scratches based on a Ghostnet architecture neural network model is characterized in that: In step 2, the image is preprocessed, specifically: The image is subjected to wavelet transformation, and then high-pass filtering is performed by removing low-frequency components. Discrete wavelet transform DWT is used here, and the first-level decomposition formula for the image signal transformation is as follows: Where n represents the number of decomposition levels, D(n) is the high-frequency coefficient, f(k) is the discrete image input signal, and g(k) is the high-pass filter coefficient of the wavelet analysis filter.
3. The aircraft skin scratch detection method based on the Ghostnet architecture neural network model according to claim 2 is characterized in that: In step 2, the image feature dataset is extracted, specifically: The preprocessed image signal is calculated and the gradient direction of the local area of the image is counted using the directional gradient histogram to obtain the directional gradient histogram dataset G; Use the convolution kernel to perform convolution operations on each pixel of the image to obtain the gradients in the horizontal and vertical directions. The formula is as follows: Where D(n) is the processed high frequency coefficient, W x represents the horizontal convolution kernel, W y Represents the convolution kernel in the vertical direction; Combine these two gradients to get the gradient magnitude G o And the gradient direction θ, we get the directional gradient histogram dataset G.
4. The aircraft skin scratch detection method based on the Ghostnet architecture neural network model according to claim 3 is characterized in that: Gradient amplitude G o And the gradient direction θ is specifically:
5. The aircraft skin scratch detection method based on the Ghostnet architecture neural network model according to claim 3 is characterized in that: Step three, training the improved neural network model according to the image feature data set, specifically: Step 31, obtain the directional gradient histogram dataset G, and obtain the basic feature map Y through conventional convolution: Y=G*f Among them, the directional gradient histogram dataset a is the data width, b is the data length, c is the number of data channels, and the convolution kernel c is the number of input channels, w is the size of the convolution kernel, u is the number of output channels, and the output feature map Step 32, each channel feature map of the intrinsic feature map Y is converted into Operation generates Ghost feature map in, is the mth feature map in the basic feature map set Y, which is obtained by The linear change of the nth Ghost feature map generated by the operation, where the operation refers to generating e Ghost feature maps by linear operation on each feature map in the corresponding basic feature map set; Step 33: combine the intrinsic feature map Y obtained in step 31 and the Ghost feature map obtained in step 32 The final result X is obtained by splicing.
6. The aircraft skin scratch detection method based on the Ghostnet architecture neural network model according to claim 5 is characterized in that: Step 3: Generate a perceptual hash value through perceptual hash construction, specifically: The perceptual hashing algorithm reduces the frequency through discrete cosine transform to achieve image compression. The formula of one-dimensional DCT transform is as follows: Among them, N=u+e, N is the number of points of the original signal, X(g) is the final signal after splicing the Ghost feature map, g=0, 1, ..., N-1, each discrete cosine coefficient F(u) is the result of a linear combination of N inputs X(g) and N cosine functions, which is the surface feature signal extracted in detecting skin scratches, c(u) is used as a compensation coefficient, u=0, 1, ..., N-1.
7. The aircraft skin scratch detection method based on the Ghostnet architecture neural network model according to claim 6 is characterized in that: Step 4 is as follows: After preprocessing the aircraft skin scratch video to be detected, input it into the trained network model for detection to obtain the final result and name it x i ; Select a set of ideal data sets without scratches on the surface, use the same image preprocessing method, input the data into the trained network model for detection, and obtain the final result and name it y i ; By comparing the Hamming distance d H Output skin scratch detection results, the corresponding mathematical expression is: Among them, i represents the i-th feature map in a dataset of length l, x i is the result obtained by the user input data, y i It is the data of the aircraft skin without scratches; When H When it is less than or equal to the preset judgment threshold, it is determined that there are no scratches on the aircraft skin surface.
8. The aircraft skin scratch detection method based on the Ghostnet architecture neural network model according to claim 7 is characterized in that: The default decision threshold is 3.