A method for aircraft interior crack segmentation based on bidirectional gating mechanism

Abstract

The present invention discloses a method for segmenting cracks in an aircraft cabin based on a bidirectional gating mechanism, which comprises the following steps: manually photographing images of various parts of an aircraft cabin by an optical camera to form a training set, inputting the training set into a trained convolutional neural network to obtain a judgment result of whether there is a crack, wherein the training method of the convolutional neural network is: performing channel compression after performing multiple pooling on the original image, superimposing the bottom feature map and the high-level feature map after the channel compression to obtain a fused feature map, and finally upsampling the fused feature map to obtain a feature extraction result, thereby obtaining a trained convolutional neural network; the present invention can accurately extract cracks of an aircraft in an image, can improve the segmentation accuracy of an object boundary, and can improve the segmentation accuracy of an object with a complex structure.

Description

Technical Field

[0001] The invention belongs to the field of aircraft interior compartment nondestructive testing, and in particular relates to an aircraft interior compartment crack segmentation method based on a bidirectional gating mechanism. Background Art

[0002] Usually, airplanes have good maneuverability because they fly fast and high. However, the better the maneuverability of an airplane, the greater the force on the airplane's fuselage structure, and the more easily the structure and parts are damaged. When an airplane leaves the factory, the strength and stiffness of the structure and parts meet the requirements. However, as the use time increases, the structure is prone to general damage such as cracks, loose rivets, and skin bulging under the action of fatigue loads and the external environment; at the same time, due to operating errors or improper maintenance, some structures may suffer from collisions. In order to improve the timeliness and reliability of maintenance, this article focuses on analyzing the types of cracks in the aircraft fuselage structure and the causes of fatigue cracks.

[0003] Cracks in aircraft structural parts can be divided into three categories according to the causes of their formation and expansion: fatigue cracks, stress corrosion cracks and corrosion fatigue cracks. In addition, there are cracks caused by vibration and accidental impact. Fatigue cracks: From a mechanical point of view, various defects inside the material, such as inclusions, pores, hairlines, etc. can be regarded as cracks. From a narrow point of view, the local breakage of metal is called a crack. It is an unstable defect formed by the local breakage of some weak parts of the intact metal under stress.

[0004] Cracks in aircraft structures and parts are cracks that occur during use, and most of them are caused by fatigue. Aircraft structures are safe enough according to static strength requirements, but under the repeated action of alternating stress below the yield limit or even below the elastic limit, cracks will occur and cause damage. The fracture caused by the repeated action of alternating stress is called fatigue fracture, and the process of fatigue fracture is called fatigue.

[0005] Stress corrosion cracking: The deterioration of the body structure due to the combined effect of tensile stress and environment is called stress corrosion. When the stress is close to the yield strength, the interaction between the corrosive environment and stress often promotes the occurrence of stress corrosion cracking. Fatigue corrosion cracking: The deterioration of the structure due to the combined effect of periodic stress and environment is called fatigue corrosion, which is a special case of stress corrosion.

[0006] Causes and process of fatigue cracks: There are three sources of fatigue cracks: original cracks in the material (such as metallurgical inclusions, loose casting, etc.), cracks in the processing process (such as forging welding, heat treatment and processing, etc.) and cracks caused by accidental impact during use. Aircraft structural fatigue damage is generally divided into four stages: crack initiation (nucleation), microcrack formation, fatigue crack extension and instantaneous fracture. The crack initiation point is often called the fatigue source position, which must be at the stress concentration point of the aircraft structure, generally located on the surface of the workpiece.

[0007] When there are cracks or penetrating cracks on the surface of aircraft components, they need to be tested. Generally, eddy current testing is used for aluminum alloy structural parts. Eddy current testing is the most common method for detecting damage to aluminum alloy components. It is most suitable for detecting surface cracks of aluminum alloy components. This flaw detection method only requires simple cleaning or not cleaning of the component surface, and the flaw detection efficiency is high; in addition, this method is not affected by low temperature. Eddy current testing is a better method for detecting damage at a specific location, but eddy current testing is not suitable for checking large-area damage because the probe must scan the entire surface, which is obviously too time-consuming. In this case, it is recommended to use penetration testing.

[0008] For steel structures, when detecting small cracks on the surface of ferromagnetic materials, the magnetic particle detection method should be used first. It is the most sensitive method for detecting small cracks on the surface of ferromagnetic materials. For in-situ inspection of structural parts, it is recommended to use a portable yoke or permanent magnet for inspection, and eddy current flaw detection can also be used for inspection.

[0009] For titanium alloy structural parts, the penetration testing method should be preferred to detect surface cracks of titanium alloy structural parts. Titanium alloy is mainly used in high-temperature structural parts, and a thin layer of oil and carbon black is easily formed on the surface of these parts. Therefore, special attention should be paid to cleaning the surface of the component during flaw detection to ensure that the cracked parts are clean and easy for penetrant liquid to penetrate. For areas that cannot be cleaned, the eddy current detection method should be used.

[0010] For the detection of hidden surface cracks, low-frequency eddy current detection, X-ray detection or ultrasonic detection are usually used to detect hidden surface cracks in aluminum alloy, steel and titanium alloy structures, see the figure below.

[0011] The eddy current detection method can detect cracks in the second layer or deeper layers, and can detect oblique cracks that are not perpendicular to the surface of the component. The sensitivity of crack detection decreases rapidly with the increase of detection depth. It cannot detect layered cracks parallel to the detection surface and cannot be used on steel structures. This is because steel structures are strong magnetic materials. Due to cold processing and other reasons, their surface magnetic permeability varies greatly with position, which is easy to generate interference signals during flaw detection.

[0012] X-ray detection can detect cracks in the second layer or deeper. The sensitivity of crack detection decreases with the increase of total material thickness. X-ray source and imaging film must be installed, and the X-ray beam must be perpendicular to the surface of the component (assuming that the crack is 90 degrees to the surface). The detectability of the crack is related to its crack width. Tightly closed cracks perpendicular to the X-ray beam cannot be detected. The crack must be in a straight line with the X-ray beam. Cracks that exceed 5 degrees from the X-ray direction cannot be detected.

[0013] The ultrasonic detection method uses an oblique probe to detect oblique cracks that are not perpendicular to the surface of the component, and can detect layered cracks parallel to the surface of the component. The ultrasonic detection method has a strong ability to detect small cracks and can detect cracks 1.5mm×1.5mm below the surface. It is limited to inspecting components that the sensor can access.

[0014] The existing methods for detecting cracks in aircraft cabins have their own limitations, so it is necessary to develop a more efficient and scientific detection method. Summary of the invention

[0015] The purpose of the present invention is to provide an aircraft interior cabin crack segmentation method based on a bidirectional gating mechanism to solve the problem that existing aircraft interior cabin crack detection methods have their own limitations.

[0016] The present invention adopts the following technical solution: a method for segmenting cracks in an aircraft interior compartment based on a bidirectional gating mechanism, which comprises the following steps:

[0017] Images of various parts of the aircraft cabin are taken manually by an optical camera to form a training set, and the training set is input into the trained convolutional neural network to obtain the judgment result of whether there is a crack. The training method of the convolutional neural network is:

[0018] The original image is pooled multiple times and then channel compressed. The bottom feature map and high-level feature map after channel compression are superimposed to obtain a fused feature map. Finally, the fused feature map is upsampled to obtain the feature extraction result and a trained convolutional neural network is obtained.

[0019] Furthermore, multiple pooling of the original image consists of the following steps:

[0020] The first pooling layer is used to downsample the original image to obtain the first pooling feature map.

[0021] The second pooling layer is used to downsample the first pooling feature map to obtain the second pooling feature map.

[0022] The third pooling layer is used to downsample the second pooling feature map to obtain the third pooling feature map.

[0023] The fourth pooling layer is used to downsample the third pooling feature map to obtain the fourth pooling feature map.

[0024] Compress the channels of the fourth pooled feature map to obtain a high-level compressed feature map.

[0025] Furthermore, the method of superimposing the bottom feature map and the high-level feature map after channel compression to obtain a fused feature map consists of the following steps:

[0026] The channel of the third pooled feature map is compressed and then weighted with the high-level compressed feature map to obtain the first high-level feature map.

[0027] The second high-level feature map is obtained by compressing the channel of the second pooling feature map and performing channel weighting with the first high-level feature map.

[0028] Compress the channels of the second pooling feature map to obtain a low-level compressed feature map.

[0029] The channel of the third pooling feature map is compressed and superimposed with the low-level compressed feature map to obtain the first low-level feature map after weighting in the spatial dimension.

[0030] The fourth pooling feature map is compressed and superimposed with the first low-level feature map to obtain the second low-level feature map after weighting in the spatial dimension.

[0031] The low-layer compression feature map and the second high-layer feature map, the first low-layer feature map and the first high-layer feature map, and the high-layer compression feature map and the second low-layer feature map are fused respectively to obtain a first fused feature map, a second fused feature map, and a third fused feature map.

[0032] Furthermore, the method of upsampling the fused feature map to obtain a feature extraction result consists of the following steps:

[0033] Upsampling the third fused feature map using the first upsampling layer to obtain a first high-level feature map;

[0034] Using a second upsampling layer, upsampling the feature map obtained by merging the first high-level feature map and the second fused feature map to obtain a second high-level feature map;

[0035] Using a third upsampling layer, upsampling the feature map fused from the second high-level feature map and the first fused feature map to obtain a third high-level feature map;

[0036] The third high-level feature map is upsampled using the fourth upsampling layer to obtain a feature extraction result.

[0037] Furthermore, the first pooling layer, the second pooling layer, the third pooling layer, and the fourth pooling layer are all composed of a first standard convolutional layer, a second standard convolutional layer, a regularization layer, and a ReLU excitation layer connected in sequence. The step size of the first standard convolutional layer is 1, and the step size of the second standard convolutional layer is 2.

[0038] Furthermore, the first upsampling layer, the second upsampling layer, the third upsampling layer, and the fourth upsampling layer are all composed of a third standard convolution layer, a transposed convolution layer, and a fourth standard convolution layer connected in sequence, the step size of the third standard convolution layer is 1, and the step size of the fourth standard convolution layer is 2.

[0039] The beneficial effects of the present invention are as follows: the present invention accurately extracts cracks of an airplane in an image, and both deep semantic information and shallow spatial detail information are important. In order to effectively combine multi-level features, semantic information and spatial detail information are taken into account in different resolution features. The deeper layer transmits semantic information to help the shallower layer better retain abstract semantic information, and the shallower layer transmits spatial detail information to help the deeper layer retain position information. Therefore, the multi-level features of the present invention can cooperate with each other to produce more accurate results. Since information needs to flow bidirectionally between feature maps of different resolutions, the "top-to-bottom" and "bottom-to-top" feature pyramid structures of the present invention are used to realize this bidirectional information transmission. The prediction using this structure can correctly classify pixels around the boundary and retain the integrity and smoothness of the crack. The present invention can improve the segmentation accuracy of the boundary of an object and can improve the segmentation accuracy of an object with a complex structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of the present invention;

[0041] Figure 2 A flow chart of the CAB of the present invention;

[0042] Figure 3 FIG. 4 is a flow chart of the SAB of the present invention. DETAILED DESCRIPTION

[0043] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] The present invention discloses a method for segmenting cracks in an aircraft cabin based on a bidirectional gating mechanism. Figure 1 As shown, it consists of the following steps:

[0045] Images of various parts of the aircraft cabin are taken manually by an optical camera to form a training set, and the training set is input into the trained convolutional neural network to obtain the judgment result of whether there is a crack. The training method of the convolutional neural network is:

[0046] The original image is pooled multiple times and then channel compressed. The underlying feature map and the high-level feature map after CCB, i.e., channel compression, are superimposed to obtain a fused feature map. Finally, the fused feature map is upsampled to obtain the feature extraction result, and a trained convolutional neural network is obtained.

[0047] Among them, multiple pooling of the original image consists of the following steps:

[0048] The first pooling layer is used to downsample the original image to obtain the first pooling feature map.

[0049] The second pooling layer is used to downsample the first pooling feature map to obtain the second pooling feature map.

[0050] The third pooling layer is used to downsample the second pooling feature map to obtain the third pooling feature map.

[0051] The fourth pooling layer is used to downsample the third pooling feature map to obtain the fourth pooling feature map.

[0052] Compress the channels of the fourth pooled feature map to obtain a high-level compressed feature map.

[0053] The first pooling layer, the second pooling layer, the third pooling layer, and the fourth pooling layer are all composed of the first standard convolution layer, the second standard convolution layer, the regularization layer, and the ReLU excitation layer connected in sequence. The step size of the first standard convolution layer is 1, and the step size of the second standard convolution layer is 2.

[0054] Among them, the method of superimposing the bottom feature map and the high-level feature map after channel compression to obtain a fused feature map consists of the following steps:

[0055] Step S1: Compress the channels of the third pooled feature map and perform channel weighting on the high-level compressed feature map to obtain the first high-level feature map.

[0056] Step S2: Compress the channels of the second pooled feature map and perform channel weighting on the first high-level feature map to obtain a second high-level feature map.

[0057] Step S3: compress the channels of the second pooled feature map to obtain a low-level compressed feature map.

[0058] Step S4: Compress the channels of the third pooled feature map and superimpose them with the low-level compressed feature map to obtain the first low-level feature map after weighting in the spatial dimension.

[0059] Step S5: Compress the channels of the fourth pooled feature map and superimpose them with the first low-level feature map to obtain a second low-level feature map after weighting in the spatial dimension.

[0060] Step S6: respectively fuse the low-layer compression feature map and the second high-layer feature map, the first low-layer feature map and the first high-layer feature map, and the high-layer compression feature map and the second low-layer feature map to obtain a first fused feature map, a second fused feature map, and a third fused feature map.

[0061] Among them, when the bottom feature map and the high-level feature map after channel compression are superimposed to obtain a fused feature map, the operations of step S1-step S2 are as follows:

[0062] CAB, such as Figure 2 Shown: Feature map after upsampling of low-resolution high-level features and the next-level feature map After channel compression and cascade operation, a new feature map is generated; the new feature map is sent to the attention unit through the regularization layer and ReLU activation; the fused feature map output by the activation unit The feature map is sent to the average pooling layer, and the obtained feature map is sent to two consecutive fully connected layers and ReLU activation layers; the output of the feature map is limited to (0,1) through the sigmoid function, which is expressed as The final output is: θ 1×1 represents 1×1 convolution, Represents bitwise multiplication.

[0063] When the bottom feature map and the high-level feature map after channel compression are superimposed to obtain a fused feature map, steps S4 and S5 are operated as follows:

[0064] SAB, such as Figure 3 Shown: Feature map after downsampling of the high-resolution underlying feature map and the next-level feature map After channel compression and bitwise addition, a new feature map is generated; the new feature map is activated by ReLU and sent to the attention unit; the fusion feature output of the activation unit is Send it to two consecutive 3×3 convolutional layers to compress the number of channels to 1; use the sigmoid function to limit the output of the feature map with a channel number of 1 to (0,1), expressed as The final output is: Among them, […] represents cascade, θ 1×1 represents 1×1 convolution, Represents the bitwise multiplication, and the final output passes through a 3×3 convolutional layer to compress the number of channels to half.

[0065] Among them, the method of upsampling the fused feature map to obtain the feature extraction result consists of the following steps:

[0066] Upsampling the third fused feature map using the first upsampling layer to obtain a first high-level feature map;

[0067] Using a second upsampling layer, upsampling the feature map obtained by merging the first high-level feature map and the second fused feature map to obtain a second high-level feature map;

[0068] Using a third upsampling layer, upsampling the feature map fused from the second high-level feature map and the first fused feature map to obtain a third high-level feature map;

[0069] The third high-level feature map is upsampled using the fourth upsampling layer to obtain a feature extraction result.

[0070] The first upsampling layer, the second upsampling layer, the third upsampling layer, and the fourth upsampling layer are all composed of a third standard convolution layer, a transposed convolution layer, and a fourth standard convolution layer connected in sequence.

[0071] The first upsampling layer, the second upsampling layer, the third upsampling layer, and the fourth upsampling layer are all composed of a third standard convolutional layer, a transposed convolutional layer, and a fourth standard convolutional layer connected in sequence. The stride of the third standard convolutional layer is 1, and the stride of the fourth standard convolutional layer is 2.

[0072] The operation methods of the first upsampling layer, the second upsampling layer, the third upsampling layer, and the fourth upsampling layer are composed of the following steps:

[0073] Use 1*1 convolution to compress the corresponding fused feature map to one-fourth of the original channel to obtain the channel-compressed feature map; use the transposed convolution layer to upsample the channel-compressed feature map by two times to obtain the upsampled feature map; use 1*1 convolution to amplify the upsampled feature map by 2 to obtain the channel-amplified feature map; add the channel-amplified feature map to the corresponding fused feature map to obtain the corresponding high-level feature map.

[0074] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for segmenting aircraft interior cracks based on a bidirectional gating mechanism, characterized in that: It consists of the following steps: Images of various parts of the aircraft cabin are manually photographed by an optical camera to form a training set, and the training set is input into a trained convolutional neural network to obtain a judgment result of whether there is a crack, wherein the training method of the convolutional neural network is: The original image is pooled multiple times and then channel compressed. The bottom feature map after channel compression and the high-level feature map are superimposed to obtain a fused feature map. Finally, the fused feature map is upsampled to obtain the feature extraction result, and a trained convolutional neural network is obtained. Multiple pooling of the original image consists of the following steps: The first pooling layer is used to downsample the original image to obtain the first pooling feature map. The second pooling layer is used to downsample the first pooling feature map to obtain the second pooling feature map. The third pooling layer is used to downsample the second pooling feature map to obtain the third pooling feature map. The fourth pooling layer is used to downsample the third pooling feature map to obtain the fourth pooling feature map. Compress the channels of the fourth pooled feature map to obtain a high-level compressed feature map; The method of superimposing the bottom feature map and the high-level feature map after channel compression to obtain a fused feature map consists of the following steps: The channel of the third pooled feature map is compressed and then weighted with the high-level compressed feature map to obtain the first high-level feature map. The second high-level feature map is obtained by compressing the channel of the second pooling feature map and performing channel weighting with the first high-level feature map. Compress the channels of the second pooling feature map to obtain a low-level compressed feature map. The channel of the third pooling feature map is compressed and superimposed with the low-level compressed feature map to obtain the first low-level feature map after weighting in the spatial dimension. The fourth pooling feature map is compressed and superimposed with the first low-level feature map to obtain the second low-level feature map after weighting in the spatial dimension. The low-layer compression feature map and the second high-layer feature map, the first low-layer feature map and the first high-layer feature map, and the high-layer compression feature map and the second low-layer feature map are respectively fused to obtain a first fused feature map, a second fused feature map, and a third fused feature map; The method of upsampling the fused feature map to obtain the feature extraction result consists of the following steps: Upsampling the third fused feature map using the first upsampling layer to obtain a first high-level feature map; Using a second upsampling layer, upsampling the feature map obtained by merging the first high-level feature map and the second fused feature map to obtain a second high-level feature map; Using a third upsampling layer, upsampling the feature map fused from the second high-level feature map and the first fused feature map to obtain a third high-level feature map; The third high-level feature map is upsampled using the fourth upsampling layer to obtain a feature extraction result.

2. The method for segmenting aircraft interior cracks based on a bidirectional gating mechanism according to claim 1, characterized in that: The first pooling layer, the second pooling layer, the third pooling layer, and the fourth pooling layer are all composed of a first standard convolutional layer, a second standard convolutional layer, a regularization layer, and a ReLU excitation layer connected in sequence. The step size of the first standard convolutional layer is 1, and the step size of the second standard convolutional layer is 2.

3. The method for segmenting aircraft interior cracks based on a bidirectional gating mechanism according to claim 2, characterized in that: The first upsampling layer, the second upsampling layer, the third upsampling layer, and the fourth upsampling layer are all composed of a third standard convolutional layer, a transposed convolutional layer, and a fourth standard convolutional layer connected in sequence, the step size of the third standard convolutional layer is 1, and the step size of the fourth standard convolutional layer is 2.

Images