An aircraft cabin crack segmentation method based on adaptive non-local feature fusion
Through the adaptive non-local feature fusion method, the problem of insufficient feature fusion and edge prediction in aircraft structure crack detection in the prior art is solved, and the accurate segmentation and precise measurement of cracks in the aircraft are realized, which improves the accuracy and efficiency of detection .
Patent Information
- Application Number
- CN202111356920.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-16
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-11-16
AI Technical Summary
The existing deep learning-based crack detection technology for aircraft structures is insufficient in dealing with multi-feature, multi-semantics and complex constraint problems. In particular, the shape, size and position of cracks cannot be judged based on image classification methods, and the recognition accuracy based on the object detection method and insufficient particle size, and the prediction results at the edge of the crack are not ideal.
Adopting the crack segmentation method of in-aircraft warehousing based on adaptive non-local feature fusion, multiple downsampling and upsampling are performed through the backbone network module, and adaptive fusion is carried out in combination with multi-scale non-local feature extraction module to capture long-distance dependence and global dependence relationships. , generate adaptive fusion features with global receptive fields.
Accurate segmentation and precise measurement of cracks in the aircraft bin are achieved, which is conducive to later three-dimensional reconstruction and improves the accuracy and efficiency of detection.
Smart Images

Figure CN114092434B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of non-destructive testing, and in particular relates to an aircraft interior compartment crack segmentation method based on adaptive non-local feature fusion. Background Art
[0002] With the vigorous development of emerging industries such as information science and artificial intelligence, computer technology is increasingly being used in all aspects of national economic production, such as industry, agriculture, and modern service industries. As an important branch of computer science, machine vision technology is based on image perception devices such as cameras, and empowers computers to have the ability to observe, identify, and understand the world. Machine vision has been widely used in some dangerous working environments that are not suitable for manual operations or in scenarios where artificial vision cannot meet measurement or detection requirements.
[0003] In the field of aircraft structure nondestructive testing, periodic testing of aircraft structures has always been a key link in structural strength fatigue testing and field aircraft operation and maintenance. Aircraft structures such as wall panels, skins, wings and control surfaces are exposed to harsh external environments for a long time, and are subjected to complex alternating loads and external impacts during flight. They are prone to cracks, which pose a major safety hazard to flight safety. In response to this problem, the Federal Aviation Administration of the United States has developed a visual inspection system based on a vacuum suction cup robot, which has realized the automatic visual inspection of surface cracks in aircraft skin structures for the first time. Air New Zealand and Invert Robotics have jointly developed a remote visual inspection system for wing and control surface inspection. Technicians only need to place robots equipped with high-definition cameras in the inspection area, and can remotely control the high-definition image acquisition of the inspection area. Further combined with recognition algorithms, effective detection of surface cracks and impact damage in aircraft structures is achieved, and the inspection efficiency and reliability have been greatly improved compared to traditional visual inspection. Airbus has developed a new drone visual inspection system based on machine vision for aircraft fuselage inspection in hangars. The system is equipped with laser sensors and can carry visual inspection hardware systems to conduct inspections along preset paths. It takes only half an hour to obtain the external structure image of the entire aircraft and upload the image to the visual analysis system in real time. By comparing and matching with the original image, it can effectively detect surface damage, greatly improving the detection efficiency and accuracy of surface damage of aircraft structures.
[0004] With the continuous expansion of the application scenarios of visual inspection technology, traditional inspection technologies based on image processing and simple machine learning algorithms have obvious shortcomings in dealing with multi-feature, multi-semantic and complex constraint problems. At the beginning of this century, the rapid development of deep learning algorithms such as convolutional neural networks provided new ideas for solving the above problems. In recent years, many domestic and foreign scholars have applied deep learning, especially deep convolutional neural network methods, to the field of crack detection. Soukup et al. applied convolutional neural networks to metal surface crack detection; Guo Yaping et al. used SegNet network to realize the detection of surface cracks on workpieces; Deng et al. proposed a crack detection method combining CNN with variational mode to solve the problem of early crack detection in cables. According to the different ways of dealing with surface crack detection problems, intelligent visual inspection technology based on deep learning can be divided into three categories: image classification-based, target detection-based and pixel-level segmentation-based methods. Among them, the method based on image classification can effectively avoid the influence of low resolution and poor quality of the original image, but this type of method can only classify the presence or absence of cracks, and cannot judge the shape, size and position of the cracks; compared with the method based on image classification, the recognition accuracy and granularity of the crack detection method based on target detection are greatly improved, but important information such as the size and texture of the crack is still not available; in addition, unsatisfactory prediction results at the crack edge are also a common problem in the current segmentation tasks. The main reasons are excessive downsampling of the network, inaccurate original labels, lack of edge preservation strategies, etc. Therefore, structural crack detection based on deep learning needs to be further deepened and improved. Summary of the invention
[0005] In order to solve the above problems existing in the prior art, the present invention provides an aircraft interior crack segmentation method based on adaptive non-local feature fusion. The technical problem to be solved by the present invention is achieved through the following technical solutions:
[0006] The present invention provides an aircraft interior crack segmentation method based on adaptive non-local feature fusion, comprising:
[0007] S1: Use the backbone network module to perform multiple downsampling on the original image to obtain the first low-level feature after the first downsampling, the second low-level feature after the second downsampling, the third low-level feature after the third downsampling, and the fourth low-level feature after the fourth downsampling;
[0008] S2: using a multi-scale non-local feature extraction module to sequentially perform multi-scale context extraction, capture long-distance dependency, and adaptive fusion on the fourth low-level feature to obtain an adaptive fusion feature with global dependency;
[0009] S3: upsampling the adaptive fusion feature by using a first upsampling module to obtain a first high-level feature;
[0010] S4: using a second upsampling module to fuse and upsample the third low-level feature and the first high-level feature to obtain a second high-level feature;
[0011] S5: using a third upsampling module to fuse and upsample the second low-level feature and the second high-level feature to obtain a third high-level feature;
[0012] S6: Use a transposed convolution module to upsample the third high-level feature to obtain a final feature extraction result.
[0013] In one embodiment of the present invention, the backbone network module is composed of an improved ResNet_34 network model, and the improved ResNet_34 network model removes the fully connected layer, the average pooling layer and the last residual module of the original ResNet_34 network model.
[0014] In one embodiment of the present invention, the S2 includes:
[0015] S21: Generate context features with different scales using a multi-scale context extraction unit, wherein the multi-scale context extraction unit includes a plurality of context extraction sub-units with different scales, each context extraction sub-unit includes a cascade of multiple dilated convolutional layers with different dilation rates, for generating context features with different scales:
[0016] F k =F k-1 *θ k , k=1,2,…K,
[0017] Among them, θ k is the dilated convolution kernel of the kth dilated convolution layer in the current context extraction subunit, K represents the total number of dilated convolution layers in the current context extraction subunit, and * represents the convolution operation;
[0018] S22: using a non-local feature extraction unit to capture the long-distance dependency of the context features of different scales, and obtaining non-local feature extraction results of the context features of different scales;
[0019] S23: using an adaptive feature fusion unit to adaptively fuse the non-local feature extraction results of different scales to obtain an adaptive fusion feature.
[0020] In one embodiment of the present invention, the multi-scale context extraction unit includes four context extraction subunits of different scales, wherein the first context extraction subunit includes a dilated convolution layer, the second context extraction subunit includes two cascaded dilated convolution layers with different dilated rates, the third context extraction subunit includes three cascaded dilated convolution layers with different dilated rates, and the fourth context extraction subunit includes four cascaded dilated convolution layers with different dilated rates.
[0021] The inputs of the four context extraction sub-units are all the fourth low-level features.
[0022] In one embodiment of the present invention, the S22 includes:
[0023] S221: Using 1*1 convolution kernel W φ , W θ , W γ Convert the context features of the current scale into different matrices φ, θ and γ:
[0024]
[0025]
[0026]
[0027] in, Represents the contextual features of the current scale of the input, C represents the number of channels, H and W represent the height and width of the convolutional layer space respectively, is the number of channels of the transformed matrix;
[0028] S222: Perform matrix multiplication on φ and θ to obtain a similarity matrix of φ and θ, and regularize the similarity matrix to obtain a global attention weight:
[0029] V=soft max(φ T ×θ);
[0030] S223: Matrix multiplication of the global attention weight and γ, and then multiplication of the matrix with the context feature F at the current scale j Add together to obtain the non-local feature extraction result of the context feature of the current scale:
[0031] F j '=W α (V×γ T )+F j ;
[0032] S224: Repeat steps S221 to S223 for context features of different scales to obtain non-local feature extraction results of context features of different scales.
[0033] In one embodiment of the present invention, the S23 includes:
[0034] The adaptive feature fusion unit is used to adaptively fuse the non-local feature extraction results of different scales to obtain the adaptive fusion feature. The fusion expression is:
[0035]
[0036] Among them, W j is the adaptive factor,
[0037] In one embodiment of the present invention, the S4 includes:
[0038] S41: Perform channel compression on the first high-level feature by using 1*1 convolution to compress it to one quarter of the original channel, and obtain a feature map after channel compression;
[0039] S42: upsampling the channel-compressed feature map by two times using transposed convolution to obtain an upsampled feature map;
[0040] S43: performing channel amplification on the upsampled feature map using a 1*1 convolution, with an amplification factor of 2, to obtain a channel amplified feature map;
[0041] S44: performing positional addition on the feature map after the channel amplification and the third-level low-level feature, so as to obtain the second-level high-level feature.
[0042] Compared with the prior art, the present invention has the following beneficial effects:
[0043] 1. The aircraft cabin crack segmentation method based on adaptive non-local feature fusion of the present invention can capture the global receptive field of features and effectively solve the fusion problem of features of different scales through an adaptive feature fusion process.
[0044] 2. The aircraft interior crack segmentation method of the present invention is conducive to the accurate three-dimensional reconstruction of the aircraft interior crack in the later stage, and is conducive to the precise measurement of the aircraft interior crack.
[0045] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 It is a flow chart of a method for segmenting aircraft interior cracks based on adaptive non-local feature fusion provided by an embodiment of the present invention;
[0047] Figure 2 It is a framework diagram of the overall processing process of an aircraft interior compartment crack segmentation method based on adaptive non-local feature fusion provided by an embodiment of the present invention;
[0048] Figure 3 is a structural module diagram of a multi-scale non-local feature extraction module provided by an embodiment of the present invention;
[0049] Figure 4 is a specific structural schematic diagram of a multi-scale non-local feature extraction module provided by an embodiment of the present invention;
[0050] Figure 5 is a structural schematic diagram of a non-local feature extraction unit provided by an embodiment of the present invention;
[0051] Figure 6 is a schematic diagram of the structure of an adaptive feature fusion unit provided by an embodiment of the present invention;
[0052] Figure 7 It is a schematic diagram of the result of actual aircraft interior crack segmentation using the aircraft interior crack segmentation method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0053] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the aircraft interior compartment crack segmentation method based on adaptive non-local feature fusion proposed by the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0054] The above and other technical contents, features and effects of the present invention are clearly presented in the following detailed description of the specific implementation modes in conjunction with the accompanying drawings. Through the description of the specific implementation modes, the technical means and effects adopted by the present invention to achieve the predetermined purpose can be more deeply and specifically understood. However, the attached drawings are only for reference and explanation purposes and are not used to limit the technical solutions of the present invention.
[0055] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants are intended to cover non-exclusive inclusion, so that an article or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed. In the absence of more restrictions, the elements defined by the statement "including one..." do not exclude the existence of other identical elements in the article or device including the elements.
[0056] See also Figure 1 , Figure 11 is a flow chart of a method for segmenting cracks in an aircraft interior compartment based on adaptive non-local feature fusion provided by an embodiment of the present invention. The method for segmenting cracks in an aircraft interior compartment includes:
[0057] S1: Use the backbone network module to perform multiple downsampling on the original image to obtain the first low-level feature after one downsampling, the second low-level feature after two downsamplings, the third low-level feature after three downsamplings, and the fourth low-level feature after four downsamplings.
[0058] In this embodiment, the implementation system of the aircraft cabin crack segmentation method adopts a U-Net-style encoding-decoding structure, and selects ResNet-34 as the backbone network module (pre-trained on ImageNet). Two modifications are made to the original ResNet-34 network to form an improved ResNet_34 network model, which is suitable for the pixel-level prediction task of target extraction. The improved ResNet_34 network model removes the fully connected layer, average pooling layer and the last residual module of the original ResNet_34 network model, so the final feature map size is 1 / 16 of the input image size. Please refer to Figure 2 , Figure 2 is a framework diagram of the overall processing process of an aircraft interior crack segmentation method based on adaptive non-local feature fusion provided by an embodiment of the present invention. The structure and parameters of the backbone network module 1 of this embodiment are as follows: Figure 2 shown.
[0059] S2: using a multi-scale non-local feature extraction module (MSNB) 2 to sequentially perform multi-scale context extraction, capture long-distance dependencies, and adaptive fusion on the fourth low-level features to obtain adaptive fusion features with global dependencies.
[0060] See also Figure 3 , Figure 3 The multi-scale non-local feature extraction module 2 of this embodiment includes a multi-scale context extraction unit 21 , a non-local feature extraction unit 22 and an adaptive feature fusion unit 23 .
[0061] Specifically, the S2 includes:
[0062] S21: Generate context features with different scales using the multi-scale context extraction unit 21;
[0063] The multi-scale context extraction unit 21 of this embodiment includes a plurality of context extraction sub-units with different scales, each of which includes a cascade of multiple dilated convolutional layers with different dilation rates, for generating context features of different scales:
[0064] Fk =F k-1 *θ k , k=1,2,…K,
[0065] Among them, θ k is the dilated convolution kernel of the kth dilated convolution layer in the current context extraction subunit, K represents the total number of dilated convolution layers in the current context extraction subunit, and * represents the convolution operation.
[0066] Dilated convolution is an effective way to explicitly adjust the receptive field and control the resolution of feature maps. Multiple dilated convolutions with different dilation rates are used in the cascade to generate multi-scale contextual information.
[0067] like Figure 4 As shown, the multi-scale context extraction unit of this embodiment includes four context extraction sub-units of different scales, wherein the first context extraction sub-unit includes a hole convolution layer (such as Figure 4 The first row in : Rate 1), then the corresponding K = 1, the second context extraction subunit includes two cascaded dilated convolutional layers with different dilation rates (such as Figure 4 The second row in : Rate 1, Rate 2), then the corresponding K = 2, the third context extraction subunit includes three cascaded dilated convolutional layers with different dilation rates (such as Figure 4 The third row in : Rate 1, Rate 2, Rate 4), then the corresponding K = 3, the fourth context extraction subunit includes four cascaded dilated convolutional layers with different dilation rates (such as Figure 4 The fourth row in FIG: Rate 1, Rate 2, Rate 4, Rate 8), the corresponding K=4, together forming the multi-scale context extraction unit.
[0068] During the processing, the fourth low-level features obtained by the backbone network module through multiple downsampling are respectively input into the context extraction subunits of four scales, thereby obtaining the context features F of four scales. j , where j=1, 2, ....., J, where J represents the number of context extraction subunits. In the present embodiment, J=4, that is, a context feature is obtained respectively using the context extraction subunits of the above four scales, and a total of four context features F1, F2, F3 and F4 of different scales are obtained.
[0069] S22: Using the non-local feature extraction unit 22 to capture the long-distance dependencies of the context features of different scales, and obtain non-local feature extraction results of the context features of different scales.
[0070] See also Figure 5 , Figure 5is a structural diagram of a non-local feature extraction unit provided by an example of the present invention. Specifically, S22 includes:
[0071] S221: Using 1*1 convolution kernel W φ , W θ , W γ Convert the context features of the current scale into different matrices φ, θ and γ:
[0072]
[0073]
[0074]
[0075] in, Represents the contextual features of the current scale of the input, C represents the number of channels, H and W represent the height and width of the convolutional layer space respectively, is the number of channels of the transformed matrix;
[0076] S222: Perform matrix multiplication on φ and θ to obtain a similarity matrix of φ and θ, and regularize the similarity matrix to obtain a global attention weight:
[0077] V=soft max(φ T ×θ);
[0078] S223: Matrix multiplication of the global attention weight and γ, and then multiplication of the matrix with the context feature F at the current scale j Add together to obtain the non-local feature extraction result of the context feature of the current scale:
[0079] F j '=W α (V×γ T )+F j ;
[0080] S224: Repeat steps S221 to S223 for context features of different scales to obtain non-local feature extraction results of context features of different scales.
[0081] For the context features obtained at each scale, the non-local feature extraction unit is used to extract features to obtain non-local features at each scale. In this embodiment, four context features F1, F2, F3 and F4 at different scales are used. Therefore, according to S22, four different non-local feature extraction results F1', F2', F3' and F4' can be obtained.
[0082] It should be noted that this embodiment further includes a feature compression subunit (such as Figure 4 Compress), which is used to compress the context features output by each context extraction sub-unit, and then use the non-local feature extraction unit 22 to capture the long-distance dependencies of the context features after compression at different scales to obtain the non-local feature extraction results of the context features at different scales.
[0083] S23: Using the adaptive feature fusion unit (such as Figure 4 (as shown in NL-block in the figure) 23 adaptively fuses the non-local feature extraction results of different scales to obtain adaptive fusion features.
[0084] In this embodiment, if Figure 4 As shown, there are four different void rates that generate context features of different scales. Non-local features are extracted for these context features respectively, and finally these features are adaptively fused to generate the final features.
[0085] See also Figure 6 , Figure 6 : is a schematic diagram of the structure of an adaptive feature fusion unit provided by an example of the present invention. The fusion process of the adaptive feature fusion unit is the weighted summation of non-local features with different void rates, and the fusion process expression is:
[0086]
[0087] Among them, W j is the adaptive factor, In this embodiment, J=4.
[0088] Specifically, by cascading non-local features and performing softmax operation on the non-local features, the adaptive factor W corresponding to each non-local feature can be obtained: j .
[0089] S3: Using a first upsampling module to upsample the result of the multi-scale non-local feature extraction to obtain a first high-level feature.
[0090] The first upsampling module is composed of a transposed convolution and a regularization layer and an activation unit. After the first upsampling module 3, the feature size is 1 / 8 of the original input.
[0091] S4: using a second upsampling module to fuse and upsample the third low-level feature with the first high-level feature to obtain a second high-level feature;
[0092] The first upsampling module is composed of a transposed convolution and a regularization layer and an activation unit, and after passing through the second upsampling module 4, the feature size is 1 / 4 of the original input.
[0093] In this embodiment, step S4 includes:
[0094] S41: Perform channel compression on the first high-level feature by using 1*1 convolution to compress it to one quarter of the original channel, and obtain a feature map after channel compression;
[0095] S42: upsampling the channel-compressed feature map by two times using transposed convolution to obtain an upsampled feature map;
[0096] S43: performing channel amplification on the upsampled feature map using a 1*1 convolution, with an amplification factor of 2, to obtain a channel amplified feature map;
[0097] S44: performing positional addition on the feature map after the channel amplification and the third-level low-level feature, so as to obtain the second-level high-level feature.
[0098] S5: using a third upsampling module to fuse and upsample the second low-level feature and the second high-level feature to obtain a third high-level feature;
[0099] The third upsampling module is composed of a transposed convolution and a regularization layer and an activation unit. After the third upsampling module 5, the feature size is 1 / 2 of the original input.
[0100] S6: Use a transposed convolution module to upsample the third high-level feature to obtain a final feature extraction result.
[0101] Next, the third high-level feature is input into the transposed convolution module 6 to upsample the third high-level feature, thereby obtaining the final feature extraction result.
[0102] It should be noted that the network model for executing the aircraft cabin crack segmentation method of this embodiment is trained using pytorch1.5 on a TITAN RTX graphics card, and a truncated ResNet-34 is used as the backbone network module for pre-training. The last average pooling layer and the fully connected layer of ResNet-34 and the last residual block are removed. The backbone network module is initialized with the pre-trained weights in the ImageNet classification task, and the remaining layers are randomly initialized. Binary cross entropy (BCE) + Dice loss is used as the loss function for end-to-end training. Stochastic gradient descent (SGD) is selected as the optimizer with a batch size of 16, a fixed momentum of 0.9, and a weight decay of 0.004.
[0103] See also Figure 7 , Figure 71 is a schematic diagram of the result of actual aircraft interior crack segmentation using the aircraft interior crack segmentation method according to the embodiment of the present invention, wherein the first column is the original image, the second column is the label image manually annotated based on the original image, and the third column is the crack image segmented using the aircraft interior crack segmentation method according to the embodiment of the present invention. Figure 7 It can be seen that the method of the embodiment of the present invention can effectively predict cracks in the aircraft interior compartment.
[0104] The aircraft interior crack segmentation method based on adaptive non-local feature fusion in this embodiment can capture the global receptive field of features and effectively solve the fusion problem of features of different scales through the adaptive feature fusion process. The aircraft interior crack segmentation method is conducive to the accurate three-dimensional reconstruction of aircraft interior cracks in the later stage, and is conducive to the precise measurement of aircraft interior cracks.
[0105] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.
Claims
1. A method for aircraft interior crack segmentation based on adaptive non-local feature fusion, characterized in that: include: S1: Use the backbone network module to perform multiple downsampling on the original image to obtain the first low-level feature after the first downsampling, the second low-level feature after the second downsampling, the third low-level feature after the third downsampling, and the fourth low-level feature after the fourth downsampling; S2: using a multi-scale non-local feature extraction module to sequentially perform multi-scale context extraction, capture long-distance dependency, and adaptive fusion on the fourth low-level feature to obtain an adaptive fusion feature with global dependency; S3: upsampling the adaptive fusion feature by using a first upsampling module to obtain a first high-level feature; S4: using a second upsampling module to fuse and upsample the third low-level feature and the first high-level feature to obtain a second high-level feature; S5: using a third upsampling module to fuse and upsample the second low-level feature and the second high-level feature to obtain a third high-level feature; S6: Use a transposed convolution module to upsample the third high-level feature to obtain a final feature extraction result.
2. The method for aircraft interior crack segmentation based on adaptive non-local feature fusion according to claim 1 is characterized in that: The backbone network module is composed of an improved ResNet_34 network model, and the improved ResNet_34 network model removes the fully connected layer, the average pooling layer and the last residual module of the original ResNet_34 network model.
3. The method for aircraft interior crack segmentation based on adaptive non-local feature fusion according to claim 1 is characterized in that: The S2 includes: S21: Generate context features with different scales using a multi-scale context extraction unit, wherein the multi-scale context extraction unit includes a plurality of context extraction sub-units with different scales, each context extraction sub-unit includes a cascade of multiple dilated convolutional layers with different dilation rates, for generating context features with different scales: F k =F k-1 *i k ,k=1,2,…K, Among them, θ k is the dilated convolution kernel of the kth dilated convolution layer in the current context extraction subunit, K represents the total number of dilated convolution layers in the current context extraction subunit, and * represents the convolution operation; S22: using a non-local feature extraction unit to capture the long-distance dependency of the context features of different scales, and obtaining non-local feature extraction results of the context features of different scales; S23: using an adaptive feature fusion unit to adaptively fuse the non-local feature extraction results of different scales to obtain an adaptive fusion feature.
4. The method for segmenting aircraft interior cracks based on adaptive non-local feature fusion according to claim 3 is characterized in that: The multi-scale context extraction unit includes four context extraction subunits of different scales, wherein the first context extraction subunit includes a dilated convolution layer, the second context extraction subunit includes two cascaded dilated convolution layers with different dilated rates, the third context extraction subunit includes three cascaded dilated convolution layers with different dilated rates, and the fourth context extraction subunit includes four cascaded dilated convolution layers with different dilated rates. The inputs of the four context extraction sub-units are all the fourth low-level features.
5. The method for segmenting aircraft interior cracks based on adaptive non-local feature fusion according to claim 3, characterized in that: The S22 includes: S221: Using 1*1 convolution kernel W φ , W θ , W γ Convert the context features of the current scale into different matrices φ, θ and γ: φ=W φ (F j ), θ=W θ (F j ), γ=W γ (F j ), in, Represents the contextual features of the current scale of the input, C represents the number of channels, H and W represent the height and width of the convolutional layer space respectively, is the number of channels of the transformed matrix; S222: Perform matrix multiplication on φ and θ to obtain a similarity matrix of φ and θ, and regularize the similarity matrix to obtain a global attention weight: V=soft max(φ T ×θ); S223: Matrix multiplication of the global attention weight and γ, and then multiplication of the matrix with the context feature F at the current scale j Add together to obtain the non-local feature extraction result of the context feature of the current scale: F j '=W α (V×γ T )+F j ; S224: Repeat steps S221 to S223 for context features of different scales to obtain non-local feature extraction results of context features of different scales.
6. The method for segmenting aircraft interior cracks based on adaptive non-local feature fusion according to claim 5, characterized in that: The S23 includes: The adaptive feature fusion unit is used to adaptively fuse the non-local feature extraction results of different scales to obtain the adaptive fusion feature. The fusion expression is: Among them, W j is the adaptive factor, 7. The method for segmenting aircraft interior cracks based on adaptive non-local feature fusion according to any one of claims 1 to 6, characterized in that: The S4 includes: S41: Perform channel compression on the first high-level feature by using 1*1 convolution to compress it to one quarter of the original channel, and obtain a feature map after channel compression; S42: upsampling the channel-compressed feature map by two times using transposed convolution to obtain an upsampled feature map; S43: performing channel amplification on the upsampled feature map using a 1*1 convolution, with an amplification factor of 2, to obtain a channel amplified feature map; S44: performing positional addition on the feature map after the channel amplification and the third-level low-level feature, so as to obtain the second-level high-level feature.
Citation Information
Patent Citations
Casting defect semantic segmentation method based on feature adaptive selection
CN111652231A
Remote sensing image target extraction system and method based on deep learning
CN112287983A
Image segmentation method, apparatus, and fully convolutional network system
EP3506200A1
KR20210112869A
