Aircraft skin surface damage detection method and medium

Abstract

The invention relates to an aircraft skin surface damage detection method and a medium, and relates to the technical field of computer vision. The specific method comprises the following steps of: performing down-sampling processing for multiple times and multi-scale feature extraction processing for multiple times on an aircraft skin image to be subjected to damage detection to obtain a plurality of multi-scale feature extraction processing results, and performing feature fusion processing by taking the plurality of multi-scale feature extraction processing results as input of a feature fusion sub-network. In any one-time multi-scale feature extraction, an input feature map is segmented according to channel dimensions to obtain a plurality of segmented feature maps, and then the plurality of segmented feature maps are subjected to convolution processing by using convolution layers of different sizes to obtain a plurality of convolution processing results in one-to-one correspondence; and then channel splicing processing is carried out on a plurality of convolution processing results, so that the feature representation capability of the detection model can be better enhanced, multi-scale information fusion can be more effectively carried out, and the detection speed is improved while the model detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and medium for detecting damage on the surface of an aircraft skin. Background Art

[0002] Aircraft fuselage surface damage detection can cover multiple links of aircraft production and is an important aviation safety inspection task. It can help crew maintenance personnel to find problems in time and improve corresponding maintenance methods, and maintain aircraft in a predictive manner. In recent years, with the rapid development of deep learning, aircraft fuselage damage assessment technology based on deep learning has been widely studied and applied. This technology uses images of the aircraft fuselage taken by a camera and automatically detects and identifies defects through a deep learning-based damage assessment model.

[0003] However, due to the problems of small aircraft fuselage damage defects, low contrast, and unclear inter-class differences, there are still problems such as low speed and low accuracy in aircraft skin surface damage detection. Summary of the invention

[0004] The technical problem to be solved by the present application is to provide a method and medium for detecting damage on the surface of aircraft skin, which has the characteristics of improving the detection speed while maintaining a high detection accuracy.

[0005] In a first aspect, an embodiment provides a method for detecting damage to an aircraft skin surface, comprising: Collect the aircraft skin image to be inspected; Input the aircraft skin image to be inspected into the trained damage detection network to obtain the damage detection results, including: Accepting an aircraft skin image to be inspected for damage, performing multiple downsampling processes and multiple multi-scale feature extraction processes, and obtaining multiple multi-scale feature extraction results; The results of multiple multi-scale feature processing are used as the input of the feature fusion sub-network to perform feature fusion processing; The result of feature fusion processing is used as the input of the detection head, and the damage type is obtained after damage type classification; Among them, any multi-scale feature extraction includes: The input feature map is split according to the channel dimension to obtain multiple split feature maps; The multiple feature maps after segmentation are subjected to convolution processing using convolution layers of different sizes to obtain multiple convolution processing results that correspond to each other. Perform channel splicing on multiple convolution processing results; Perform 1×1 convolution on the result after channel splicing; The multi-scale feature extraction processing result is obtained based on the 1×1 convolution processing result.

[0006] In one embodiment, the step of segmenting the input feature map according to the channel dimension includes: In the training of the damage detection network, the input feature map is compressed into a two-dimensional feature map; Calculate the norm value of each channel position in the two-dimensional feature map; The channel position corresponding to the maximum norm value is regarded as the most important position containing feature information; The channel dimension is split based on the most important position containing feature information; The method of performing convolution processing on the multiple feature maps after segmentation using convolution layers of different sizes includes: performing convolution processing on the multiple feature maps after segmentation using convolution layers of different sizes based on the most important positions containing feature information obtained.

[0007] In one embodiment, the step of calculating the norm value of each channel position in the two-dimensional feature map includes: ; in, represents the norm value of any channel i position, 1≤i≤c, c represents the number of channels; h and w represent the height and width of the input feature map respectively, j represents any row where any channel i is located, 1≤j≤ ; Represents the element value corresponding to any channel i in any row j.

[0008] In one embodiment, the input feature map is segmented according to the channel dimension to obtain a plurality of segmented feature maps, including: The input feature map is divided into four equal parts according to the channel dimension to obtain a first segmentation feature map, a second segmentation feature map, a third segmentation feature map and a fourth segmentation feature map; wherein the second segmentation feature map and the third segmentation feature map are feature maps of the intermediate channel.

[0009] In one embodiment, the multiple feature maps after segmentation are subjected to convolution processing using convolution layers of different sizes to obtain multiple convolution processing results corresponding to each other, including: The first segmentation feature map is identically connected to obtain the first scale feature extraction result; the second segmentation feature map is subjected to 3×3 convolution processing to obtain the second scale feature extraction result; the third segmentation feature map is subjected to 5×5 convolution processing to obtain the third scale feature extraction result; the fourth segmentation feature map is identically connected to obtain the fourth scale feature extraction result.

[0010] In one embodiment, the multi-scale feature extraction processing result obtained based on the 1×1 convolution processing result includes: The attention mechanism is used to process the 1×1 convolution processing results to obtain multi-scale feature extraction processing results.

[0011] In one embodiment, the attention mechanism processing includes: The 1×1 convolution processing result is pooled in the vertical dimension and the horizontal dimension respectively to obtain the vertical direction perception feature map and the horizontal direction perception feature map; The horizontal feature map is converted to obtain a vertical feature map of the same scale tensor as the vertical perception feature map, and the vertical feature map and the vertical perception feature map are concatenated to obtain a first spatial feature map; The first spatial feature map is segmented to obtain a feature map of a vertical dimension and a feature map of a horizontal dimension; Perform sigmoid activation processing on the first spatial feature map to obtain a second spatial feature map of the same scale as the first spatial feature map, and segment the second spatial feature map to obtain a first feature weight isomorphic to the feature map of the vertical dimension and a second feature weight isomorphic to the feature map of the horizontal dimension; The feature map of the vertical dimension is multiplied element by element by the first feature weight to obtain a third spatial feature map, and the feature map of the horizontal dimension is multiplied element by element by the second feature weight to obtain a fourth spatial feature map; After performing global average pooling on the 1×1 convolution processing result, the result of the global average pooling processing is multiplied element by element with the second spatial feature map to obtain a channel feature map; The 1×1 convolution processing result, the third spatial feature map, the fourth spatial feature map and the channel feature map are multiplied element by element to obtain a multi-scale feature extraction processing result.

[0012] In one embodiment, the receiving of the aircraft skin image to be damaged is subjected to multiple downsampling processes and multiple multi-scale feature extraction processes, including: An aircraft skin image to be damaged is received, and a first downsampling and a second downsampling process are performed in sequence to obtain a first feature map; and a first multi-scale feature extraction process is performed on the first feature map to obtain a second feature map; The second feature map is subjected to a third downsampling process to obtain a third feature map; the third feature map is subjected to a second multi-scale feature extraction process to obtain a fourth feature map; The fourth feature map is subjected to a fourth downsampling process to obtain a fifth feature map; the fifth feature map is subjected to a third multi-scale feature extraction process to obtain a sixth feature map; The sixth feature map is subjected to a fifth downsampling process to obtain a seventh feature map; the seventh feature map is subjected to a fourth multi-scale feature extraction process to obtain an eighth feature map; The eighth feature map is subjected to global pooling processing to obtain the ninth feature map.

[0013] In one embodiment, the method of using the multiple multi-scale feature processing results as input of the feature fusion sub-network includes: using the fourth feature map, the sixth feature map and the ninth feature map as input of the feature fusion sub-network.

[0014] In a second aspect, an embodiment provides a computer-readable storage medium, wherein the medium stores a program, and the program can be loaded by a processor to execute the aircraft skin surface damage detection method described in any one of the above embodiments.

[0015] The beneficial effects of the present invention are: In any multi-scale feature extraction, the input feature map is segmented according to the channel dimension to obtain multiple segmented feature maps, and then the multiple segmented feature maps are convolved using convolution layers of different sizes to obtain multiple convolution processing results that correspond to each other. The multiple convolution processing results are then channel-spliced, which can better enhance the feature representation capability of the detection model, more effectively perform multi-scale information fusion, and improve the detection accuracy of the model while increasing the detection speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of a damage detection network structure of an embodiment of the present application; Figure 2 This is a schematic diagram of a flow chart of a method for detecting damage to an aircraft skin surface according to an embodiment of the present application; Figure 3 This application Figure 2 A method flow diagram of an embodiment of step S10; Figure 4 It is a schematic diagram of a multi-scale feature extraction method flow in one embodiment of the present application; Figure 5 This application Figure 4 A method flow chart of an embodiment of step S100; Figure 6 This is a schematic diagram of a multi-scale feature extraction subnetwork structure of an embodiment of the present application; Figure 7 This is a schematic diagram of a flow chart of a bidirectional positioning attention processing method according to an embodiment of the present application; Figure 8 This is a schematic diagram of the bidirectional positioning attention processing sub-network structure of an embodiment of the present application. DETAILED DESCRIPTION

[0017] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0018] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0019] The serial numbers assigned to the components in this article, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning.

[0020] To facilitate the description of the inventive concept of the present application, the aircraft skin surface damage detection technology is briefly described below.

[0021] In the current aircraft skin surface damage detection technology, with the rapid development of deep learning, the aircraft fuselage damage assessment technology based on deep learning has been widely studied and applied. Among the current solutions, one solution adopts a dual-structure attention network based on ResNet (residual neural network). The network extracts multi-scale features through attention pooling technology and uses the channel attention mechanism to fuse features of different scales. In one solution, a multi-path multi-scale feature fusion module is constructed, combining RPN (region proposal network) and spatial attention mechanism for damage location and detection.

[0022] However, the applicant found in the study that for the former solution, the model complexity of the solution is relatively high, which may lead to significant computational and memory overhead when processing large-scale data. In addition, global average pooling aggregates information by calculating the average value of feature map weights, which may lead to the loss of important spatial features. Especially for detection tasks with rich details or fine-grained, attention pooling relies too much on the global information of feature maps, which may lead to the neglect of local features, thereby affecting the overall detection performance of the model. For the latter solution, RPN needs to generate a large number of candidate regions on each detection feature map, which brings high computational costs and long processing time, reducing the detection efficiency of the model. At the same time, RPN relies on predefined detection anchor boxes. If the size and proportion of the selected anchor box are inappropriate, it will directly affect the model performance and may even lead to missed detections or false detections. When faced with tiny defects, RPN's ability to generate suitable candidate areas is also insufficient, which further affects the detection accuracy.

[0023] In view of this, an embodiment of the present application provides a method and medium for detecting damage to the surface of an aircraft skin. In the specific method, the aircraft skin image to be damaged is first subjected to multiple downsampling processes and multiple multi-scale feature extraction processes to obtain multiple multi-scale feature extraction processing results, and then the multiple multi-scale feature processing results are used as the input of the feature fusion sub-network for feature fusion processing. Since in any multi-scale feature extraction, the input feature map is segmented according to the channel dimension to obtain multiple segmented feature maps, and then the multiple segmented feature maps are convoluted using convolution layers of different sizes to obtain multiple convolution processing results that correspond to each other; then the multiple convolution processing results are channel spliced, so that the detection model feature characterization capability can be better enhanced, multi-scale information fusion can be more effectively performed, and the detection accuracy of the model can be improved while the detection speed is improved.

[0024] In order to clearly explain the aircraft skin surface damage detection method, the damage detection network relied on in the detection method is first introduced below.

[0025] The damage detection network, please refer to Figure 1 , including feature extraction sub-network, feature fusion sub-network and detection head sub-network. Among them, the feature extraction sub-network is used to extract features from the input image, including multiple downsampling and multi-scale feature extraction, the feature fusion sub-network is used to fuse the features after feature extraction, and the detection head sub-network is used to accept the results of feature fusion and classify the damage type.

[0026] In the training of the damage detection network, for the images to be trained input, in one embodiment, the acquisition method includes: first, using a camera to collect aircraft skin surface damage images of different weather conditions, different lighting conditions, different angles, different types of damage and different damage stages, and selecting suitable and clear pictures to construct an original skin damage data set. For example, skin damage pictures are collected at school aprons, aviation museums, etc. Secondly, the labelimg image annotation tool is used to annotate the original images of aircraft skin surface damage collected by the camera, and the aircraft skin surface damage is selected using a bounding box (such as a rectangular box) and labeled according to the type of skin surface damage. The types of skin surface damage include scratches, rivet damage, paint peeling and rust. The label information is stored in a txt suffix file to ensure that the image and the annotated txt file correspond one to one. Finally, the original aircraft skin surface defect data set is preprocessed, including data cleaning of the original image, processing of noise, missing values ​​and duplicate data in the image; data enhancement is performed by flipping, cropping, mosaic enhancement and scaling to construct a high-quality skin damage data set and improve the performance of the detection model.

[0027] In one embodiment, we divide the constructed data set into a training set and a validation set in a ratio of 8:2, which are used for training the model, determining the model weights, and adjusting the hyperparameters in the model, respectively. Figure 1 The damage detection network architecture in the paper is combined with the loss function to train the damage detection network for detecting aircraft skin surface damage. The loss function during training includes: ; Among them, L represents the loss function, IoU is the intersection over union ratio of the predicted box and the real box, and Respectively represent the minimum width and height of the coverage box that covers both the predicted box and the real box, Represents the Euclidean distance between two points; and Represent the midpoint of the predicted box and the midpoint of the real box respectively; represents the weighting coefficient, , v is used to measure the aspect ratio similarity, , and Represent the width of the predicted box and the width of the real box respectively, and Represent the height of the predicted box and the height of the true box respectively.

[0028] Based on the above loss function, the similarity between the detection box and the true box is measured, so that the network can capture the exact shape of the skin damage more accurately.

[0029] By deploying the above-trained damage detection network on an industrial computer, the skin damage images transmitted in real time can be modeled and tested to obtain the specific damage type, quantity, location, and size. Through automated and intelligent surface damage detection methods, the causes of damage can be analyzed, making maintenance work more targeted and effectively reducing maintenance and repair costs.

[0030] Based on the above-trained damage detection network, an aircraft skin surface damage detection method provided in one embodiment of the present application includes: first, collecting an aircraft skin image to be detected, and secondly, inputting the aircraft skin image to be detected into the trained damage detection network to obtain a damage detection result. In the method of obtaining a damage detection result using a damage detection network, please refer to Figure 2 ,include: Step S10, receiving an aircraft skin image to be inspected for damage, performing multiple downsampling processes and multiple multi-scale feature extraction processes, and obtaining multiple multi-scale feature extraction results.

[0031] Please refer to Figure 3 , step S10 may include: Step S101, receiving an aircraft skin image to be damaged, performing a first downsampling and a second downsampling process in sequence to obtain a first feature map; performing a first multi-scale feature extraction process on the first feature map to obtain a second feature map.

[0032] Step S102: Perform a third downsampling process on the second feature map to obtain a third feature map; perform a second multi-scale feature extraction process on the third feature map to obtain a fourth feature map.

[0033] Step S103, performing a fourth downsampling process on the fourth feature map to obtain a fifth feature map; performing a third multi-scale feature extraction process on the fifth feature map to obtain a sixth feature map.

[0034] Step S104, performing a fifth downsampling process on the sixth feature map to obtain a seventh feature map; performing a fourth multi-scale feature extraction process on the seventh feature map to obtain an eighth feature map.

[0035] Step S105, performing global pooling processing on the eighth feature map to obtain a ninth feature map.

[0036] Based on multiple multi-scale feature extraction after multiple sampling, the network's recognition and understanding of damage feature information is further improved.

[0037] Please refer to Figure 4 , any multi-scale feature extraction includes: Step S100, segmenting the input feature map according to the channel dimension to obtain a plurality of segmented feature maps.

[0038] In order to obtain a more reasonable segmentation scheme for segmenting the input feature map according to the channel dimension, in one embodiment, please refer to Figure 5 and Figure 6 , segmentation methods include: Step S1001, in the training of the damage detection network, compress the input feature map into a two-dimensional feature map. If the original size of the input feature map is H×W×C=h×w×c, the size of the compressed two-dimensional feature map is (h×w)×c, where h×w represents the length of the compressed two-dimensional feature map, and c represents the width of the compressed two-dimensional feature map.

[0039] Step S1002, calculating the norm value of each channel position in the two-dimensional feature map.

[0040] In one embodiment, the calculation of the norm value can be expressed as: ; in, represents the norm value of any channel i position, 1≤i≤c, c represents the number of channels; h and w represent the height and width of the input feature map respectively, j represents any row where any channel i is located, 1≤j≤ ; Represents the element value corresponding to any channel i in any row j.

[0041] If the norm value of a position is greater than the norm values ​​of other positions, the norm value of this position is often more important. By calculating the norm value, the norm value of each channel can be obtained. The channel corresponding to the largest norm value among all the norm values ​​obtained is the channel with the highest degree of importance.

[0042] Step S1003: The channel position corresponding to the maximum norm value is regarded as the most important position containing feature information.

[0043] In one embodiment, it is verified by experiments that the center position is the most frequently occurring significant position in the convolution, and the weight of the center position will be higher than that of the surrounding neighboring positions.

[0044] Step S1004, dividing the channel dimension based on the most important position containing feature information.

[0045] In one embodiment, the input feature map I is divided into four equal parts according to the channel dimension to obtain a first divided feature map I1, a second divided feature map I2, a third divided feature map I3 and a fourth divided feature map I4; wherein the second divided feature map I2 and the third divided feature map I3 are feature maps of the intermediate channels.

[0046] Step S200, performing convolution processing on the multiple feature maps after segmentation using convolution layers of different sizes to obtain multiple convolution processing results that correspond to each other.

[0047] In one embodiment, based on the premise that the center position is the most frequently occurring prominent position, the first segmentation feature map I1 is identically connected to obtain the first scale feature extraction result; the second segmentation feature map I2 is subjected to 3×3 convolution processing to obtain the second scale feature extraction result ; Perform 5×5 convolution on the third segmentation feature map I3 to obtain the third scale feature extraction result ; Perform identity connection on the fourth segmentation feature map I4 to obtain the fourth scale feature extraction result.

[0048] The second segmentation feature map I2 and the third segmentation feature map I3 are subjected to 3×3 convolution and 5×5 convolution respectively to extract feature information of different scales. Standard convolution with different kernel sizes is used on half of the input channels to achieve spatial feature extraction of different scales.

[0049] Multi-scale feature extraction uses the characteristics of different receptive fields of convolutions with different kernel sizes to extract defect information at multiple scales. In addition, compared with standard convolution operations, the computational complexity of multi-scale feature extraction after channel segmentation is significantly reduced, which can optimize resource utilization and operating efficiency in industrial environments with limited computing resources and real-time requirements.

[0050] Step S300, performing channel splicing processing on multiple convolution processing results.

[0051] The multiple convolution results are concatenated to keep the shape of the output tensor unchanged.

[0052] Step S400, performing 1×1 convolution processing on the result after the channel splicing processing.

[0053] Then the 1×1 convolution processing result here can be expressed as: ; Among them, O represents the result of 1×1 convolution processing, represents a concatenation operation, and Conv represents a convolution operation.

[0054] Multi-scale feature extraction based on segmentation of channel dimensions improves the detection speed while maintaining high detection accuracy.

[0055] The 1×1 convolution processing here promotes information exchange between channels, that is, shuffling all channels so that damage features at different levels can exchange multi-scale information within the channel.

[0056] Step S500, obtaining a multi-scale feature extraction processing result based on the 1×1 convolution processing result.

[0057] In one embodiment, please refer to Figure 5 , step S500 includes: performing attention mechanism processing on the 1×1 convolution processing result to obtain a multi-scale feature extraction processing result.

[0058] The attention processing here can sample the attention processing mechanism of the prior art. In one embodiment of the present application, a new attention processing mechanism is provided, namely, a bidirectional positioning attention mechanism, which establishes the correlation between spatial information and channel information, thereby enhancing and improving the extracted defect features. And integrate the complexity of space and channel to accurately locate the spatial position of the defect target and highlight its channel attributes.

[0059] Please refer to Figure 7 and Figure 8 , Figure 8 In the figure, H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map. Represents element-by-element multiplication. The attention mechanism processing given in this application includes: Step S1000, performing pooling operations in the vertical dimension and the horizontal dimension on the 1×1 convolution processing result to obtain a vertical direction perception feature map and a horizontal direction perception feature map.

[0060] Please refer to Figure 8 , H×1×C and (H+W)×1×C represent feature maps in the vertical direction, and 1×W×C represents feature maps in the horizontal direction.

[0061] In the spatial branch, the corresponding direction-aware feature maps are obtained by pooling operations in different directions in the horizontal and vertical dimensions.

[0062] Step S2000, converting the horizontal feature map to obtain a vertical feature map of the same scale tensor as the vertical perception feature map, and concatenating the vertical feature map with the vertical perception feature map to obtain a first spatial feature map.

[0063] In one embodiment, in the conversion process of the horizontal feature map, the conversion process is performed through the permute dimension conversion function.

[0064] Through step S2000, information is exchanged along the horizontal direction and the vertical direction, so that the first spatial feature map S contains feature information from two directions.

[0065] Step S3000, dividing the first spatial feature map to obtain a feature map of a vertical dimension and a feature map of a horizontal dimension.

[0066] Step S4000, perform sigmoid activation processing on the first spatial feature map to obtain a second spatial feature map of the same scale as the first spatial feature map, split the second spatial feature map to obtain a first feature weight isomorphic to the feature map of the vertical dimension and a second feature weight isomorphic to the feature map of the horizontal dimension.

[0067] The first spatial feature map is processed with sigmoid activation to generate another second spatial feature map T of the same size. Each element in the first spatial feature map and the second spatial feature map reflects whether the object of interest exists in the corresponding row and column. This process allows our attention to more accurately locate the exact location of the object of interest, thereby helping the entire network to better identify it.

[0068] Step S5000, multiply the feature map of the vertical dimension by the first feature weight element by element to obtain a third spatial feature map, and multiply the feature map of the horizontal dimension by the second feature weight element by element to obtain a fourth spatial feature map.

[0069] Matrix multiplication is performed through weights of the same structure to refine and enhance the spatial information weights.

[0070] Step S6000: After performing global average pooling processing on the 1×1 convolution processing result, the result of the global average pooling processing is multiplied element by element with the second spatial feature map to obtain a channel feature map.

[0071] In the channel branch, a tensor Z containing global information is generated by performing global average pooling on the 1×1 convolution result. C In order to prevent the channel branch information from being isolated, the width and height direction information and the channel information are interacted by multiplying with the second spatial feature map T.

[0072] Step S7000, multiply the 1×1 convolution processing result, the third spatial feature map, the fourth spatial feature map and the channel feature map element by element to obtain a multi-scale feature extraction processing result.

[0073] Finally, the input (1×1 convolution processing result), spatial branch output (third spatial feature map and fourth spatial feature map) and channel branch output (channel feature map) are re-weighted, and different weights are assigned according to different parts of the input data to enhance the importance of features of different dimensions in the model, so that the model can focus more on information that is useful for the current task, thereby improving the detection network’s recognition and understanding of the characteristic information of aircraft fuselage skin damage.

[0074] Step S20, using the multiple multi-scale feature processing results as input of the feature fusion sub-network to perform feature fusion processing.

[0075] In one embodiment, multiple multi-scale feature processing results are used as inputs of the feature fusion sub-network, including: using the fourth feature map, the sixth feature map, and the ninth feature map as inputs of the feature fusion sub-network.

[0076] In step S30, the result of feature fusion processing is used as the input of the detection head to classify the damage types and obtain the damage detection type.

[0077] In one embodiment of the present application, a computer-readable storage medium is provided, on which a program is stored. The stored program includes a method that can be loaded by a processor and process any of the above embodiments.

[0078] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer program. When all or part of the functions in the above-mentioned embodiments are implemented by computer program, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, CD, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer program, the program can also be stored in a storage medium such as a server, another computer, disk, CD, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0079] The above specific examples are used to illustrate the present invention, which are only used to help understand the present invention and are not used to limit the present invention. For those skilled in the art to which the present invention belongs, some simple deductions, modifications or substitutions can be made according to the idea of ​​the present invention.

Claims

1. A method for detecting damage to aircraft skin surface, characterized in that: include: Collect the aircraft skin image to be inspected; Input the aircraft skin image to be inspected into the trained damage detection network to obtain the damage detection results, including: Accepting an aircraft skin image to be inspected for damage, performing multiple downsampling processes and multiple multi-scale feature extraction processes, and obtaining multiple multi-scale feature extraction results; The results of multiple multi-scale feature processing are used as the input of the feature fusion sub-network to perform feature fusion processing; The result of feature fusion processing is used as the input of the detection head, and the damage type is obtained after damage type classification; Among them, any multi-scale feature extraction includes: The input feature map is split according to the channel dimension to obtain multiple split feature maps; The multiple feature maps after segmentation are subjected to convolution processing using convolution layers of different sizes to obtain multiple convolution processing results that correspond to each other. Perform channel splicing on multiple convolution processing results; Perform 1×1 convolution on the result after channel splicing; The multi-scale feature extraction processing result is obtained based on the 1×1 convolution processing result.

2. The aircraft skin surface damage detection method according to claim 1, characterized in that: The step of dividing the input feature map according to the channel dimension includes: In the training of the damage detection network, the input feature map is compressed into a two-dimensional feature map; Calculate the norm value of each channel position in the two-dimensional feature map; The channel position corresponding to the maximum norm value is regarded as the most important position containing feature information; The channel dimension is split based on the most important position containing feature information; The method of performing convolution processing on the multiple feature maps after segmentation using convolution layers of different sizes includes: performing convolution processing on the multiple feature maps after segmentation using convolution layers of different sizes based on the most important positions containing feature information obtained.

3. The aircraft skin surface damage detection method according to claim 2, characterized in that: The calculating of the norm value of each channel position in the two-dimensional feature map includes: ； in, represents the norm value of any channel i position, 1≤i≤c, c represents the number of channels; h and w represent the height and width of the input feature map respectively, j represents any row where any channel i is located, 1≤j≤ ; Represents the element value corresponding to any channel i in any row j.

4. The aircraft skin surface damage detection method according to claim 2, characterized in that: The input feature map is segmented according to the channel dimension to obtain multiple segmented feature maps, including: The input feature map is divided into four equal parts according to the channel dimension to obtain a first segmentation feature map, a second segmentation feature map, a third segmentation feature map and a fourth segmentation feature map; wherein the second segmentation feature map and the third segmentation feature map are feature maps of the intermediate channel.

5. The aircraft skin surface damage detection method according to claim 4, characterized in that: The multiple feature maps after segmentation are subjected to convolution processing using convolution layers of different sizes to obtain multiple convolution processing results corresponding to each other, including: The first segmentation feature map is identically connected to obtain the first scale feature extraction result; the second segmentation feature map is subjected to 3×3 convolution processing to obtain the second scale feature extraction result; the third segmentation feature map is subjected to 5×5 convolution processing to obtain the third scale feature extraction result; the fourth segmentation feature map is identically connected to obtain the fourth scale feature extraction result.

6. The aircraft skin surface damage detection method according to claim 1, characterized in that: The multi-scale feature extraction processing result obtained based on the 1×1 convolution processing result includes: The attention mechanism is used to process the 1×1 convolution processing results to obtain multi-scale feature extraction processing results.

7. The aircraft skin surface damage detection method according to claim 6, characterized in that: The attention mechanism processing includes: The 1×1 convolution processing result is pooled in the vertical dimension and the horizontal dimension respectively to obtain the vertical direction perception feature map and the horizontal direction perception feature map; The horizontal feature map is converted to obtain a vertical feature map of the same scale tensor as the vertical perception feature map, and the vertical feature map and the vertical perception feature map are concatenated to obtain a first spatial feature map; The first spatial feature map is segmented to obtain a feature map of a vertical dimension and a feature map of a horizontal dimension; Perform sigmoid activation processing on the first spatial feature map to obtain a second spatial feature map of the same scale as the first spatial feature map, and segment the second spatial feature map to obtain a first feature weight isomorphic to the feature map of the vertical dimension and a second feature weight isomorphic to the feature map of the horizontal dimension; The feature map of the vertical dimension is multiplied element by element by the first feature weight to obtain a third spatial feature map, and the feature map of the horizontal dimension is multiplied element by element by the second feature weight to obtain a fourth spatial feature map; After performing global average pooling on the 1×1 convolution processing result, the result of the global average pooling processing is multiplied element by element with the second spatial feature map to obtain a channel feature map; The 1×1 convolution processing result, the third spatial feature map, the fourth spatial feature map and the channel feature map are multiplied element by element to obtain a multi-scale feature extraction processing result.

8. The aircraft skin surface damage detection method according to claim 1, characterized in that: The receiving of the aircraft skin image to be damaged and inspected, performing multiple downsampling processes and multiple multi-scale feature extraction processes, comprises: An aircraft skin image to be damaged is received, and a first downsampling and a second downsampling process are performed in sequence to obtain a first feature map; and a first multi-scale feature extraction process is performed on the first feature map to obtain a second feature map; The second feature map is subjected to a third downsampling process to obtain a third feature map; the third feature map is subjected to a second multi-scale feature extraction process to obtain a fourth feature map; The fourth feature map is subjected to a fourth downsampling process to obtain a fifth feature map; the fifth feature map is subjected to a third multi-scale feature extraction process to obtain a sixth feature map; The sixth feature map is subjected to a fifth downsampling process to obtain a seventh feature map; the seventh feature map is subjected to a fourth multi-scale feature extraction process to obtain an eighth feature map; The eighth feature map is subjected to global pooling processing to obtain the ninth feature map.

9. The aircraft skin surface damage detection method according to claim 8, characterized in that: The method of using the multiple multi-scale feature processing results as input of the feature fusion sub-network includes: using the fourth feature map, the sixth feature map and the ninth feature map as input of the feature fusion sub-network.

10. A computer-readable storage medium, characterized in that: The medium stores a program, which can be loaded by a processor and execute the aircraft skin surface damage detection method as claimed in any one of claims 1 to 9.

Images