Aircraft panel defect detection method, model training method and related equipment

Abstract

This application relates to the field of aviation detection technology, and discloses an aircraft panel defect detection method, a model training method and related devices. The training method includes: obtaining a sample image of an aircraft panel; inputting the sample image into a pre-trained teacher model and a student model to be trained to obtain a first feature map set, a second feature map set and a target prediction result; obtaining knowledge distillation loss information based on the local deviation and global deviation of the feature maps of the first feature map set and the second feature map set; obtaining adversarial loss information based on the feature maps of the first feature map set and the second feature map set; obtaining prediction loss information based on the target prediction result; and adjusting the network parameters of the student model according to the knowledge distillation loss information, the adversarial loss information and the prediction loss information. The embodiments of this application can improve the accuracy of aircraft panel defect detection under limited hardware conditions.

Description

Technical Field

[0001] The present application relates to the field of aviation detection technology, and in particular to an aircraft wall panel defect detection method, a model training method and related equipment. Background Art

[0002] Aircraft panels are an important part of the aircraft structure and play a key role in maintaining the strength and stability of the fuselage. Their integrity directly affects flight safety. However, during the manufacturing and assembly process of the panels, defects such as scratches, cracks and deformation may occur. These defects will have an adverse effect on the panels and seriously affect flight safety. Therefore, aircraft panel defect detection is crucial to ensure safe flight of the aircraft.

[0003] At present, aircraft siding defect detection mainly relies on manual visual inspection, which requires a lot of human resources and time, and is easily affected by subjective factors of personnel, resulting in missed detection and false detection. In recent years, with the rapid development and application of deep learning technology in many fields, the aircraft manufacturing industry has gradually adopted deep learning-based target detection technology to realize the automatic detection of siding defects. Compared with traditional methods, deep learning technology has obvious advantages in detection accuracy and efficiency. However, in industrial scenarios, edge computing usually faces the challenges of insufficient hardware computing power and limited storage space, making it difficult to deploy large models. Although small models can be deployed, their detection accuracy is often difficult to meet the requirements of industrial applications. Therefore, how to improve the detection performance of small models under limited hardware conditions has become a key issue to be solved in aircraft siding defect detection in industrial scenarios. Summary of the invention

[0004] The purpose of this application is to provide an aircraft wall panel defect detection method, a model training method and related equipment, which can improve the accuracy of aircraft wall panel defect detection under limited hardware conditions.

[0005] The present application embodiment provides a model training method, including:

[0006] Acquire a sample image obtained by photographing an aircraft siding;

[0007] The sample image is input into a pre-trained teacher model and a student model to be trained to obtain a first feature atlas, a second feature atlas and a target prediction result; the first feature atlas includes a plurality of feature maps obtained by the teacher model through multi-scale feature extraction of the sample image, the second feature atlas includes a plurality of feature maps obtained by the student model through multi-scale feature extraction of the sample image, and the target prediction result is an aircraft panel defect prediction result of the student model for the sample image;

[0008] Calculate the knowledge distillation loss based on the local and global deviations of the feature maps in both the first feature map set and the second feature map set to obtain knowledge distillation loss information;

[0009] Classify the feature maps based on the feature maps in both the first feature map set and the second feature map set, and calculate the adversarial loss based on the feature map classification results to obtain adversarial loss information;

[0010] Calculate the detection loss based on the target prediction result to obtain prediction loss information;

[0011] Adjust the network parameters of the student model based on the knowledge distillation loss information, the adversarial loss information, and the prediction loss information.

[0012] In some embodiments, the step of inputting the sample image into the pre-trained teacher model and the student model to be trained to obtain the first feature map set, the second feature map set, and the target prediction result includes:

[0013] Input the sample image into the teacher model, and obtain the feature maps extracted by multiple feature map extraction networks in the teacher model to obtain the first feature map set;

[0014] Input the sample image into the teacher model, and obtain the feature maps extracted by multiple feature map extraction networks in the student model, as well as the result of fusing and extracting multiple feature maps in the student model for aircraft panel defect prediction, to obtain the first feature map set and the target prediction result.

[0015] In some embodiments, the knowledge distillation loss information includes local distillation loss information and global distillation loss information. The step of calculating the knowledge distillation loss based on the local and global deviations of the feature maps in both the first feature map set and the second feature map set to obtain the knowledge distillation loss information includes:

[0016] Calculate the knowledge distillation loss information for the feature maps with the same scale in the first feature map set and the second feature map set to obtain the local distillation loss information;

[0017] Input the first feature map set and the second feature map set into a preset global information extraction network to extract the global information of the feature maps, and obtain a first global feature map and a second global feature map;

[0018] Calculate the knowledge distillation loss information for the first global feature map and the second global feature map to obtain the global distillation loss information.

[0019] In some embodiments, calculating knowledge distillation loss information for feature maps with the same scale in the first feature map set and the second feature map set to obtain the local distillation loss information includes:

[0020] Constructing a region separation mask and a scale mask; the region separation mask is used to distinguish the foreground and background of the feature maps in both the first feature map set and the second feature map set, and the scale mask is used to distinguish the foreground and background of the feature maps with different scales in both the first feature map set and the second feature map set according to the region separation mask;

[0021] Performing attention weight operations on the feature maps in the first feature map set to obtain spatial attention weight information and channel attention weight information;

[0022] Generating a spatial attention mask according to the spatial attention weight information, and generating a channel attention mask according to the channel attention weight information;

[0023] Calculating knowledge distillation loss information according to the region separation mask, the scale mask, the spatial attention mask and the channel attention mask to obtain the local distillation loss information.

[0024] In some embodiments, the adversarial loss information includes feature adversarial loss information and feature loss information. Classifying the feature maps based on the feature maps of both the first feature map set and the second feature map set and calculating the adversarial loss according to the feature map classification result to obtain the adversarial loss information includes:

[0025] Inputting the first feature map set and the second feature map set into a preset discriminant network for feature map classification to obtain the feature map classification results of both the teacher model and the student model;

[0026] Calculating the feature adversarial loss information according to the feature map classification results of both the teacher model and the student model;

[0027] Calculating the feature loss information according to the feature map classification result of the student model.

[0028] In some embodiments, the prediction loss information includes classification loss information and regression loss information. Calculating the detection loss based on the target prediction result to obtain the prediction loss information includes:

[0029] Calculating the classification loss information according to the predicted defect type corresponding to the target prediction result and the true defect type of the sample image, and calculating the regression loss information according to the deviation between the predicted defect type and the true defect type.

[0030] In some embodiments, adjusting the network parameters of the student model according to the knowledge distillation loss information, the adversarial loss information, and the prediction loss information includes:

[0031] Fitting the knowledge distillation loss information, the adversarial loss information, and the prediction loss information to obtain fused loss information;

[0032] Determining whether the fused loss information is within a loss threshold range;

[0033] If not, adjusting the weight parameters of the student model; returning to the step of inputting the sample image into the pre-trained teacher model and the student model to be trained to obtain the first feature map set, the second feature map set, and the target prediction result;

[0034] If so, ending the training to obtain the trained student model.

[0035] An embodiment of the present application further provides an aircraft panel defect detection method, including:

[0036] Obtaining a to-be-detected image captured from an aircraft panel;

[0037] Inputting the to-be-detected image into an aircraft panel defect detection model to obtain an aircraft panel defect detection result of the to-be-detected image; the aircraft panel defect detection model is a student model trained by the above model training method.

[0038] An embodiment of the present application further provides an electronic device, including a memory and a processor, where the memory stores a computer program, and the processor implements the above method when executing the computer program.

[0039] An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0040] Advantages of the present application: Based on the adversarial distillation method for model training, the sample images obtained by photographing the aircraft panel are input into a pre-trained teacher model and a student model to be trained, so as to obtain a first feature map set from the teacher model, a second feature map set and a target prediction result from the student model. Among them, both the first feature map set and the second feature map set include multiple feature maps obtained by performing multi-scale feature extraction on the sample images. The target prediction result is the prediction result of the aircraft panel defect of the student model for the sample image, as well as the first feature map set, the second feature map set and the target prediction result. The knowledge distillation loss information, the adversarial loss information and the prediction loss information of the student model are obtained successively, and the network parameters of the student model are adjusted according to the knowledge distillation loss information, the adversarial loss information and the prediction loss information, so as to train a student model that meets the requirements. Finally, the trained student model can achieve good results and fast running speed. When the trained student model is deployed in a scenario with limited hardware conditions, it can still achieve relatively accurate aircraft panel defect detection, and can improve the accuracy of aircraft panel defect detection under limited hardware conditions. Description of the Drawings

[0041] Figure 1 It is a flowchart of the model training method provided by an embodiment of the present application.

[0042] Figure 2 It is a flowchart of the specific method of step S103 provided by an embodiment of the present application.

[0043] Figure 3 It is a flowchart of the specific method of step S104 provided by an embodiment of the present application.

[0044] Figure 4 It is a flowchart of the specific method of step S106 provided by an embodiment of the present application.

[0045] Figure 5 It is a flowchart of the aircraft panel defect detection method provided by an embodiment of the present application.

[0046] Figure 6 It is a schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present application.

[0047] Figure 7 It is a schematic diagram of the scenario of the model training method provided by an embodiment of the present application. Detailed Embodiments

[0048] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] It should be noted that although the functional modules are divided in the device schematic diagram and the logical sequence is shown in the flowchart, in some cases, the steps shown can be executed in a different module division from that in the device or a different order from that in the flowchart. Terms such as "first" and "second" in the specification, claims and drawings are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0051] An embodiment of this application provides a model training method.

[0052] Figure 1 is a flowchart of the model training method provided by the embodiment of this application. Figure 7 is a schematic diagram of the scenario of the model training method provided by the embodiment of this application. Referring jointly to Figure 1 and Figure 7 , in some embodiments, the method includes but is not limited to steps S101 to S106.

[0053] It should be noted that the execution subject of the model training method provided by the embodiments of this specification can be applied to end-side devices or cloud-side devices, and this embodiment does not make any limitation in this regard.

[0054] Step S101, obtain a sample image obtained by photographing an aircraft panel.

[0055] Among them, the sample image is an image obtained by photographing a local position of the aircraft panel.

[0056] The execution subject can obtain the sample image obtained by photographing the aircraft panel, which can be to obtain the sample images obtained by photographing each local position of the aircraft panel at different time periods and / or different locations.

[0057] In some embodiments, after obtaining the sample image, preprocess the sample image. In specific implementation, it can be to accurately label the defects in the sample image using Label Me software to ensure that the bounding box of each defect is correctly covered, including operations such as scaling, cropping, horizontal flipping, contrast enhancement, etc., and then obtain the labeled sample image, and perform an image enhancement operation on the labeled sample image to obtain a sample image set composed of the sample images after image enhancement.

[0058] Step S102, input the sample image into a pre-trained teacher model and a student model to be trained to obtain a first feature map set, a second feature map set and a target prediction result.

[0059] The first feature map set contains multiple feature maps obtained by the teacher model through multi-scale feature extraction of the sample image, the second feature map set contains multiple feature maps obtained by the student model through multi-scale feature extraction of the sample image, and the target prediction result is the prediction result of the student model for the aircraft panel defect of the sample image.

[0060] It can be understood that the teacher model and the student model are independent models. Relatively speaking, the teacher model has good performance but a large number of parameters and slow running speed, while the student model has relatively poor performance but fewer parameters and fast running speed. The teacher model and the student model respectively perform multi-scale feature extraction on the sample image to generate multiple feature maps with different scale features, and perform aircraft panel defect prediction based on the extracted feature maps to obtain the corresponding prediction results of the aircraft panel defects for the sample image.

[0061] In some embodiments, after the execution entity inputs the sample image into the pre-trained teacher model and the student model to be trained, it obtains the feature maps output by each feature map extraction network of both the teacher model and the student model through multi-scale feature extraction, and obtains the corresponding prediction result of the aircraft panel defect for the sample image by the student model based on the extracted feature maps, so as to obtain the first feature map set, the second feature map set and the target prediction result.

[0062] Step S103, calculate the knowledge distillation loss based on the local deviation and global deviation of the feature maps in the first feature map set and the second feature map set to obtain the knowledge distillation loss information.

[0063] In some embodiments, the execution entity calculates the knowledge distillation loss based on the local deviation and global deviation of the feature maps in the first feature map set and the second feature map set, and uses the calculated knowledge distillation loss information to perform knowledge distillation training on the student model by the teacher model.

[0064] In specific implementation, the execution entity calculates the feature deviation of the feature maps with the same scale in the first feature map set and the second feature map set to obtain the local deviation of the feature maps in the first feature map set and the second feature map set. By respectively performing global feature extraction on the feature maps in the first feature map set and the second feature map set and calculating the feature deviation, the global deviation of the feature maps in the first feature map set and the second feature map set is obtained. Based on the local deviation and global deviation of the feature maps in the first feature map set and the second feature map set, using the local feature information and global feature information of the feature maps in the first feature map set as knowledge, the knowledge distillation loss is calculated to obtain the knowledge distillation loss information.

[0065] Step S104: Classify the feature maps based on the feature maps of the first feature map set and the second feature map set, and calculate the adversarial loss based on the feature map classification results to obtain adversarial loss information.

[0066] In some embodiments, the execution entity classifies the feature maps of both the first feature map set and the second feature map set to obtain corresponding feature map classification results, and calculates the adversarial loss based on the feature map classification results to obtain adversarial loss information, so as to perform adversarial training on the student model using the teacher model by combining the calculated adversarial loss information.

[0067] In specific implementation, the execution entity respectively fuses the feature maps of both the first feature map set and the second feature map set to obtain fused feature maps, classifies the fused feature maps to obtain the feature map classification results of both the first feature map set and the second feature map set, and then calculates the adversarial loss for the feature map classification results using the preset adversarial loss information to obtain adversarial loss information.

[0068] Step S105: Calculate the detection loss based on the target prediction result to obtain prediction loss information.

[0069] In some embodiments, after the student model predicts the defects of the aircraft panel for the sample image and obtains the target prediction result, the execution entity compares the predicted defect type corresponding to the target prediction result with the true defect type of the corresponding sample image, and calculates the detection loss based on the comparison result to obtain prediction loss information representing the deviation between the target prediction result and the true defect type.

[0070] Step S106: Adjust the network parameters of the student model according to the knowledge distillation loss information, adversarial loss information, and prediction loss information.

[0071] In some embodiments, the execution entity determines the error between the student model and the teacher model during multi-scale feature extraction based on the knowledge distillation loss information, adversarial loss information, and prediction loss information, so as to optimize the parameters of the student network, make the error smaller and smaller, and enable the student model to imitate the teacher model selectively during multi-scale feature extraction, achieving the goal that the effects of the student model and the teacher model are similar. Since the student model has fewer parameters and the characteristic of fast running speed, a student model with good output effect and fast running speed can be obtained.

[0072] Optionally, the network parameters of the student model can be adjusted by integrating the knowledge distillation loss information, adversarial loss information, and prediction loss information. For example, calculate the sum of the knowledge distillation loss information, adversarial loss information, and prediction loss information to obtain fused loss information.

[0073] In the above embodiments, the method of adversarial distillation training is adopted. The multi-scale feature extraction results of the teacher model are used to supervise the training of the student model. The multi-scale feature extraction results of the teacher model serve as both knowledge and adversarial samples, enabling the multi-scale feature extraction results of the student model to be close to those of the teacher model and better cope with potential attacks and challenges. Ultimately, the student model can achieve the purpose of good performance and fast running speed, so as to improve the accuracy of aircraft panel defect detection under limited hardware conditions. Among them, adversarial distillation training is a method that combines adversarial training and knowledge distillation training, aiming to improve the sampling speed of the diffusion model while maintaining high sampling fidelity. In knowledge distillation, the parameters of the teacher model are fixed, and the multi-scale feature extraction results of the teacher model can be used as knowledge to supervise the training of the student model and optimize the network parameters of the student model. In adversarial training, the multi-scale feature extraction results of the teacher model are used as adversarial samples to train the student model, enabling the student model to better cope with potential attacks and challenges, thereby improving its accuracy and stability.

[0074] In some embodiments, the above step S102 specifically includes: inputting the sample image into the teacher model, obtaining the feature maps extracted by multiple feature map extraction networks in the teacher model to obtain a first feature map set; inputting the sample image into the teacher model, obtaining the feature maps extracted by multiple feature map extraction networks in the student model, and the result of fusing and extracting multiple feature maps in the student model and performing aircraft panel defect prediction to obtain a first feature map set and a target prediction result.

[0075] In specific implementation, the teacher model and the student model respectively perform multi-scale feature extraction on the input sample image. During the multi-scale feature extraction process, multiple feature maps with different feature scales are generated to obtain a first feature map set and a second feature map set. Then, the extracted feature maps are fused to obtain a global feature map obtained by fusing the feature maps of the teacher model and the student model. Next, classification processing and linear regression processing are performed on the global feature map to obtain the prediction results of the teacher model and the student model for the sample image, and the prediction result of the student model for the sample image is the target prediction result.

[0076] In a specific embodiment, the network structures of both the teacher model and the student model respectively include a backbone network, a feature pyramid network, and a detection head. The backbone network consists of multiple residual blocks and can perform multi-scale feature extraction. Among them, the backbone network of the teacher model adopts a ResNet with 50 layers in depth, and the backbone network of the student model adopts a ResNet with 18 layers in depth. The feature pyramid network is used to fuse the multi-layer feature maps output by the backbone network. This structure establishes lateral connections between different feature maps to fuse the low-level high-resolution features with the high-level semantic features. Such connections help to transmit more refined information and provide a more comprehensive feature representation. The detection head is connected after the feature pyramid structure. The detection head includes two parts: a classification branch and a regression branch. The classification branch is used to output the category of the target, and the regression branch is used to predict the location of the target. The classification branch is a small fully convolutional network structure connected to each feature pyramid layer, and the regression branch attaches another small fully convolutional network to each pyramid level to restore the offset of each anchor box to the nearby real object.

[0077] Figure 2 It is a flowchart of the specific method for step S103 provided by the embodiments of the present application. In some embodiments, the knowledge distillation loss information includes local distillation loss information and global distillation loss information. Refer to Figure 2 This method includes but is not limited to steps S201 to S203.

[0078] Step S201: Calculate the knowledge distillation loss information for the feature maps with the same scale in the first feature map set and the second feature map set to obtain the local distillation loss information.

[0079] Step S202: Input the first feature map set and the second feature map set into a preset global information extraction network to perform global information extraction of the feature maps, and obtain the first global feature map and the second global feature map.

[0080] Step S203: Calculate the knowledge distillation loss information for the first global feature map and the second global feature map to obtain the global distillation loss information.

[0081] In a specific embodiment, step S201 specifically includes: constructing a region separation mask and a scale mask; performing attention weight operations on the feature maps in the first feature map set to obtain spatial attention weight information and channel attention weight information; generating a spatial attention mask based on the spatial attention weight information, and generating a channel attention mask based on the channel attention weight information; calculating knowledge distillation loss information based on the region separation mask, the scale mask, the spatial attention mask, and the channel attention mask to obtain local distillation loss information. Among them, the region separation mask is used to distinguish the foreground and background of the feature maps in both the first feature map set and the second feature map set, and the scale mask is used to distinguish the foreground and background of the feature maps with different scales in both the first feature map set and the second feature map set based on the region separation mask.

[0082] The calculation formulas for the region separation mask and the scale mask are respectively:

[0083] ,

[0084] ,

[0085] ,

[0086] where r is the ground truth box, i and j are respectively the horizontal coordinate and vertical coordinate of the feature map, is the region separation mask, is the scale mask, and are respectively the height and width of the ground truth box, H and W are respectively the width and height of the feature map, is the total number of background pixels.

[0087] The calculation formulas for the spatial attention weight information and the channel attention weight information are respectively:

[0088] ,

[0089] ,

[0090] where, is the spatial attention weight information, is the channel attention weight information, C is the total number of channels of the feature map, is the feature map with channel number c, is the feature map with horizontal coordinate and vertical coordinate being i and j respectively.

[0091] The calculation formulas for the spatial attention mask and the channel attention mask are respectively:

[0092] ,

[0093] ,

[0094] Among them, is the spatial attention mask, is the channel attention mask, and T is a hyperparameter.

[0095] The calculation formula for the local distillation loss information is:

[0096] ,

[0097] Among them, is the local distillation loss information, and are hyperparameters, is the feature map with channel number c, horizontal coordinate i and vertical coordinate j in the first feature map set, is the feature map with channel number c, horizontal coordinate i and vertical coordinate j in the second feature map set, is the feature map with channel number c, horizontal coordinate i and vertical coordinate j in the first feature map set, is the feature map with channel number c, horizontal coordinate i and vertical coordinate j in the second feature map set.

[0098] In a specific embodiment, the first feature map set and the second feature map set are input into a preset global information extraction network, and global context modeling and feature fusion are performed in the global information extraction network to obtain enhanced feature representations, namely the first global feature map and the second global feature map.

[0099] Preferably, the global information extraction network is a GC Block network.

[0100] The calculation formulas for the first global feature map and the second global feature map are respectively:

[0101] ,

[0102] ,

[0103] Among them, is the first global feature map, v is the second global feature map, is the feature map in the first feature map set, is the feature map in the second feature map set, , and are all convolution operations, Relu is the Relu activation operation, LN is the normalization operation, is the number of pixels in the feature layer, e is the natural constant, is the foreground area of the feature map in the first feature map set, is the foreground area of the feature map in the first feature map set.

[0104] The calculation formula of the global distillation loss information is:

[0105] ,

[0106] where, is the global distillation loss information, is a hyperparameter.

[0107] Figure 3 is the flowchart of the specific method of step S104 provided by the embodiment of the present application. In some embodiments, the adversarial loss information includes feature adversarial loss information and feature loss information. Refer to Figure 3 , and the method includes but is not limited to steps S301 to S303.

[0108] Step S301: Input the first feature map set and the second feature map set into a preset discriminant network to perform feature map classification, and obtain the feature map classification results of both the teacher model and the student model.

[0109] Step S302: Calculate the feature adversarial loss information according to the feature map classification results of both the teacher model and the student model.

[0110] Step S303: Calculate the feature loss information according to the feature map classification result of the student model.

[0111] In a specific embodiment, a discriminant network is pre-built. The discriminant network consists of two parts: an adaptive feature size adjustment network and a classification network. Input the first feature map set and the second feature map set into the discriminant network. First, pass through the adaptive feature size adjustment network to process the input feature map into a preset size to obtain an adaptive feature map, and then input the adaptive feature map into the classification network to obtain the feature map classification results of both the teacher model and the student model. Among them, the feature map classification result represents the defect type of the aircraft panel in the feature map.

[0112] The calculation formula of the adaptive feature map is:

[0113] ,

[0114] ,

[0115] where, is the adaptive feature map obtained by processing the feature map of the first feature map set, is the adaptive feature map obtained by processing the feature map of the second feature map set, AdaptivePool is an adaptive pooling operation, is a convolution operation.

[0116] The calculation formulas for the feature adversarial loss information and the feature loss information are as follows:

[0117] ,

[0118] ,

[0119] wherein, is the feature adversarial loss information, is the feature loss information, is the classification result of the feature map of the teacher model, is the classification result of the feature map of the student model.

[0120] In some embodiments, the above step S105 specifically includes: calculating the classification loss information according to the predicted defect type corresponding to the target prediction result and the true defect type of the sample image, and calculating the regression loss information according to the deviation between the predicted defect type and the true defect type.

[0121] The calculation formulas for the classification loss information and the regression loss information are as follows:

[0122] ,

[0123] ,

[0124] wherein, is the classification loss information, is the regression loss information, and are both weight factors, p is the probability that the target prediction result is the predicted defect type, y is the probability that the target prediction result is the true defect type, and x is the deviation between the predicted defect type and the true defect type.

[0125] Figure 4 is the flowchart of the specific method of step S106 provided by the embodiments of the present application. In some embodiments, referring to Figure 4 , the method includes but is not limited to steps S401 to S404.

[0126] Step S401, fitting the knowledge distillation loss information, the adversarial loss information and the prediction loss information to obtain the fusion loss information.

[0127] Step S402, determining whether the fusion loss information is within the loss threshold range.

[0128] If not, execute step S403; if so, execute step S404.

[0129] Step S403, adjusting the weight parameters of the student model. Return to step S102.

[0130] Step S404, end the training and obtain the trained student model.

[0131] In a specific embodiment, the calculation formula of the fusion loss information is:

[0132] ,

[0133] where, is the fusion loss information.

[0134] In a specific embodiment, a loss threshold interval is preset as the training end condition. When the fusion loss information is within the loss threshold interval, end the training, and the student model obtained in the last iteration is the trained student model. When the fusion loss information is not within the loss threshold interval, adjust the weight parameters of the student model according to the deviation degree of the fusion loss information from the loss threshold interval, and re-obtain the first feature map set, the second feature map set, and the target prediction result, so that the fusion loss information gradually approaches the loss threshold interval and finally falls within the loss threshold interval during the iteration process, and then end the training to obtain the trained student model.

[0135] The embodiment of the present application provides an aircraft panel defect detection method.

[0136] Figure 5 is the flowchart of the aircraft panel defect detection method provided by the embodiment of the present application. Refer to Figure 5 , in some embodiments, the method includes but is not limited to steps S501 to S502.

[0137] Step S501, obtain the image to be detected taken of the aircraft panel.

[0138] Step S502, input the image to be detected into the aircraft panel defect detection model to obtain the aircraft panel defect detection result of the image to be detected.

[0139] Among them, the aircraft panel defect detection model is the student model trained by the model training method described in the above embodiment.

[0140] The aircraft panel defect detection method provided by the embodiments of the present application obtains a to-be-detected image captured from an aircraft panel, inputs the to-be-detected image into an aircraft panel defect detection model trained by a model training method, and the aircraft panel defect detection model predicts the aircraft panel defect type of the aircraft panel in the to-be-detected image to obtain the aircraft panel defect detection result of the to-be-detected image. Since the aircraft panel defect detection model is trained based on the above model training method, when the aircraft panel defect detection model is deployed in scenarios with insufficient hardware computing power and limited storage space, relatively accurate aircraft panel defect detection can still be achieved, and the accuracy of aircraft panel defect detection can be improved under limited hardware conditions.

[0141] Figure 6 is a block diagram of an electronic device shown according to an exemplary embodiment.

[0142] Next, refer to Figure 6 to describe the electronic device 600 according to this embodiment of the present disclosure. Figure 6 The electronic device 600 shown is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0143] As Figure 6 shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0144] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present disclosure described in the method part of this specification.

[0145] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.

[0146] The storage unit 620 may also include a program / utilities 6204 having a set (at least one) of program modules 6205. Such program modules 6205 include but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0147] The bus 630 can represent one or more of several types of bus structures, including a memory unit bus or a memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of the various bus structures.

[0148] The electronic device 600 can also communicate with one or more external devices 600' (such as a keyboard, a pointing device, a Bluetooth device, etc.), can also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or can communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 650. Moreover, the electronic device 600 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 660. The network adapter 660 can communicate with other modules of the electronic device 600 through the bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0149] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned method is implemented.

[0150] The aircraft panel defect detection method, model training method and related devices provided by the embodiments of the present application are based on the adversarial distillation method for model training. The sample images obtained by photographing the aircraft panel are input into a pre-trained teacher model and a student model to be trained, so as to obtain a first feature map set from the teacher model, a second feature map set and a target prediction result from the student model. Among them, both the first feature map set and the second feature map set include multiple feature maps obtained by performing multi-scale feature extraction on the sample images. The target prediction result is the aircraft panel defect prediction result of the student model for the sample images, and for the first feature map set, the second feature map set and the target prediction result, knowledge distillation loss information, adversarial loss information and prediction loss information of the student model are obtained successively, and the network parameters of the student model are adjusted according to the knowledge distillation loss information, the adversarial loss information and the prediction loss information, so as to train a student model that meets the requirements. Finally, the trained student model can achieve good effects and fast running speed. When the trained student model is deployed in a scenario with limited hardware conditions, it can still achieve relatively accurate aircraft panel defect detection, and can improve the accuracy of aircraft panel defect detection under limited hardware conditions.

[0151] Those skilled in the art can easily understand from the description of the above embodiments that the exemplary embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a portable hard drive, etc.) or on a network, including several instructions to enable a computing device (such as a personal computer, a server, or a network device, etc.) to execute the above methods according to the embodiments of the present disclosure.

[0152] The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0153] The computer-readable storage medium may include a data signal propagated in a baseband or as a part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program used by or in combination with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted by any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0154] Those skilled in the art can understand that the above-mentioned modules can be distributed in the device according to the description of the embodiments, or can be correspondingly changed and distributed in one or more devices that are uniquely different from this embodiment. The modules of the above embodiments can be combined into one module, or further split into multiple sub-modules.

[0155] The above specifically shows and describes the exemplary embodiments of the present disclosure. It should be understood that the present disclosure is not limited to the detailed structures, settings, or implementation methods described herein; on the contrary, the present disclosure is intended to cover various modifications and equivalent settings included within the spirit and scope of the appended claims.

Claims

1. A model training method, characterized in that: include: Acquire a sample image obtained by photographing an aircraft siding; The sample image is input into a pre-trained teacher model and a student model to be trained to obtain a first feature atlas, a second feature atlas and a target prediction result; the first feature atlas includes a plurality of feature maps obtained by the teacher model through multi-scale feature extraction of the sample image, the second feature atlas includes a plurality of feature maps obtained by the student model through multi-scale feature extraction of the sample image, and the target prediction result is an aircraft panel defect prediction result of the student model for the sample image; Calculate the knowledge distillation loss based on the local deviation and the global deviation of the feature graphs of the first feature graph set and the second feature graph set to obtain knowledge distillation loss information; According to the feature graphs of the first feature graph set and the second feature graph set, feature graph classification is performed and adversarial loss calculation is performed according to the feature graph classification results to obtain adversarial loss information; Calculating the detection loss based on the target prediction result to obtain prediction loss information; Adjusting the network parameters of the student model according to the knowledge distillation loss information, the adversarial loss information and the prediction loss information; The knowledge distillation loss information includes local distillation loss information and global distillation loss information. The knowledge distillation loss is calculated based on the local deviation and the global deviation of the feature graphs of the first feature graph set and the second feature graph set to obtain the knowledge distillation loss information, including: Calculate the knowledge distillation loss information on the feature maps of the same scale in the first feature map set and the second feature map set to obtain the local distillation loss information; Inputting the first feature atlas set and the second feature atlas set into a preset global information extraction network to perform feature map global information extraction to obtain a first global feature map and a second global feature map; Knowledge distillation loss information is calculated for the first global feature map and the second global feature map to obtain the global distillation loss information.

2. The model training method according to claim 1, characterized in that: The step of inputting the sample image into a pre-trained teacher model and a student model to be trained to obtain a first feature atlas, a second feature atlas and a target prediction result includes: Inputting the sample image into the teacher model, obtaining feature maps extracted by multiple feature map extraction networks in the teacher model, and obtaining the first feature map set; The sample image is input into the teacher model, and the feature maps extracted by the multiple feature map extraction networks in the student model are obtained, as well as the result of the multiple feature maps fused and extracted by the student model and aircraft panel defect prediction, to obtain the first feature map set and the target prediction result.

3. The model training method according to claim 1, characterized in that: The calculating of knowledge distillation loss information on feature maps of the same scale in the first feature map set and the second feature map set to obtain the local distillation loss information includes: Constructing a region separation mask and a scale mask; the region separation mask is used to distinguish the foreground and background of feature maps in both the first feature atlas set and the second feature atlas set, and the scale mask is used to distinguish the foreground and background of feature maps of different scales in both the first feature atlas set and the second feature atlas set based on the region separation mask; Performing an attention weight operation on the feature graph in the first feature graph set to obtain spatial attention weight information and channel attention weight information; Generate a spatial attention mask according to the spatial attention weight information, and generate a channel attention mask according to the channel attention weight information; The knowledge distillation loss information is calculated based on the region separation mask, the scale mask, the spatial attention mask and the channel attention mask to obtain the local distillation loss information.

4. The model training method according to claim 1, characterized in that: The adversarial loss information includes feature adversarial loss information and feature loss information. The adversarial loss information is obtained by performing feature map classification based on the feature maps of both the first feature map set and the second feature map set and performing adversarial loss calculation based on the feature map classification result, including: Inputting the first feature atlas set and the second feature atlas set into a preset discriminant network to perform feature map classification, and obtaining feature map classification results of both the teacher model and the student model; Calculating the feature adversarial loss information according to the feature graph classification results of the teacher model and the student model; The feature loss information is calculated based on the feature map classification result of the student model.

5. The model training method according to claim 1, characterized in that: The prediction loss information includes classification loss information and regression loss information. The detection loss calculation is performed based on the target prediction result to obtain the prediction loss information, including: The classification loss information is calculated based on the predicted defect type corresponding to the target prediction result and the actual defect type of the sample image, and the regression loss information is calculated based on the deviation between the predicted defect type and the actual defect type.

6. The model training method according to claim 1, characterized in that: The adjusting the network parameters of the student model according to the knowledge distillation loss information, the adversarial loss information and the prediction loss information includes: Fitting the knowledge distillation loss information, the adversarial loss information, and the prediction loss information to obtain fusion loss information; Determining whether the fusion loss information is within a loss threshold range; If not, adjust the weight parameters of the student model; return to the step of inputting the sample image into the pre-trained teacher model and the student model to be trained to obtain the first feature atlas, the second feature atlas and the target prediction result; If it is, end the training and get the trained student model.

7. A method for detecting defects in aircraft panels, characterized in that: include: Acquire the image to be detected obtained by photographing the aircraft wall panel; Inputting the image to be detected into an aircraft panel defect detection model to obtain an aircraft panel defect detection result of the image to be detected; The aircraft panel defect detection model is a student model trained by the model training method described in any one of claims 1 to 6.

8. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images