Aircraft panel defect detection method, model training method and related equipment

Abstract

The invention relates to the technical field of aviation detection, and discloses an aircraft panel defect detection method, a model training method and related equipment. The training method comprises the following steps: acquiring a sample image obtained by shooting an aircraft panel; inputting the sample image into a pre-trained teacher model and a to-be-trained student model to obtain a first feature image set, a second feature image set and a target prediction result; obtaining knowledge distillation loss information according to local deviation and global deviation of feature maps of the first feature map set and the second feature map set; obtaining anti-loss information according to the feature maps of the first feature map set and the second feature map set; obtaining prediction loss information according to the target prediction result; and adjusting network parameters of the student model according to the knowledge distillation loss information, the confrontation loss information and the prediction loss information. According to the embodiment of the invention, the precision of aircraft panel defect detection can be improved under the limited hardware condition.

Description

Technical Field

[0001] The present application relates to the field of aviation detection technology, and in particular to an aircraft wall panel defect detection method, a model training method and related equipment. Background Art

[0002] Aircraft panels are an important part of the aircraft structure and play a key role in maintaining the strength and stability of the fuselage. Their integrity directly affects flight safety. However, during the manufacturing and assembly process of the panels, defects such as scratches, cracks and deformation may occur. These defects will have an adverse effect on the panels and seriously affect flight safety. Therefore, aircraft panel defect detection is crucial to ensure safe flight of the aircraft.

[0003] At present, aircraft siding defect detection mainly relies on manual visual inspection, which requires a lot of human resources and time, and is easily affected by subjective factors of personnel, resulting in missed detection and false detection. In recent years, with the rapid development and application of deep learning technology in many fields, the aircraft manufacturing industry has gradually adopted deep learning-based target detection technology to realize the automatic detection of siding defects. Compared with traditional methods, deep learning technology has obvious advantages in detection accuracy and efficiency. However, in industrial scenarios, edge computing usually faces the challenges of insufficient hardware computing power and limited storage space, making it difficult to deploy large models. Although small models can be deployed, their detection accuracy is often difficult to meet the requirements of industrial applications. Therefore, how to improve the detection performance of small models under limited hardware conditions has become a key issue to be solved in aircraft siding defect detection in industrial scenarios. Summary of the invention

[0004] The purpose of this application is to provide an aircraft wall panel defect detection method, a model training method and related equipment, which can improve the accuracy of aircraft wall panel defect detection under limited hardware conditions.

[0005] The present application embodiment provides a model training method, including: Acquire a sample image obtained by photographing an aircraft siding; The sample image is input into a pre-trained teacher model and a student model to be trained to obtain a first feature atlas, a second feature atlas and a target prediction result; the first feature atlas includes a plurality of feature maps obtained by the teacher model through multi-scale feature extraction of the sample image, the second feature atlas includes a plurality of feature maps obtained by the student model through multi-scale feature extraction of the sample image, and the target prediction result is an aircraft panel defect prediction result of the student model for the sample image; Calculate the knowledge distillation loss based on the local deviation and the global deviation of the feature graphs of the first feature graph set and the second feature graph set to obtain knowledge distillation loss information; According to the feature graphs of the first feature graph set and the second feature graph set, feature graph classification is performed and adversarial loss calculation is performed according to the feature graph classification results to obtain adversarial loss information; Calculating the detection loss based on the target prediction result to obtain prediction loss information; Adjust the network parameters of the student model according to the knowledge distillation loss information, the adversarial loss information and the prediction loss information.

[0006] In some embodiments, the step of inputting the sample image into a pre-trained teacher model and a student model to be trained to obtain a first feature atlas, a second feature atlas, and a target prediction result includes: Inputting the sample image into the teacher model, obtaining feature maps extracted by multiple feature map extraction networks in the teacher model, and obtaining the first feature map set; The sample image is input into the teacher model, and the feature maps extracted by the multiple feature map extraction networks in the student model are obtained, as well as the result of the multiple feature maps fused and extracted by the student model and aircraft panel defect prediction, to obtain the first feature map set and the target prediction result.

[0007] In some embodiments, the knowledge distillation loss information includes local distillation loss information and global distillation loss information, and the knowledge distillation loss is calculated based on the local deviation and the global deviation of the feature graphs of the first feature graph set and the second feature graph set to obtain the knowledge distillation loss information, including: Calculate the knowledge distillation loss information on the feature maps of the same scale in the first feature map set and the second feature map set to obtain the local distillation loss information; Inputting the first feature atlas set and the second feature atlas set into a preset global information extraction network to perform feature map global information extraction to obtain a first global feature map and a second global feature map; Knowledge distillation loss information is calculated for the first global feature map and the second global feature map to obtain the global distillation loss information.

[0008] In some embodiments, the calculating of knowledge distillation loss information on feature maps of the same scale in the first feature map set and the second feature map set to obtain the local distillation loss information includes: Constructing a region separation mask and a scale mask; the region separation mask is used to distinguish the foreground and background of feature maps in both the first feature atlas set and the second feature atlas set, and the scale mask is used to distinguish the foreground and background of feature maps of different scales in both the first feature atlas set and the second feature atlas set based on the region separation mask; Performing an attention weight operation on the feature graph in the first feature graph set to obtain spatial attention weight information and channel attention weight information; Generate a spatial attention mask according to the spatial attention weight information, and generate a channel attention mask according to the channel attention weight information; The knowledge distillation loss information is calculated based on the region separation mask, the scale mask, the spatial attention mask and the channel attention mask to obtain the local distillation loss information.

[0009] In some embodiments, the adversarial loss information includes feature adversarial loss information and feature loss information, and the feature graph classification based on the feature graphs of both the first feature graph set and the second feature graph set and the adversarial loss calculation based on the feature graph classification result are performed to obtain the adversarial loss information, including: Inputting the first feature atlas set and the second feature atlas set into a preset discriminant network to perform feature map classification, and obtaining feature map classification results of both the teacher model and the student model; Calculating the feature adversarial loss information according to the feature graph classification results of the teacher model and the student model; The feature loss information is calculated based on the feature map classification result of the student model.

[0010] In some embodiments, the prediction loss information includes classification loss information and regression loss information, and the detection loss calculation is performed based on the target prediction result to obtain the prediction loss information, including: The classification loss information is calculated based on the predicted defect type corresponding to the target prediction result and the actual defect type of the sample image, and the prediction loss information is calculated based on the deviation between the predicted defect type and the actual defect type.

[0011] In some embodiments, adjusting the network parameters of the student model according to the knowledge distillation loss information, the adversarial loss information, and the prediction loss information includes: Fitting the knowledge distillation loss information, the adversarial loss information, and the prediction loss information to obtain fusion loss information; Determining whether the fusion loss information is within a loss threshold range; If not, adjust the weight parameters of the student model; return to the step of inputting the sample image into the pre-trained teacher model and the student model to be trained to obtain the first feature atlas, the second feature atlas and the target prediction result; If it is, end the training and get the trained student model.

[0012] The present application also provides an aircraft panel defect detection method, comprising: Acquire the image to be detected obtained by photographing the aircraft wall panel; The image to be detected is input into an aircraft panel defect detection model to obtain an aircraft panel defect detection result of the image to be detected; the aircraft panel defect detection model is a student model trained by the above-mentioned model training method.

[0013] An embodiment of the present application further provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above method when executing the computer program.

[0014] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0015] The beneficial effects of the present application are as follows: model training is performed based on the adversarial distillation method, and sample images obtained by photographing aircraft sidings are input into a pre-trained teacher model and a student model to be trained, so as to obtain a first feature atlas from the teacher model and a second feature atlas and a target prediction result from the student model, wherein the first feature atlas and the second feature atlas both include a plurality of feature maps obtained by multi-scale feature extraction of the sample image, the target prediction result is the aircraft siding defect prediction result of the student model for the sample image, as well as the first feature atlas, the second feature atlas and the target prediction result, and knowledge distillation loss information, adversarial loss information and prediction loss information of the student model are successively obtained, and based on the knowledge distillation loss information, the adversarial loss information and the prediction loss information, the network parameters of the student model are adjusted to train a student model that meets the requirements, so that the trained student model can finally achieve good results and fast running speed, and when the trained student model is deployed in a scenario with limited hardware conditions, relatively accurate aircraft siding defect detection can still be achieved, and the accuracy of aircraft siding defect detection can be improved under limited hardware conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the model training method provided in the embodiment of the present application.

[0017] Figure 2 It is a flowchart of the specific method of step S103 provided in an embodiment of the present application.

[0018] Figure 3 It is a flowchart of the specific method of step S104 provided in an embodiment of the present application.

[0019] Figure 4 It is a flowchart of the specific method of step S106 provided in an embodiment of the present application.

[0020] Figure 5 It is a flow chart of the aircraft wall panel defect detection method provided in an embodiment of the present application.

[0021] Figure 6 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application.

[0022] Figure 7 It is a scenario diagram of the model training method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown can be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second" and the like in the specification, claims and drawings are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0026] An embodiment of the present application provides a model training method.

[0027] Figure 1 It is a flow chart of the model training method provided in the embodiment of the present application. Figure 7 Schematic diagram of the model training method provided in the embodiment of the present application. Figure 1 and Figure 7 In some embodiments, the method includes but is not limited to steps S101 to S106.

[0028] It should be noted that the execution entity of the model training method provided in the embodiment of this specification can be applied to terminal-side devices or cloud-side devices, and this embodiment does not limit this.

[0029] Step S101, obtaining a sample image obtained by photographing an aircraft wall panel.

[0030] The sample image is an image obtained by photographing a local position of an aircraft wall panel.

[0031] The execution subject may obtain sample images obtained by photographing the aircraft wall panel, which may be sample images obtained by photographing various local positions of the aircraft wall panel in different time periods and / or at different locations.

[0032] In some embodiments, after obtaining the sample image, the sample image is preprocessed. In a specific implementation, the defects in the sample image can be accurately labeled using Label Me software to ensure that the bounding box of each defect is correctly covered, including operations such as scaling, cropping, horizontal flipping, and contrast enhancement, thereby obtaining the labeled sample image, and performing image enhancement operations on the labeled sample image to obtain a sample image set consisting of the sample images after image enhancement.

[0033] Step S102, input the sample image into the pre-trained teacher model and the student model to be trained to obtain a first feature atlas, a second feature atlas and a target prediction result.

[0034] The first feature atlas contains multiple feature maps obtained by the teacher model through multi-scale feature extraction of sample images, and the second feature atlas contains multiple feature maps obtained by the student model through multi-scale feature extraction of sample images. The target prediction result is the aircraft panel defect prediction result of the student model for the sample image.

[0035] It can be understood that the teacher model and the student model are independent models. Relatively speaking, the teacher model has good effect, but has many parameters and runs slowly, while the student model has relatively poor effect, but has few parameters and runs quickly. The teacher model and the student model respectively extract multi-scale features from the sample image to generate multiple feature maps with different scale features, and predict the aircraft wall panel defects based on the extracted feature maps to obtain the corresponding aircraft wall panel defect prediction results for the sample image.

[0036] In some embodiments, after the execution subject inputs the sample image into the pre-trained teacher model and the student model to be trained, the feature map output by each feature map extraction network of the teacher model and the student model for multi-scale feature extraction is obtained, and the student model is used to predict the aircraft panel defects based on the extracted feature maps to obtain the corresponding aircraft panel defect prediction results for the sample image, so as to obtain the first feature map set, the second feature map set and the target prediction result.

[0037] Step S103, performing knowledge distillation loss calculation based on the local deviation and global deviation of the feature graphs of the first feature graph set and the second feature graph set to obtain knowledge distillation loss information.

[0038] In some embodiments, the execution entity calculates the knowledge distillation loss based on the local deviation and global deviation of the feature maps of the first feature map set and the second feature map set, and uses the teacher model to perform knowledge distillation training on the student model based on the calculated knowledge distillation loss information.

[0039] In the specific implementation, the execution entity calculates the feature deviation of the feature maps of the same scale of the first feature atlas and the second feature atlas to obtain the local deviation of the feature maps of the first feature atlas and the second feature atlas, extracts global features from the feature maps of the first feature atlas and the second feature atlas respectively and calculates the feature deviation to obtain the global deviation of the feature maps of the first feature atlas and the second feature atlas, and based on the local deviation and global deviation of the feature maps of the first feature atlas and the second feature atlas, uses the local feature information and global feature information of the feature maps of the first feature atlas as knowledge to calculate the knowledge distillation loss information.

[0040] Step S104, based on the feature maps of both the first feature map set and the second feature map set, feature map classification is performed and adversarial loss is calculated based on the feature map classification results to obtain adversarial loss information.

[0041] In some embodiments, the execution entity performs feature map classification on the feature maps of both the first feature map set and the second feature map set to obtain corresponding feature map classification results, and performs adversarial loss calculation based on the feature map classification results to obtain adversarial loss information, and uses the teacher model to perform adversarial training on the student model in combination with the calculated adversarial loss information.

[0042] In the specific implementation, the execution entity respectively fuses the feature maps of the first feature atlas and the second feature atlas to obtain a fused feature map, and performs feature map classification on the fused feature map to obtain feature map classification results of both the first feature atlas and the second feature atlas, and then uses preset adversarial loss information to calculate adversarial loss on the feature map classification results to obtain adversarial loss information.

[0043] Step S105, performing detection loss calculation based on the target prediction result to obtain prediction loss information.

[0044] In some embodiments, after the student model predicts aircraft panel defects for a sample image and obtains a target prediction result, the execution entity compares the predicted defect type corresponding to the target prediction result with the actual defect type of the corresponding sample image, calculates the detection loss based on the comparison result, and obtains prediction loss information that characterizes the deviation between the target prediction result and the actual defect type.

[0045] Step S106, adjusting the network parameters of the student model according to the knowledge distillation loss information, the adversarial loss information and the prediction loss information.

[0046] In some embodiments, the execution entity determines the errors between the student model and the teacher model when performing multi-scale feature extraction based on the knowledge distillation loss information, the adversarial loss information and the prediction loss information to optimize the parameters of the student network to make the error smaller and smaller. The student model can focus on imitating the teacher model when performing multi-scale feature extraction to achieve the goal of making the effect of the student model similar to that of the teacher model. Since the student model has a small number of parameters and a fast running speed, a student model with good output effect and fast running speed can be obtained.

[0047] Optionally, the network parameters of the student model can be adjusted by comprehensively using the knowledge distillation loss information, the adversarial loss information, and the prediction loss information. For example, the sum of the knowledge distillation loss information, the adversarial loss information, and the prediction loss information can be calculated to obtain the fusion loss information.

[0048] In the above embodiment, the adversarial distillation training method is adopted to supervise and train the student model through the multi-scale feature extraction results of the teacher model. The multi-scale feature extraction results of the teacher model are used as both knowledge and adversarial samples, so that the multi-scale feature extraction results of the student model are close to the multi-scale feature extraction results of the teacher model and can better cope with potential attacks and challenges, so that the student model can achieve good results and fast running speed, so as to improve the accuracy of aircraft wall panel defect detection under limited hardware conditions. Among them, adversarial distillation training is a method that combines adversarial training and knowledge distillation training, aiming to improve the sampling speed of the diffusion model while maintaining high sampling fidelity. In knowledge distillation, the parameters of the teacher model are fixed, and the multi-scale feature extraction results of the teacher model can be used as knowledge to supervise the training of the student model and optimize the network parameters of the student model. In adversarial training, the multi-scale feature extraction results of the teacher model are used as adversarial samples, and the student model is trained through adversarial samples, so that the student model can better cope with potential attacks and challenges, thereby improving its accuracy and stability.

[0049] In some embodiments, the above step S102 specifically includes: inputting the sample image into the teacher model, obtaining the feature maps extracted by multiple feature map extraction networks in the teacher model, and obtaining a first feature map set; inputting the sample image into the teacher model, obtaining the feature maps extracted by multiple feature map extraction networks in the student model, and the student model fusion extracted multiple feature maps and performing aircraft wall panel defect prediction results to obtain a first feature map set and a target prediction result.

[0050] In the specific implementation, the teacher model and the student model respectively perform multi-scale feature extraction on the input sample image, generate multiple feature maps of different feature scales in the multi-scale feature extraction process, and obtain a first feature map set and a second feature map set. The extracted feature maps are then fused to obtain a global feature map obtained by fusion of the feature maps of the teacher model and the student model. The global feature map is then classified and linearly regressed to obtain the prediction results of both the teacher model and the student model for the sample image, and the prediction result of the student model for the sample image is the target prediction result.

[0051] In a specific embodiment, the network structures of the teacher model and the student model respectively include a backbone network, a feature pyramid network and a detection head. The backbone network is composed of multiple residual blocks and can perform multi-scale feature extraction. The backbone network of the teacher model adopts a residual network with a depth of 50 layers, and the skeleton network of the student model adopts a residual network with a depth of 18 layers. The feature pyramid network is used to fuse the multi-layer feature maps output by the backbone network. The structure fuses low-level high-resolution features with high-level semantic features by establishing lateral connections between different feature maps. This connection helps to transmit more refined information and provide a more comprehensive feature representation. The detection head is connected after the feature pyramid structure. The detection head includes two parts: a classification branch and a regression branch. The classification branch is used to output the category of the target, and the regression branch is used to predict the location of the target. The classification branch is a small full convolutional network structure connected to each feature pyramid layer. The regression branch attaches another small full convolutional network to each pyramid level to restore the offset of each anchor box to the real object nearby.

[0052] Figure 2 is a flowchart of a specific method of step S103 provided in an embodiment of the present application. In some embodiments, the knowledge distillation loss information includes local distillation loss information and global distillation loss information, see Figure 2 The method includes but is not limited to steps S201 to S203.

[0053] Step S201, calculating knowledge distillation loss information for feature maps of the same scale in the first feature map set and the second feature map set to obtain local distillation loss information.

[0054] In step S202, the first feature map set and the second feature map set are input into a preset global information extraction network to perform feature map global information extraction to obtain a first global feature map and a second global feature map.

[0055] Step S203, calculating knowledge distillation loss information on the first global feature map and the second global feature map to obtain global distillation loss information.

[0056] In a specific embodiment, step S201 specifically includes: constructing a region separation mask and a scale mask; performing attention weight calculation on the feature map in the first feature map set to obtain spatial attention weight information and channel attention weight information; generating a spatial attention mask based on the spatial attention weight information, and generating a channel attention mask based on the channel attention weight information; performing knowledge distillation loss information calculation based on the region separation mask, the scale mask, the spatial attention mask and the channel attention mask to obtain local distillation loss information. The region separation mask is used to distinguish the foreground and background of the feature maps in both the first feature map set and the second feature map set, and the scale mask is used to distinguish the foreground and background of the feature maps of different scales in both the first feature map set and the second feature map set based on the region separation mask.

[0057] The calculation formulas for the region separation mask and scale mask are: , , , Among them, r is the real box, i and j are the horizontal coordinate and vertical coordinate of the feature map respectively. is the region separation mask, is the scale mask, and are the height and width of the real box, H and W are the width and height of the feature map, is the total number of background pixels.

[0058] The calculation formulas for spatial attention weight information and channel attention weight information are: , , in, is the spatial attention weight information, is the channel attention weight information, C is the total number of channels in the feature map, is a feature map with c channels, is a feature map with horizontal and vertical coordinates i and j respectively.

[0059] The calculation formulas for the spatial attention mask and channel attention mask are: , , in, is the spatial attention mask, is the channel attention mask and T is a hyperparameter.

[0060] The calculation formula of local distillation loss information is: , in, is the local distillation loss information, and is a hyperparameter, is the feature map with c channels, i horizontal coordinates and j vertical coordinates in the first feature map set, is the feature map with the number of channels c, horizontal coordinates i and vertical coordinates j in the second feature map set, is the feature map with c channels, i horizontal coordinates and j vertical coordinates in the first feature map set, is the feature map with c channels, i horizontal coordinates and j vertical coordinates in the second feature map set.

[0061] In a specific embodiment, the first feature atlas and the second feature atlas are input into a preset global information extraction network, and global context modeling and feature fusion are performed in the global information extraction network to obtain enhanced feature representation, namely, the first global feature map and the second global feature map.

[0062] Preferably, the global information extraction network is a GC Block network.

[0063] The calculation formulas for the first global feature map and the second global feature map are: , , in, is the first global feature map, v is the second global feature map, is the feature map in the first feature map set, is the feature map in the second feature map set, , and All are convolution operations, Relu is Relu activation operation, LN is normalization operation, is the number of pixels in the feature layer, e is a natural constant, is the foreground area of ​​the feature map in the first feature map set, is the foreground area of ​​the feature map in the first feature map set.

[0064] The calculation formula of global distillation loss information is: , in, is the global distillation loss information, is a hyperparameter.

[0065] Figure 3is a flowchart of a specific method of step S104 provided in an embodiment of the present application. In some embodiments, the adversarial loss information includes feature adversarial loss information and feature loss information, see Figure 3 The method includes but is not limited to steps S301 to S303.

[0066] Step S301, input the first feature atlas and the second feature atlas into a preset discriminant network to perform feature map classification, and obtain feature map classification results of both the teacher model and the student model.

[0067] Step S302, calculating feature adversarial loss information based on the feature graph classification results of both the teacher model and the student model.

[0068] Step S303, calculating feature loss information based on the feature map classification result of the student model.

[0069] In a specific embodiment, a discriminant network is pre-built, and the discriminant network consists of two parts: an adaptive feature size adjustment network and a classification network. The first feature map set and the second feature map set are input into the discriminant network, and the input feature map is first processed into a preset size through the adaptive feature size adjustment network to obtain an adaptive feature map, and then the adaptive feature map is input into the classification network to obtain the feature map classification results of both the teacher model and the student model. The feature map classification results represent the defect type of the aircraft wall panel in the feature map.

[0070] The calculation formula of the adaptive feature map is: , , in, The adaptive feature map obtained by processing the feature map of the first feature map set, To process the feature map of the second feature map set to obtain the adaptive feature map, AdaptivePool is an adaptive pooling operation. is the convolution operation.

[0071] The calculation formulas for feature adversarial loss information and feature loss information are: , , in, is the feature adversarial loss information, is the feature loss information, is the feature map classification result of the teacher model, It is the feature map classification result of the student model.

[0072] In some embodiments, the above step S105 specifically includes: calculating classification loss information based on the predicted defect type corresponding to the target prediction result and the actual defect type of the sample image, and calculating prediction loss information based on the deviation between the predicted defect type and the actual defect type.

[0073] The calculation formulas for classification loss information and prediction loss information are: , , in, is the classification loss information, To predict loss information, and are all weight factors, p is the probability that the target prediction result is the predicted defect type, y is the probability that the target prediction result is the true defect type, and x is the deviation between the predicted defect type and the true defect type.

[0074] Figure 4 is a flowchart of a specific method of step S106 provided in an embodiment of the present application. Figure 4 The method includes but is not limited to steps S401 to S404.

[0075] Step S401, fitting knowledge distillation loss information, adversarial loss information and prediction loss information to obtain fusion loss information.

[0076] Step S402, determining whether the fusion loss information is within a loss threshold range.

[0077] If not, execute step S403; if so, execute step S404.

[0078] Step S403, adjust the weight parameters of the student model. Return to step S102.

[0079] Step S404, end the training and obtain the trained student model.

[0080] In a specific embodiment, the calculation formula of the fusion loss information is: , in, is the fusion loss information.

[0081] In a specific embodiment, a loss threshold interval is preset as a training end condition. When the fusion loss information is within the loss threshold interval, the training is terminated, and the student model obtained in the last iteration is the trained student model. When the fusion loss information is not within the loss threshold interval, the weight parameters of the student model are adjusted according to the degree of deviation of the fusion loss information from the loss threshold interval, and the first feature atlas, the second feature atlas and the target prediction result are re-obtained, so that the fusion loss information gradually approaches the loss threshold interval during the iteration process and is finally within the loss threshold interval, the training is terminated, and the trained student model is obtained.

[0082] An embodiment of the present application provides a method for detecting defects in aircraft wall panels.

[0083] Figure 5 is a flow chart of the aircraft panel defect detection method provided by the embodiment of the present application. Figure 5 In some embodiments, the method includes but is not limited to steps S501 to S502.

[0084] Step S501, obtaining an image to be detected by photographing an aircraft wall panel.

[0085] Step S502: input the image to be detected into the aircraft panel defect detection model to obtain the aircraft panel defect detection result of the image to be detected.

[0086] Among them, the aircraft wall panel defect detection model is a student model trained by the model training method described in the above embodiment.

[0087] The aircraft siding defect detection method provided in the embodiment of the present application obtains an image to be detected by photographing an aircraft siding, inputs the image to be detected into an aircraft siding defect detection model trained by a model training method, and predicts the aircraft siding defect type of the aircraft siding in the image to be detected by the aircraft siding defect detection model to obtain an aircraft siding defect detection result of the image to be detected. Since the aircraft siding defect detection model is trained based on the above-mentioned model training method, when the aircraft siding defect detection model is deployed in a scenario where the hardware computing power is insufficient and the storage space is limited, relatively accurate aircraft siding defect detection can still be achieved, and the accuracy of aircraft siding defect detection can be improved under limited hardware conditions.

[0088] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment.

[0089] Refer to the following Figure 6 The electronic device 600 according to this embodiment of the present disclosure is described. Figure 6 The electronic device 600 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0090] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0091] The storage unit stores program codes, and the program codes can be executed by the processing unit 610, so that the processing unit 610 executes the steps described in the above method part of this specification according to various exemplary embodiments of the present disclosure.

[0092] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0093] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0094] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0095] The electronic device 600 may also communicate with one or more external devices 600' (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. In addition, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0096] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program implements the above method when executed by a processor.

[0097] The aircraft siding defect detection method, model training method and related equipment provided in the embodiments of the present application perform model training based on the adversarial distillation method, input the sample image obtained by shooting the aircraft siding into the pre-trained teacher model and the student model to be trained, so as to obtain the first feature atlas from the teacher model and the second feature atlas and the target prediction result from the student model, wherein the first feature atlas and the second feature atlas both include multiple feature maps obtained by multi-scale feature extraction of the sample image, the target prediction result is the aircraft siding defect prediction result of the student model for the sample image, as well as the first feature atlas, the second feature atlas and the target prediction result, successively obtain knowledge distillation loss information, adversarial loss information and prediction loss information of the student model, and adjust the network parameters of the student model based on the knowledge distillation loss information, the adversarial loss information and the prediction loss information to train a student model that meets the requirements, so that the trained student model can finally achieve good effect and fast running speed, and when the trained student model is deployed in a scenario with limited hardware conditions, it can still achieve relatively accurate aircraft siding defect detection, and can improve the accuracy of aircraft siding defect detection under limited hardware conditions.

[0098] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above method according to the implementation of the present disclosure.

[0099] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0100] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0101] Those skilled in the art will appreciate that the above modules can be distributed in the device according to the description of the embodiment, or can be changed accordingly and only used in one or more devices different from the embodiment. The modules of the above embodiments can be combined into one module, or further divided into multiple sub-modules.

[0102] The exemplary embodiments of the present disclosure are specifically shown and described above. It should be understood that the present disclosure is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present disclosure is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

Claims

1. A model training method, characterized in that: include: Acquire a sample image obtained by photographing an aircraft siding; The sample image is input into a pre-trained teacher model and a student model to be trained to obtain a first feature atlas, a second feature atlas and a target prediction result; the first feature atlas includes a plurality of feature maps obtained by the teacher model through multi-scale feature extraction of the sample image, the second feature atlas includes a plurality of feature maps obtained by the student model through multi-scale feature extraction of the sample image, and the target prediction result is an aircraft panel defect prediction result of the student model for the sample image; Calculate the knowledge distillation loss based on the local deviation and the global deviation of the feature graphs of the first feature graph set and the second feature graph set to obtain knowledge distillation loss information; According to the feature graphs of the first feature graph set and the second feature graph set, feature graph classification is performed and adversarial loss calculation is performed according to the feature graph classification results to obtain adversarial loss information; Calculating the detection loss based on the target prediction result to obtain prediction loss information; Adjust the network parameters of the student model according to the knowledge distillation loss information, the adversarial loss information and the prediction loss information.

2. The model training method according to claim 1, characterized in that: The step of inputting the sample image into a pre-trained teacher model and a student model to be trained to obtain a first feature atlas, a second feature atlas and a target prediction result includes: Inputting the sample image into the teacher model, obtaining feature maps extracted by multiple feature map extraction networks in the teacher model, and obtaining the first feature map set; The sample image is input into the teacher model, and the feature maps extracted by the multiple feature map extraction networks in the student model are obtained, as well as the result of the multiple feature maps fused and extracted by the student model and aircraft panel defect prediction, to obtain the first feature map set and the target prediction result.

3. The model training method according to claim 1, characterized in that: The knowledge distillation loss information includes local distillation loss information and global distillation loss information. The knowledge distillation loss is calculated based on the local deviation and the global deviation of the feature graphs of the first feature graph set and the second feature graph set to obtain the knowledge distillation loss information, including: Calculate the knowledge distillation loss information on the feature maps of the same scale in the first feature map set and the second feature map set to obtain the local distillation loss information; Inputting the first feature atlas set and the second feature atlas set into a preset global information extraction network to perform feature map global information extraction to obtain a first global feature map and a second global feature map; Knowledge distillation loss information is calculated for the first global feature map and the second global feature map to obtain the global distillation loss information.

4. The model training method according to claim 3, characterized in that: The calculating of knowledge distillation loss information on feature maps of the same scale in the first feature map set and the second feature map set to obtain the local distillation loss information includes: Constructing a region separation mask and a scale mask; the region separation mask is used to distinguish the foreground and background of feature maps in both the first feature atlas set and the second feature atlas set, and the scale mask is used to distinguish the foreground and background of feature maps of different scales in both the first feature atlas set and the second feature atlas set based on the region separation mask; Performing an attention weight operation on the feature graph in the first feature graph set to obtain spatial attention weight information and channel attention weight information; Generate a spatial attention mask based on the spatial attention weight information, and generate a channel attention mask based on the channel attention weight information; The knowledge distillation loss information is calculated based on the region separation mask, the scale mask, the spatial attention mask and the channel attention mask to obtain the local distillation loss information.

5. The model training method according to claim 1, characterized in that: The adversarial loss information includes feature adversarial loss information and feature loss information. The adversarial loss information is obtained by performing feature map classification based on the feature maps of both the first feature map set and the second feature map set and performing adversarial loss calculation based on the feature map classification result, including: Inputting the first feature atlas set and the second feature atlas set into a preset discriminant network to perform feature map classification, and obtaining feature map classification results of both the teacher model and the student model; Calculating the feature adversarial loss information according to the feature graph classification results of the teacher model and the student model; The feature loss information is calculated based on the feature map classification result of the student model.

6. The model training method according to claim 1, characterized in that: The prediction loss information includes classification loss information and regression loss information. The detection loss calculation is performed based on the target prediction result to obtain the prediction loss information, including: The classification loss information is calculated based on the predicted defect type corresponding to the target prediction result and the actual defect type of the sample image, and the prediction loss information is calculated based on the deviation between the predicted defect type and the actual defect type.

7. The model training method according to claim 1, characterized in that: The adjusting the network parameters of the student model according to the knowledge distillation loss information, the adversarial loss information and the prediction loss information includes: Fitting the knowledge distillation loss information, the adversarial loss information, and the prediction loss information to obtain fusion loss information; Determining whether the fusion loss information is within a loss threshold range; If not, adjust the weight parameters of the student model; return to the step of inputting the sample image into the pre-trained teacher model and the student model to be trained to obtain the first feature atlas, the second feature atlas and the target prediction result; If it is, end the training and get the trained student model.

8. A method for detecting defects in aircraft panels, characterized in that: include: Acquire the image to be detected obtained by photographing the aircraft wall panel; The image to be detected is input into an aircraft panel defect detection model to obtain an aircraft panel defect detection result of the image to be detected; the aircraft panel defect detection model is a student model trained by the model training method described in any one of claims 1 to 7.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images