A method and system for automobile parts defect detection based on AMEF

Abstract

The present invention discloses an AMEF-based automotive parts defect detection method and system. The method involves building an automotive parts defect detection model, training the model using a training set and calculating the loss to obtain a trained model. Real-world automotive parts images and attention images are then acquired and input into the trained model. The AMEF module of the automotive parts defect detection model receives the input image, extracts and fuses features, and outputs fused features. A backbone feature extraction module receives and processes the fused features, outputting deep features. A classification module classifies and identifies the deep features and outputs automotive parts defect detection results. This method is suitable for automotive parts defect detection in complex backgrounds and with varying scales of the same type of defects, achieving excellent recognition results.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile component defect detection, and in particular to an automobile component defect detection method and system based on AMEF. Background Art

[0002] Automotive parts defect detection aims to identify material defects and manufacturing process defects in parts, achieving efficient, accurate, and automatic defect detection for auto parts, which is crucial for improving overall vehicle quality and ensuring safety performance. In actual production, defects in automotive parts defect detection typically appear in a small area of ​​the product being tested. Defects in parts are often discovered manually based on subjective experience, which is inefficient and can be affected by personal factors such as fatigue and vision of relevant technicians, leading to false detections and missed detections. In addition, defect patterns are often unpredictable, making it difficult to directly describe the defects. Existing defect classifications still cannot cover all types, resulting in the inability of existing methods to identify newly emerging defect types.

[0003] Existing techniques typically classify automotive component defects by building and training generator and discriminator networks. However, this approach is limited by the scarcity of defect samples and cannot cover all defect types. Furthermore, due to the large differences in scales of defects of the same type and the complex background, the identification of the product under test is poor, which urgently needs improvement. Summary of the Invention

[0004] In response to the above problems, the present invention proposes an automotive parts defect detection method and system based on AMEF, which is used to solve the problem in related technologies that, due to the scarcity of defect samples, the classification of defect categories cannot cover all defect types. In addition, there are also problems such as complex background and large differences in the scales of defects of the same type, which lead to poor recognition effect of the product to be tested.

[0005] In one aspect, the present invention provides an automobile parts defect detection method based on AMEF, comprising the following steps:

[0006] S1. Build an automotive parts defect detection model, which includes an AMEF module, a backbone feature extraction module, an intra-class consistency constraint module, and a classification module. The AMEF module is connected to the backbone feature extraction module, and the intra-class consistency constraint module and the classification module are respectively connected to the backbone feature extraction module.

[0007] S2. Obtain several original images containing defects of automobile parts and attention images corresponding to the original images to form a training set;

[0008] S3. Using the training set to train the automobile parts defect detection model, designing a loss function for the automobile parts defect detection model and calculating the loss, selecting parameters when the loss value is no longer updated to update the model, and obtaining a trained automobile parts defect detection model;

[0009] S4. Obtain images of automobile parts in real scenes and corresponding attention images and input them into the trained automobile parts defect detection model for processing. The AMEF module receives the input image and performs feature extraction and fusion, and outputs fused features. The backbone feature extraction module receives the fused features and processes them, outputs deep features. The classification module receives the deep features and classifies known defect categories and identifies unknown defect categories, and outputs automobile parts defect detection results.

[0010] Preferably, the AMEF module in S1 includes a spatial domain feature extraction module, a wavelet convolution transform module, a frequency expert module and a fusion module, wherein the wavelet convolution transform module, the frequency expert module and the fusion module are connected in sequence, and the spatial domain feature extraction module is connected to the fusion module.

[0011] Preferably, the specific process of S3 is as follows:

[0012] S31, randomly selecting a preset number of original images from the training set and attention images corresponding to the selected original images as input to the automobile parts defect detection model;

[0013] S32, AMEF module receives the input original image and attention image and performs feature extraction and fusion processing, and outputs fusion features;

[0014] S33, the backbone feature extraction module is used to receive and process the fusion features and output deep features;

[0015] S34, the intra-class consistency constraint module receives the deep features and performs high-dimensional space mapping to obtain high-dimensional mapping features, optimizes the distribution of the high-dimensional mapping features through consistency constraints, and obtains high-dimensional mapping features of different categories;

[0016] S35, the classification module receives the deep features and classifies known defect categories and identifies unknown defect categories to obtain defect detection results;

[0017] S36. Design a loss function for an automotive parts defect detection model based on different categories of high-dimensional mapping features and defect detection results, and calculate the loss;

[0018] S37. Select a preset number of original images and attention images corresponding to the selected original images from the training set, repeat steps S31 to S36, select the parameter update model when the loss value is no longer updated, and obtain the trained automobile parts defect detection model.

[0019] Preferably, the specific process of S32 is as follows:

[0020] S321, the spatial domain feature extraction module receives and processes the input original image to obtain spatial domain features;

[0021] S322, the wavelet convolution transform module receives and processes the input original image to obtain a frequency domain image;

[0022] S323, the frequency expert module receives and processes the attention image and the frequency domain image to obtain frequency domain fusion features;

[0023] S324, the fusion module receives the spatial domain features and the frequency domain fusion features and concatenates them based on the channel dimension to obtain fusion features.

[0024] Preferably, S322 specifically includes the following:

[0025] S3221, performing scale transformation and feature extraction on the input original image by using Haar wavelet to obtain two-dimensional low-pass filter coefficients and high-pass filter coefficients, and obtaining a two-dimensional wavelet filter by combining the low-pass filter coefficients and the high-pass filter coefficients;

[0026] S3222, applying a two-dimensional wavelet filter to the input original image, and extracting a low-frequency component corresponding to the input original image through a convolution operation;

[0027] S3223, performing recursive wavelet transform on the low-frequency components corresponding to the original image using a two-dimensional wavelet filter, and extracting low-frequency components and high-frequency components at different levels from the low-frequency components layer by layer;

[0028] S3224, unifying the low-frequency components and high-frequency components of different layers extracted layer by layer into the same size through an upsampling method, thereby obtaining low-frequency components and high-frequency components of different layers with the same size;

[0029] S3225. Adjust the feature distribution through 1×1 convolution operation, and splice the low-frequency components and high-frequency components of different levels with the same size based on the channel dimension to obtain a frequency domain image.

[0030] Preferably, the frequency domain image in S3225 is specifically represented as:

[0031] ;

[0032] in, is the weight of the 1×1 convolution kernel, and the weight size is , is the bias term, and the size of the bias term is , is the number of input channels, is the input image at position The frequency domain image of is the pixel value after upsampling.

[0033] Preferably, the frequency expert module includes a gating network and a plurality of frequency expert networks respectively connected to the gating network. S323 specifically includes the following:

[0034] S3231, the gated network receives and processes the attention image and generates sparse weights;

[0035] S3232, adding random noise to the sparse weight to obtain a sparse weight with random noise added;

[0036] S3233, multiple frequency expert networks perform frequency decomposition and feature extraction on the input frequency domain image respectively, to obtain feature maps processed by each expert network;

[0037] S3234. Perform weighted combination on the feature maps processed by each expert network according to the sparse weight of the random noise to obtain frequency domain fusion features.

[0038] Preferably, the frequency domain fusion feature in S3234 is specifically expressed as:

[0039] ;

[0040] in, ;

[0041] ;

[0042] Where, is the frequency domain fusion feature, is the sparse gating weight after adding random noise, represents sparse weights, is the gating weight, M is the total number of frequency expert networks, is selected from M frequency expert networks A frequency expert network, is selected The frequency expert network The output of the frequency expert network, is the attention image.

[0043] Preferably, the loss function of the automobile parts defect detection model in S36 is specifically:

[0044] ;

[0045] in, ;

[0046] ;

[0047] Where, represents the total loss of the automotive parts defect detection model, represents the classification loss of known class defects, represents the intra-class consistency loss, is the load loss, represents the intra-class constraint loss, represents the inter-class constraint loss, It is The number of times the frequency expert network is selected, is the total number of inputs, M is the total number of frequency expert networks, 、 、 and are weight coefficients respectively.

[0048] Another aspect of the present invention provides an automobile parts defect detection system, which uses the above-mentioned automobile parts defect detection method based on AMEF to detect automobile parts defects. The automobile parts defect detection system includes an image acquisition module, an image processing module and a computer system. The automobile parts defect detection model is set in the computer system. The image acquisition module is connected to the image processing module and the computer system respectively. The image processing module is connected to the computer system, wherein:

[0049] The image acquisition module is used to obtain the image of the automobile parts to be inspected and transmit the image of the automobile parts to be inspected to the image processing module and the computer system;

[0050] The image processing module is used to process the input image of the automobile parts to be inspected to obtain the corresponding attention image;

[0051] The computer system is used to receive the image of the automobile part to be inspected and its corresponding attention image and input them into the automobile part defect detection model for processing, and output the detection result of the image of the automobile part to be inspected.

[0052] The AMEF-based automotive parts defect detection method and system described above builds an automotive parts defect detection model. Several original images containing automotive part defects and attention images corresponding to these original images are obtained to form a training set. The automotive parts defect detection model is trained using the training set and loss is calculated to obtain a trained automotive parts defect detection model. Real-world automotive part images and corresponding attention images are then input into the trained automotive parts defect detection model for processing. The AMEF module receives the input automotive part images and corresponding attention images and performs feature extraction and fusion processing, outputting fused features. The backbone feature extraction module receives and processes the fused features, outputting deep features. The classification module uses the deep features to classify known defect categories and identify unknown defect categories, outputting automotive parts defect detection results. This method is suitable for automotive parts defect detection in complex backgrounds and where defects of the same type vary widely in scale, achieving excellent recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of an automobile parts defect detection method based on AMEF in one embodiment of the present invention;

[0054] Figure 2 4 is a schematic structural diagram of an automobile parts defect detection model in one embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below with reference to the accompanying drawings.

[0056] See also Figure 1 , an automobile parts defect detection method based on AMEF, the method comprising the following steps:

[0057] S1. Build an automotive parts defect detection model, which includes an AMEF module, a backbone feature extraction module, an intra-class consistency constraint module, and a classification module. The AMEF module is connected to the backbone feature extraction module, and the intra-class consistency constraint module and the classification module are respectively connected to the backbone feature extraction module.

[0058] See also Figure 2The automotive parts defect detection model includes an AMEF module (Attention-guided Mutli-frequency Expert Fusion Net), a backbone feature extraction module, an intra-class consistency constraint module, and a classification module. The AMEF module is connected to the backbone feature extraction module, while the intra-class consistency constraint module and the classification module are connected to the backbone feature extraction module. The AMEF module receives the input original image and attention image, extracts and fuses features, and outputs fused features. The backbone feature extraction module receives and processes the fused features, outputting deep features. The intra-class consistency constraint module receives and processes the deep features and performs high-dimensional mapping to obtain high-dimensional mapping features. The classification module receives and processes the deep features and outputs the automotive parts defect detection results.

[0059] Furthermore, the AMEF module includes a spatial domain feature extraction module, a wavelet convolution transform module, a frequency expert module and a fusion module, wherein the wavelet convolution transform module, the frequency expert module and the fusion module are connected in sequence, and the spatial domain feature extraction module is connected to the fusion module.

[0060] S2. Obtain several original images containing defects in automobile parts and attention images corresponding to the original images to form a training set.

[0061] Specifically, several original images containing automobile parts are obtained, and the original images are divided into normal samples and defect samples of different categories. The attention images corresponding to each normal sample or defect sample of different categories are obtained through the existing anomaly detection method. The normal samples, defect samples of different categories and their corresponding attention maps are combined into a dataset.

[0062] S3. Use the training set to train the automobile parts defect detection model, design the loss function of the automobile parts defect detection model and calculate the loss, select the parameters whose loss value is not updated during the update to update the model, and obtain the trained automobile parts defect detection model.

[0063] Furthermore, the training set is used in S3 to train the automotive parts defect detection model. The specific process is as follows:

[0064] S31. Randomly select a preset number of original images from the training set and the attention images corresponding to the selected original images as input into the automobile parts defect detection model.

[0065] S32, AMEF module receives the input original image and attention image and performs feature extraction and fusion processing, and outputs fusion features.

[0066] Furthermore, the specific process of S32 is as follows:

[0067] S321, the spatial domain feature extraction module receives the input original image And process to obtain spatial domain features .

[0068] S322, the wavelet convolution transform module receives the input original image And process to get the frequency domain image .

[0069] Furthermore, S322 specifically includes the following:

[0070] S3221, the original image input by Haar wavelet Perform scale transformation and feature extraction to obtain two-dimensional low-pass filter coefficients and high-pass filter coefficients , by combining the low-pass filter coefficients and high-pass filter coefficients Get a two-dimensional wavelet filter.

[0071] The two-dimensional wavelet filter can be specifically expressed as:

[0072] (1) -1

[0073] (1)-2

[0074] (1)-3

[0075] (1)-4

[0076] in, is a low-pass-low-pass filter, is a low-pass-high-pass filter, is a high-pass-low-pass filter, is a high-pass-high-pass filter, and represents the pixel position of the filter convolution kernel, is the corresponding row coordinate, is the corresponding column coordinate.

[0077] S3222, apply the two-dimensional wavelet filter to the input original image , extract the original input image through convolution operation Corresponding low-frequency and high-frequency components.

[0078] Specifically, in the first wavelet transform, a two-dimensional wavelet filter is applied to the input image , use convolution operation to extract low-frequency components and high-frequency components:

[0079] (2) -1

[0080] (2) -2

[0081] (2)-3

[0082] (2)-4

[0083] in, represents the input image, is the low-frequency component of the input image, 、 、 is the high-frequency component of the input image, is the pixel position of the input image, is the pixel position of the filter convolution kernel.

[0084] S3223, the original image The corresponding low-frequency components are recursively transformed using a two-dimensional wavelet filter to extract low-frequency components and high-frequency components at different levels layer by layer.

[0085] Specifically, in order to capture multi-scale features, the low-frequency components of the input image are repeatedly subjected to recursive wavelet transform, and low-frequency components and high-frequency components at different levels are extracted layer by layer. The formula is as follows:

[0086] (3) -1

[0087] (3) -2

[0088] (3) -3

[0089] (3)-4

[0090] in, Represents the first Layer low-frequency components, Represents the first Layer high frequency components, Represents the first Layer high frequency components, Represents the first Layer high frequency components, Indicates the The low-pass filter of the layer, Indicates the The low-pass-high-pass filter of the layer, Indicates the The high-pass-low-pass filter of the layer, Indicates the High-pass-high-pass filter of the layer, Indicates the The low-frequency component of the layer.

[0091] S3224. Unify the low-frequency components and high-frequency components of different levels extracted layer by layer into the same size through an upsampling method to obtain low-frequency components and high-frequency components of different levels with the same size.

[0092] Specifically, for frequency domain features of different sizes, they are unified into the same size through upsampling method, and the formula is as follows:

[0093] (4)

[0094] (4) -1

[0095] in, is the pixel value before upsampling, that is, the low-frequency component and high-frequency component at different levels, is the pixel value after upsampling, that is, the low-frequency component and high-frequency component after unified size, and is the interpolation weight function, represents the height of the feature before upsampling, Indicates the width of the feature before upsampling.

[0096] S3225, adjust the feature distribution through 1×1 convolution operation, and splice the low-frequency components and high-frequency components of different levels with the same size based on the channel dimension to obtain the frequency domain image .

[0097] Furthermore, the frequency domain image in S3225 Specifically expressed as:

[0098] (5)

[0099] in, is the weight of the 1×1 convolution kernel, and the weight size is , is the bias term, and the size of the bias term is , is the number of input channels, is the input image at position The frequency domain image of This is the pixel value after upsampling.

[0100] S323, the frequency expert module is used to receive the attention image and frequency domain images And process to obtain frequency domain fusion features .

[0101] Furthermore, the frequency expert module includes a gate control network and multiple frequency expert networks respectively connected to the gate control network. S323 specifically includes the following:

[0102] S3231, the gated network receives the attention image And process to generate sparse weights.

[0103] (6)

[0104] in, Represents sparse weights, used to transform frequency domain images Assigned to different frequency expert networks, , is the attention image, , is the gate weight, is selected from M frequency expert networks A frequency expert network, is the batch size, is the dimension of the flattened image.

[0105] S3232. Add random noise to the sparse weight to obtain a sparse weight with random noise added.

[0106] Specifically, to enhance the exploration capability of the gating network, random noise is added to the sparse weights:

[0107] (7)

[0108] in, represents the sparse weight after adding random noise, Represents random noise with mean 0 and variance The normal distribution of Its size is controlled by specific parameters to avoid excessive noise affecting the stability of the model.

[0109] S3233, multiple frequency expert networks for input frequency domain images Frequency decomposition and feature extraction are performed separately to obtain the feature map processed by each expert network.

[0110] For example, for the K selected frequency expert networks, Frequency Expert Network , which outputs:

[0111] (8)

[0112] in, Indicates the Frequency Expert Network For the input frequency domain image The feature map obtained after processing, , is the batch size, is the number of channels output by the wavelet convolution transform module, and is the size of the frequency domain image.

[0113] S3234, the feature map processed by each frequency expert network is weighted and combined according to the sparse weight of random noise to obtain the frequency domain fusion feature .

[0114] Furthermore, the frequency domain fusion feature in S3234 Specifically expressed as:

[0115] (9)

[0116] Where, is the frequency domain fusion feature, is the sparse gating weight after adding random noise, K is the Selected from M frequency expert networks The number of frequency expert networks with the most important weights, is selected The frequency expert network The output of a frequency expert network.

[0117] S324, fusion module receives spatial domain features and frequency domain fusion features And splice the two based on the channel dimension to obtain the fusion feature .

[0118] Furthermore, the fusion features It can be expressed as:

[0119] (10)

[0120] To address the problem of defect detection of automotive parts in complex backgrounds, this paper proposes an AMEF module. This module first uses wavelet transform to obtain feature information of different scales and frequencies to reduce the influence of background noise. Then, it guides the frequency expert module to focus on the foreground target area through attention, which can further improve the adaptability to complex backgrounds.

[0121] S33, the backbone feature extraction module is used to receive and process the fused features and output deep features.

[0122] Furthermore, the backbone feature extraction module can be specifically a ResNet-18 network, which integrates the features Input to the ResNet-18 network for feature extraction to extract high-order semantic features and finally output deep features .

[0123] S34. The intra-class consistency constraint module receives deep features and performs high-dimensional space mapping to obtain high-dimensional mapping features. The high-dimensional mapping features are optimized and distributed through consistency constraints to obtain high-dimensional mapping features of different categories.

[0124] Specifically, the intra-class consistency constraint module transforms deep features into Mapping to high-dimensional space, the mapping function is:

[0125] , (11)

[0126] in, Indicates the mapping result of the input deep features in the high-dimensional space, that is, the high-dimensional mapping feature, From the normal distribution A random matrix sampled from , is from a uniform distribution The offset vector sampled in , Is the dimension of the high-dimensional space. The high-dimensional features after random Fourier mapping can help capture the correlation of features.

[0127] Using the above method, a preset number of original images are selected from the training set each time. For each of the selected original images, its corresponding deep features will be obtained. The deep features are mapped into a high-dimensional space using the intra-class consistency constraint module to obtain high-dimensional mapping features.

[0128] This high-dimensional mapping method efficiently maps deep input features into a high-dimensional space, thereby measuring the nonlinear correlations between the original features using an approximate kernel approach. This method not only enriches the expressive power of features but also better captures fine-grained information, providing strong support for subsequent feature optimization.

[0129] Next, the extracted high-dimensional mapping features are further optimized for distribution using consistency constraints. First, to reduce the dispersion of defect samples of the same category in the high-dimensional feature space, the class consistency constraint module introduces an intra-class consistency constraint loss. This loss minimizes the differences in feature distribution between samples of the same category, making intra-class features more compact. It is defined as:

[0130] (12)

[0131] in, (12) -1

[0132] Where, represents the intra-class constraint loss, It is a sample and High-dimensional mapping features and The covariance matrix between Represents the mean of the high-dimensional mapping features corresponding to all samples in each batch, represents the trace operation of the matrix, represents the square of the Frobenius norm of the matrix, Representation and Sample A collection of samples of the same category, Indicates the batch size.

[0133] By minimizing the deviation of the covariance matrix, this loss constrains the feature vectors of samples in the same category to be more similar, thereby reducing the difference in feature distribution within the category.

[0134] Secondly, to enhance the ability to distinguish between defects of different categories, the class consistency constraint module introduces an inter-class separation constraint loss. The inter-class separation loss maximizes the difference in feature distribution between defect samples of different categories, ensuring that features between categories are more dispersed. It is defined as follows:

[0135] (13)

[0136] in, (13) -1

[0137] Where, represents the inter-class constraint loss, It is a sample and samples The high-dimensional feature vector of and The covariance matrix between Representation and Sample A collection of samples from different categories.

[0138] The inter-class separation constraint maximizes the Frobenius norm of the covariance matrix between features of samples of different categories while reducing its trace value, thereby enhancing the feature discrimination between categories.

[0139] Through the intra-class constraint loss measurement and inter-class constraint loss measurement, the compactness of features of the same class and the separability of features of different classes can be improved, thereby improving the classification accuracy.

[0140] S35. The classification module receives the deep features and classifies known defect categories and identifies unknown defect categories to obtain defect detection results.

[0141] S36. Design the loss function of the automotive parts defect detection model based on different categories of high-dimensional mapping features and defect detection results and calculate the loss.

[0142] Furthermore, the loss function of the automobile parts defect detection model in S36 is specifically:

[0143] (14)

[0144] in, (14) -1

[0145] (14) -2

[0146] Where, represents the total loss of the automotive parts defect detection model, represents the classification loss of known class defects, represents the intra-class consistency loss, is the load loss, represents the intra-class constraint loss, represents the inter-class constraint loss, It is The number of times the frequency expert network is selected, is the total number of inputs, M is the total number of frequency expert networks, 、 、 and are weight coefficients respectively.

[0147] Specifically, the total loss design process of the automotive parts defect detection model is as follows:

[0148] 1) In order to make the selection of all frequency expert networks more balanced, the load loss is defined as follows:

[0149] ;

[0150] in, is the load loss, It is The number of times the frequency expert network is selected, is the total number of inputs, N=B*number of batches, is the total number of frequency expert networks, is the loss weight.

[0151] 2) Based on the intra-class constraint loss and inter-class constraint loss, the intra-class consistency loss function is designed as follows:

[0152] ;

[0153] in, represents the intra-class consistency loss, represents the intra-class loss, represents the inter-class loss, and They are weight coefficients, which are used to balance the intra-class loss and inter-class loss.

[0154] 3) According to load loss , intra-class consistency loss and the classification loss for known class defects Design the total loss of the automotive parts defect detection model:

[0155] ;

[0156] S37. Select a preset number of original images and attention images corresponding to the selected original images from the training set, repeat steps S31 to S36, select the parameter update model when the loss value is no longer updated, and obtain the trained automobile parts defect detection model.

[0157] Specifically, after each training, the total loss of the automobile parts defect detection model is calculated using the above loss function until the loss value is no longer updated, and the parameters of the automobile parts defect detection model are optimized by backpropagation to obtain the trained automobile parts defect detection model.

[0158] S4. Obtain images of automobile parts in real scenes and corresponding attention images and input them into the trained automobile parts defect detection model for processing. The AMEF module receives the input image and performs feature extraction and fusion, and outputs fused features. The backbone feature extraction module receives the fused features and processes them, outputs deep features. The classification module receives the deep features and classifies known defect categories and identifies unknown defect categories, and outputs automobile parts defect detection results.

[0159] Specifically, in the application stage, the intra-class consistency constraint module no longer participates in defect recognition. Multiple known defect categories are preset. The AMEF module receives the input automobile parts image and the corresponding attention image and performs feature extraction and fusion processing, outputting the fused features. The backbone feature extraction module (specifically the ResNet-18 network) performs deep representation extraction on the fused features and outputs the deep features. The deep features are input into the classification module for processing and the automobile parts defect detection results are output. The defect detection results can be divided into unknown class defects and known class defects. When a known class defect is identified, the specific known defect category will also be output.

[0160] Assume that the defect category probability vector output by the classification module is:

[0161] ;

[0162] in, is the weight of the classification module, Indicates that it is a preset The predicted probability vector of known defect categories, It represents the deep features extracted by the backbone network from the fusion features of the real-life automobile parts image and the corresponding attention image. Represents the Softmax function.

[0163] In order to identify unknown defect categories, a rejection mechanism is designed, assuming that the highest value of the defect category probability vector is:

[0164] ;

[0165] Define a threshold for distinguishing known defects from unknown defects ,like , then the defect category corresponding to the input image is determined to be an unknown defect, and the label is , where unknown defects include uncommon defects or defects that are difficult to collect; if , the defect category corresponding to the input image is determined to be a known defect. When it is a known defect, the classification module determines the known defect category label based on the maximum index:

[0166] ;

[0167] in, represents the known defect category label predicted by the classification module, Represents solving the predicted probability vector The index operation of the maximum value in .

[0168] In one embodiment, an automobile part defect detection system employs the AMEF-based automobile part defect detection method to detect defects in automobile parts. The automobile part defect detection system includes an image acquisition module, an image processing module, and a computer system. An automobile part defect detection model is provided in the computer system. The image acquisition module is connected to the image processing module and the computer system, respectively. The image processing module is connected to the computer system, wherein:

[0169] The image acquisition module is used to obtain the image of the automobile parts to be inspected and transmit the image of the automobile parts to be inspected to the image processing module and the computer system;

[0170] The image processing module is used to process the input image of the automobile parts to be inspected to obtain the corresponding attention image;

[0171] The computer system is used to receive the image of the automobile part to be inspected and its corresponding attention image and input them into the automobile part defect detection model for processing, and output the detection result of the image of the automobile part to be inspected.

[0172] For the specific definition of a method system for defect detection of automobile parts, please refer to the definition of an automobile parts defect detection method based on AMEF above, which will not be repeated here.

[0173] Furthermore, the effectiveness of the AMEF-based automobile parts defect detection method in the present invention was verified through comparative tests.

[0174] In the comparative experiment, first, we constructed a dataset of automobile parts defect categories and conducted experiments based on the existing open set recognition method. The experimental results are shown in Table 1, which demonstrates the performance of different methods in terms of accuracy (ACC), F1 score and AUROC indicators.

[0175] Table 1 Performance data corresponding to different methods

[0176]

[0177] Experimental comparisons show that the AMEF-based automotive parts defect detection method proposed in this invention (corresponding to the "method in this application" in Table 1) significantly improves the performance of automotive parts appearance defect recognition, achieving an accuracy of 98.35%, an F1 score of 81.36%, and an AUROC of 94.93%. These results demonstrate that the proposed method, when handling automotive parts appearance defect recognition tasks, can maintain high accuracy while balancing prediction performance across categories, significantly improving the F1 score and AUROC, demonstrating superior overall performance.

[0178] The AMEF-based automotive parts defect detection method and system builds an automotive parts defect detection model, which includes an AMEF module, a backbone feature extraction module, an intra-class consistency constraint module, and a classification module. Several original images containing automotive part defects and their corresponding attention images are obtained to form a training set. The automotive parts defect detection model is trained using the training set and loss is calculated to obtain a trained automotive parts defect detection model. Real-world automotive part images and their corresponding attention images are input into the trained automotive parts defect detection model for processing. The AMEF module receives the input automotive part images and their corresponding attention images and performs feature extraction and fusion processing, outputting fused features. The backbone feature extraction module receives and processes the fused features, outputting deep features. The classification module classifies the high-dimensional defect features into known defect categories and identifies unknown defect categories, outputting automotive parts defect detection results. This method is suitable for automotive parts defect detection in complex backgrounds and with large variations in the scale of the same type of defects, achieving excellent recognition results.

[0179] The above describes in detail the AMEF-based automotive parts defect detection method and system provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above examples is intended only to facilitate understanding of the core concepts of the present invention. It should be noted that those skilled in the art will be able to make various improvements and modifications to the present invention without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the claims of the present invention.

Claims

1. An automobile parts defect detection method based on AMEF, characterized in that: The method comprises: S1. Build an automotive parts defect detection model, which includes an AMEF module, a backbone feature extraction module, an intra-class consistency constraint module, and a classification module. The AMEF module is connected to the backbone feature extraction module, and the intra-class consistency constraint module and the classification module are respectively connected to the backbone feature extraction module. S2. Obtain several original images containing defects of automobile parts and attention images corresponding to the original images to form a training set; S3. Using the training set to train the automobile parts defect detection model, designing a loss function for the automobile parts defect detection model and calculating the loss, selecting parameters when the loss value is no longer updated to update the model, and obtaining a trained automobile parts defect detection model; S4. Obtain an image of an automobile part in a real scene and the corresponding attention image and input them into the trained automobile part defect detection model for processing. The AMEF module receives the input image and performs feature extraction and fusion, outputting fused features. The backbone feature extraction module receives and processes the fused features and outputs deep features. The classification module receives the deep features and classifies known defect categories and identifies unknown defect categories, and outputs automobile part defect detection results. The AMEF module in S1 includes a spatial domain feature extraction module, a wavelet convolution transform module, a frequency expert module and a fusion module, wherein the wavelet convolution transform module, the frequency expert module and the fusion module are connected in sequence, and the spatial domain feature extraction module is connected to the fusion module.

2. The automobile parts defect detection method based on AMEF according to claim 1, characterized in that: The specific process of S3 is as follows: S31, randomly selecting a preset number of original images from the training set and attention images corresponding to the selected original images as input to the automobile parts defect detection model; S32, AMEF module receives the input original image and attention image and performs feature extraction and fusion processing, and outputs fusion features; S33, the backbone feature extraction module is used to receive and process the fusion features and output deep features; S34, the intra-class consistency constraint module receives the deep features and performs high-dimensional space mapping to obtain high-dimensional mapping features, optimizes the distribution of the high-dimensional mapping features through consistency constraints, and obtains high-dimensional mapping features of different categories; S35, the classification module receives the deep features and classifies known defect categories and identifies unknown defect categories to obtain defect detection results; S36. Design a loss function for an automotive parts defect detection model based on different categories of high-dimensional mapping features and defect detection results, and calculate the loss; S37. Select a preset number of original images and attention images corresponding to the selected original images from the training set, repeat steps S31 to S36, select the parameter update model when the loss value is no longer updated, and obtain the trained automobile parts defect detection model.

3. The automobile parts defect detection method based on AMEF according to claim 2, characterized in that: The specific process of S32 is as follows: S321, the spatial domain feature extraction module receives and processes the input original image to obtain spatial domain features; S322, the wavelet convolution transform module receives and processes the input original image to obtain a frequency domain image; S323, the frequency expert module receives and processes the attention image and the frequency domain image to obtain frequency domain fusion features; S324, the fusion module receives the spatial domain features and the frequency domain fusion features and concatenates them based on the channel dimension to obtain fusion features.

4. The automobile parts defect detection method based on AMEF according to claim 3, characterized in that: S322 specifically includes the following: S3221, performing scale transformation and feature extraction on the input original image by using Haar wavelet to obtain two-dimensional low-pass filter coefficients and high-pass filter coefficients, and obtaining a two-dimensional wavelet filter by combining the low-pass filter coefficients and the high-pass filter coefficients; S3222, applying a two-dimensional wavelet filter to the input original image, and extracting a low-frequency component corresponding to the input original image through a convolution operation; S3223, performing recursive wavelet transform on the low-frequency components corresponding to the original image using a two-dimensional wavelet filter, and extracting low-frequency components and high-frequency components at different levels from the low-frequency components layer by layer; S3224, unifying the low-frequency components and high-frequency components of different layers extracted layer by layer into the same size through an upsampling method, thereby obtaining low-frequency components and high-frequency components of different layers with the same size; S3225. Adjust the feature distribution through 1×1 convolution operation, and splice the low-frequency components and high-frequency components of different levels with the same size based on the channel dimension to obtain a frequency domain image.

5. The automobile parts defect detection method based on AMEF according to claim 4, characterized in that: The frequency domain image in S3225 is specifically expressed as: ； in, is the weight of the 1×1 convolution kernel, and the weight size is , is the bias term, and the size of the bias term is , is the number of input channels, is the input image at position The frequency domain image of is the pixel value after upsampling.

6. The automobile parts defect detection method based on AMEF according to claim 5, characterized in that: The frequency expert module includes a gate control network and multiple frequency expert networks connected to the gate control network. S323 specifically includes the following: S3231, the gated network receives and processes the attention image and generates sparse weights; S3232, adding random noise to the sparse weight to obtain a sparse weight with random noise added; S3233, multiple frequency expert networks perform frequency decomposition and feature extraction on the input frequency domain image respectively, to obtain feature maps processed by each expert network; S3234. Perform weighted combination on the feature maps processed by each expert network according to the sparse weight of the random noise to obtain frequency domain fusion features.

7. The automobile parts defect detection method based on AMEF according to claim 6, characterized in that: The frequency domain fusion features in S3234 are specifically expressed as: ； in, ; ； Where, is the frequency domain fusion feature, is the sparse gating weight after adding random noise, represents sparse weights, is the gating weight, M is the total number of frequency expert networks, is selected from M frequency expert networks A frequency expert network, is selected The frequency expert network The output of the frequency expert network, is the attention image.

8. The automobile parts defect detection method based on AMEF according to claim 2, characterized in that: The loss function of the automotive parts defect detection model in S36 is specifically: ； in, ; ； Where, represents the total loss of the automotive parts defect detection model, represents the classification loss of known class defects, represents the intra-class consistency loss, is the load loss, represents the intra-class constraint loss, represents the inter-class constraint loss, It is The number of times the frequency expert network is selected, is the total number of inputs, M is the total number of frequency expert networks, 、 、 and are weight coefficients respectively.

9. An automobile parts defect detection system, which detects automobile parts defects using the automobile parts defect detection method based on AMEF according to any one of claims 1 to 8, characterized in that: The automobile parts defect detection system includes an image acquisition module, an image processing module and a computer system. The automobile parts defect detection model is set in the computer system. The image acquisition module is connected to the image processing module and the computer system respectively. The image processing module is connected to the computer system, wherein: The image acquisition module is used to obtain the image of the automobile parts to be inspected and transmit the image of the automobile parts to be inspected to the image processing module and the computer system; The image processing module is used to process the input image of the automobile parts to be inspected to obtain the corresponding attention image; The computer system is used to receive the image of the automobile part to be inspected and its corresponding attention image and input them into the automobile part defect detection model for processing, and output the detection result of the image of the automobile part to be inspected.

Images