Industrial part defect classification method and device for realizing inter-domain category self-adaption, processor and computer readable storage medium thereof

Abstract

The invention relates to an industrial part defect classification method capable of realizing inter-domain category self-adaption, which comprises the following steps of: acquiring a source domain data set with label information and a target domain data set without label information, and preprocessing the source domain data set and the target domain data set; inputting a to-be-detected target domain sample into the trained neural network detection model for defect detection; and removing a domain adaptation structure which is not needed in the domain adaptation detection network, and carrying out defect detection on the target domain scene. According to the industrial part defect classification method and device for realizing inter-domain category self-adaption, the processor and the computer readable storage medium, the domain invariant feature information aiming at the category is decoupled from the middle layer of the feature extraction network and is fused into the original feature so as to enhance the classification capability; an ELA attention mechanism is added to solve the key problems of small defect size and difficult feature extraction of part detection, and finally, class labels are taken as conditions during classification, inter-domain alignment is carried out for classes, and the cross-domain classification capability is further improved.

Description

Technical Field

[0001] The present invention relates to the field of defect target detection, and in particular to the field of defect detection on the surface of industrial parts, and specifically refers to an industrial part defect classification method, device, processor and computer-readable storage medium thereof for realizing inter-domain category adaptation. Background Art

[0002] The application of unsupervised domain adaptation (UDA) in defect target detection has received increasing attention in recent years. With the advancement of Industry 4.0, automated and intelligent inspection systems play a vital role in the manufacturing industry. However, due to the high cost of data annotation and inter-domain differences, directly applying models from one domain (such as laboratory environment) to another domain (such as actual production environment) usually leads to a significant drop in performance. This challenge has prompted researchers to explore how to achieve efficient defect detection on unlabeled target domains.

[0003] In the task of defect target detection, domain differences are mainly manifested in differences in imaging conditions, object surface characteristics, defect types, etc. For example, high-quality images collected in a laboratory environment and images collected in an actual production environment may have significant differences in lighting, background, noise, etc. These differences will lead to poor generalization performance of the trained model in the target domain.

[0004] Unsupervised domain adaptation aims to train the model by utilizing the features of the source domain (labeled data) and the target domain (unlabeled data) so that it can also perform well in the target domain. The specific methods mainly include the following categories:

[0005] Adversarial training method: By introducing an adversarial training mechanism, the feature distribution of the source domain and the target domain is aligned as much as possible. Typical methods include Domain-Adversarial Neural Networks (DANN) and Adversarial Discriminative Domain Adaptation (ADDA).

[0006] Feature mapping method: By learning a common feature space, the data of the source domain and the target domain are mapped into the same feature space, thereby reducing the difference between domains. This type of method includes CORAL (Correlation Alignment) and DeepCORAL.

[0007] Self-supervised learning: Use unlabeled data in the target domain for self-supervised learning to improve the model's feature extraction capabilities in the target domain.

[0008] Since most defects in industrial parts defect detection are too small, feature extraction is difficult. After extracting features through multiple convolutional layers, defect information is often lost, which in turn affects the accuracy of classification. In addition, since the overall environment of the parts is single, when a defect occurs, its features will be significantly different from the surrounding features. For this reason, we add the ELA attention mechanism to the feature extraction network to pay more attention to defect information. ELA uses strip pooling in the spatial dimension to extract horizontal and vertical features, maintains an elongated kernel shape to obtain long-distance dependencies, and avoids interference from irrelevant areas on label prediction, so that features of the target position with rich information can be generated in each direction. ELA processes features in different directions independently for attention prediction, and then integrates them through multiplication, which can accurately capture the location of the region of interest, maintain the dimension of the input channel, and maintain its lightweight characteristics.

[0009] At present, the research on unsupervised domain adaptation defect object detection mainly focuses on the following aspects:

[0010] Adversarial domain alignment: Through adversarial training methods, the features of the source domain and the target domain are aligned as much as possible in the feature space, thereby improving the detection performance of the model in the target domain.

[0011] Self-supervision and pseudo-labeling: Generate pseudo-labels in the target domain and improve the utilization efficiency of target domain data through self-supervised learning.

[0012] Multi-level feature alignment: Not only global features but also local features and multi-level features are aligned to achieve more refined domain adaptation effects.

[0013] Although unsupervised domain adaptation technology has made some progress in defect target detection, it still faces many challenges, such as the complexity of domain differences, the scarcity and diversity of defect samples, etc. Future research can further explore more efficient feature alignment methods, more robust pseudo-label generation mechanisms, and multi-task learning frameworks to improve the performance of unsupervised domain adaptation technology in practical applications.

[0014] In summary, the application of unsupervised domain adaptation in defect target detection has important significance and broad prospects. Through continuous optimization and innovation, related technologies will provide more reliable and efficient solutions for industrial automation detection. Summary of the invention

[0015] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide an industrial parts defect classification method, device, processor and computer-readable storage medium thereof that are reliable, efficient, easy to operate and have a wide range of applications and can achieve inter-domain category adaptation.

[0016] In order to achieve the above objectives, the method, device, processor and computer-readable storage medium for realizing inter-domain category adaptation of industrial part defect classification of the present invention are as follows:

[0017] The industrial parts defect classification method realizing inter-domain category adaptation has the following main features:

[0018] (1) Obtain a source domain dataset with labeled information and a target domain dataset without labeled information and perform preprocessing;

[0019] (2) Input the target domain samples to be tested into the trained neural network detection model for defect detection;

[0020] (3) Remove unnecessary domain adaptation structures in the domain adaptation detection network and perform defect detection on the target domain scene.

[0021] Preferably, the step (1) is specifically as follows:

[0022] A public dataset is obtained, and the public dataset is divided into a source domain and a target domain. The source domain is a dataset made of flat steel images, and the target domain is a NEU surface defect dataset. The four defect categories of plaque, cracking, pitting and inclusions contained in the two datasets are extracted to produce source domain and target domain datasets of the same category but different domains.

[0023] Preferably, the step (2) specifically comprises the following steps:

[0024] (2.1) Inputting the source domain dataset with label information into the basic target detection network for training, and obtaining a pre-trained parameter model to initialize the basic target detection network;

[0025] (2.2) Design the category feature enhancement module, category domain alignment module and ELA attention mechanism module in sequence;

[0026] (2.3) Adding multiple domain adaptation modules to the basic object detection network is called a domain adaptation detection network;

[0027] (2.4) Input a set of source domain and target domain images into the domain adaptation detection network to extract inter-domain consistency features Input to the category feature enhancement module to extract the features of the inter-domain category And add it to the inter-domain consistency feature to get At the same time Input the basic domain discriminator to obtain the mutual information loss function of inter-domain consistency features and inter-domain category features, the focal loss loss function, and the basic domain discrimination loss function;

[0028] (2.5) Input the subsequent feature extraction part to extract global features

[0029] (2.6) Add the ELA attention mechanism to the feature extraction network;

[0030] (2.7) The input category domain alignment module makes the distribution of samples of the same category consistent and calculates the loss function of the final classification stage.

[0031] Preferably, the step (2.6) uses a ResNet50 network as a feature extraction network.

[0032] Preferably, the category feature enhancement module in step (2.2) includes a feature extractor E di and feature extractor E ds , feature extractor E ds Cascade a basic domain adapter to extract the invariant features F between domains ds , feature extractor E di It is used to extract instance-specific distribution invariant features F between domains. di The category feature enhancement module also includes a mutual information loss function to increase F di and F ds The difference between them can achieve feature decoupling capability;

[0033] In the category domain alignment module in step (2.2), the prediction output of the target domain in the network is used as a pseudo label of the target domain data;

[0034] The ELA attention mechanism module in the step (2.2) is specifically as follows: in the ResNet50 feature extraction network, an ELA attention mechanism is added at the end of each stage to perform attention prediction on the features extracted at each stage.

[0035] Preferably, the step (2.3) specifically comprises the following steps:

[0036] (2.3.1) Feature extraction part in Stage 1 of feature extraction network Then connect the category feature enhancement module to Extracted features As input, output domain-specific features

[0037] (2.3.2) In the feature extraction network Stage 2 feature extraction part After that, Extracted features As input to the category domain alignment module;

[0038] (2.3.3) Add the ELA attention mechanism module directly to the end of each stage of the ResNet50 feature extraction network, and represent the output of a stage as R H×W×C , where H, W, and C represent width, height, and number of channels, respectively. Pooling is performed on each channel in the horizontal direction (H, 1) and vertical direction (1, W) to obtain the output of the cth channel at height h. and the output of the cth channel at width w;

[0039] (2.3.4) Input the category domain alignment module to ensure that the distribution of samples of the same category remains consistent and calculate the loss function of the final classification stage.

[0040] Preferably, the mutual information loss function, focal loss function and basic domain discrimination loss function of the inter-domain consistency features and inter-domain category features obtained in step (2.4) are specifically:

[0041] The mutual information loss function, Focalloss loss function and basic domain discrimination loss function of inter-domain consistency features and inter-domain category features are obtained according to the following formula:

[0042]

[0043] L FL = -α(1-p) γ log(p)

[0044] L D =L FL (D b (F b ))+L FL (D ds (F ds ))

[0045] Among them, F di represents the inter-domain category feature, F ds represents the inter-domain consistency feature, F d ' i represents the inter-domain category features after shuffling, T(θ) is a fully connected network, θ is a parameter, and D b and D ds is the domain discriminator, L FL is the Focal Loss loss, p is the predicted probability that the domain discriminator identifies the sample as the target domain, and α and γ are two adjustable hyperparameters.

[0046] Preferably, the step (2.6) specifically comprises the following steps:

[0047] One-dimensional convolution is applied to enhance the position information in the horizontal and vertical directions, and the enhanced position information is processed by group normalization to obtain the sum of the position attention in the horizontal and vertical directions; the final output of ELA is obtained.

[0048] Preferably, the position attention sum in the horizontal and vertical directions is obtained in the step (2.6), specifically:

[0049] The horizontal and vertical position attention sums are obtained according to the following formula:

[0050] y h =σ(G n (F h (z h )))

[0051] y w =σ(G n (F w (z w )))

[0052] Among them, F is a one-dimensional convolution, and its convolution kernel size is set to 7, G n is group normalization, σ is the sigmoid nonlinear activation function;

[0053] The final output of ELA obtained in step (2.6) is specifically:

[0054] The final output of ELA is obtained according to the following formula:

[0055] F ELA =x c ×y h ×y w ;

[0056] Among them, y h and w is the horizontal and vertical position attention sum, x c is the input feature of the cth channel.

[0057] Preferably, the step (2.7) specifically comprises the following steps:

[0058] The final classification stage loss function is calculated in step (2.7), specifically:

[0059] The final classification stage loss function is calculated according to the following formula:

[0060]

[0061] Among them, J(,) is the cross entropy loss function, is the domain adaptation loss, n s is the number of source domain samples, Represents the model's response to the input sample The predicted output is represents the true label of the source domain sample, Measures the distribution difference between the source domain and the target domain. p and q represent the distribution of the source domain and the target domain respectively, and λ is the weight parameter.

[0062] Preferably, the step (3) is specifically as follows:

[0063] The trained model is stripped of all domain adaptation structures and used to perform defect detection on the target domain without label information.

[0064] The device for realizing inter-domain category adaptive industrial parts defect classification is characterized in that the device comprises:

[0065] a processor configured to execute computer executable instructions;

[0066] The memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the various steps of the above-mentioned industrial part defect classification method for realizing inter-domain category adaptation are implemented.

[0067] The processor for implementing inter-domain category adaptive industrial part defect classification is characterized in that the processor is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the various steps of the above-mentioned method for implementing inter-domain category adaptive industrial part defect classification are implemented.

[0068] The computer-readable storage medium is characterized in that a computer program is stored thereon, and the computer program can be executed by a processor to implement the various steps of the above-mentioned industrial part defect classification method for realizing inter-domain category adaptation.

[0069] The industrial parts defect classification method, device, processor and computer-readable storage medium for realizing inter-domain category adaptation of the present invention are adopted. The domain-invariant feature information for the category is decoupled in the middle layer of the feature extraction network and integrated into the original feature to enhance the classification ability, and the ELA attention mechanism is added to solve the key problem of small defect size and difficult feature extraction in parts detection. Finally, during classification, the category is aligned between domains based on the class label to further improve the cross-domain classification capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a schematic diagram of the overall domain adaptation network structure of the industrial part defect classification method that realizes inter-domain category adaptation of the present invention.

[0071] Figure 2It is a schematic diagram of the feature extraction module structure of the industrial parts defect classification method for realizing inter-domain category adaptation of the present invention.

[0072] Figure 3 It is a schematic diagram of the structure of the category feature enhancement module of the industrial parts defect classification method for realizing inter-domain category adaptation of the present invention.

[0073] Figure 4 Schematic diagram of the ELA module structure of the industrial parts defect classification method for realizing inter-domain category adaptation of the present invention.

[0074] Figure 5 The present invention is a flow chart of the industrial parts defect classification method for realizing inter-domain category adaptation. DETAILED DESCRIPTION

[0075] In order to more clearly describe the technical content of the present invention, further description is given below in conjunction with specific embodiments.

[0076] The industrial parts defect classification method for realizing inter-domain category adaptation of the present invention comprises the following steps:

[0077] (1) Obtain a source domain dataset with labeled information and a target domain dataset without labeled information and perform preprocessing;

[0078] (2) Input the target domain samples to be tested into the trained neural network detection model for defect detection;

[0079] (3) Remove unnecessary domain adaptation structures in the domain adaptation detection network and perform defect detection on the target domain scene.

[0080] As a preferred embodiment of the present invention, the step (1) is specifically:

[0081] A public dataset is obtained, and the public dataset is divided into a source domain and a target domain. The source domain is a dataset made of flat steel images, and the target domain is a NEU surface defect dataset. The four defect categories of plaque, cracking, pitting and inclusions contained in the two datasets are extracted to produce source domain and target domain datasets of the same category but different domains.

[0082] As a preferred embodiment of the present invention, the step (2) specifically comprises the following steps:

[0083] (2.1) Inputting the source domain dataset with label information into the basic target detection network for training, and obtaining a pre-trained parameter model to initialize the basic target detection network;

[0084] (2.2) Design the category feature enhancement module, category domain alignment module and ELA attention mechanism module in sequence;

[0085] (2.3) Adding multiple domain adaptation modules to the basic object detection network is called a domain adaptation detection network;

[0086] (2.4) Input a set of source domain and target domain images into the domain adaptation detection network to extract inter-domain consistency features Input to the category feature enhancement module to extract the features of the inter-domain category And add it to the inter-domain consistency feature to get At the same time Input the basic domain discriminator to obtain the mutual information loss function of inter-domain consistency features and inter-domain category features, the focal loss loss function, and the basic domain discrimination loss function;

[0087] (2.5) Input the subsequent feature extraction part to extract global features

[0088] (2.6) Add the ELA attention mechanism to the feature extraction network;

[0089] (2.7) The input category domain alignment module makes the distribution of samples of the same category consistent and calculates the loss function of the final classification stage.

[0090] As a preferred embodiment of the present invention, the step (2.6) uses the ResNet50 network as the feature extraction network.

[0091] As a preferred embodiment of the present invention, the category feature enhancement module in step (2.2) includes a feature extractor E di and feature extractor E ds , feature extractor E ds Cascade a basic domain adapter to extract the invariant features F between domains ds , feature extractor E di It is used to extract instance-specific distribution invariant features F between domains. di The category feature enhancement module also includes a mutual information loss function to increase F di and F ds The difference between them can achieve feature decoupling capability;

[0092] In the category domain alignment module in step (2.2), the prediction output of the target domain in the network is used as a pseudo label of the target domain data;

[0093] The ELA attention mechanism module in the step (2.2) is specifically as follows: in the ResNet50 feature extraction network, an ELA attention mechanism is added at the end of each stage to perform attention prediction on the features extracted at each stage.

[0094] As a preferred embodiment of the present invention, the step (2.3) specifically includes the following steps:

[0095] (2.3.1) Feature extraction part in Stage 1 of feature extraction network Then connect the category feature enhancement module to Extracted features As input, output domain-specific features

[0096] (2.3.2) In the feature extraction network Stage 2 feature extraction part After that, Extracted features As input to the category domain alignment module;

[0097] (2.3.3) Add the ELA attention mechanism module directly to the end of each stage of the ResNet50 feature extraction network, and represent the output of a stage as R H×W×C , where H, W, and C represent width, height, and number of channels, respectively. Pooling is performed on each channel in the horizontal direction (H, 1) and vertical direction (1, W) to obtain the output of the cth channel at height h. and the output of the cth channel at width w;

[0098] (2.3.4) Input the category domain alignment module to ensure that the distribution of samples of the same category remains consistent and calculate the loss function of the final classification stage.

[0099] As a preferred embodiment of the present invention, the mutual information loss function, focal loss function and basic domain discrimination loss function of the inter-domain consistency features and inter-domain category features obtained in step (2.4) are specifically:

[0100] The mutual information loss function, Focalloss loss function and basic domain discrimination loss function of inter-domain consistency features and inter-domain category features are obtained according to the following formula:

[0101]

[0102] L FL = -α(1-p) γ log(p)

[0103] L D =L FL (D b (F b ))+L FL (D ds (Fds ))

[0104] Among them, F di represents the inter-domain category feature, F ds represents the inter-domain consistency feature, F d ' i represents the inter-domain category features after shuffling, T(θ) is a fully connected network, θ is a parameter, and D b and D ds is the domain discriminator, L FL is the Focal Loss loss, p is the predicted probability that the domain discriminator identifies the sample as the target domain, and α and γ are two adjustable hyperparameters.

[0105] As a preferred embodiment of the present invention, the step (2.6) specifically includes the following steps:

[0106] One-dimensional convolution is applied to enhance the position information in the horizontal and vertical directions, and the enhanced position information is processed by group normalization to obtain the sum of the position attention in the horizontal and vertical directions; the final output of ELA is obtained.

[0107] As a preferred embodiment of the present invention, the position attention sum in the horizontal and vertical directions is obtained in the step (2.6), specifically:

[0108] The horizontal and vertical position attention sums are obtained according to the following formula:

[0109] y h =σ(G n (F h (z h )))

[0110] y w =σ(G n (F w (z w )))

[0111] Among them, F is a one-dimensional convolution, and its convolution kernel size is set to 7, G n is group normalization, σ is the sigmoid nonlinear activation function;

[0112] The final output of ELA obtained in step (2.6) is specifically:

[0113] The final output of ELA is obtained according to the following formula:

[0114] F ELA =x c ×y h ×y w ;

[0115] Among them, yh and w is the horizontal and vertical position attention sum, x c is the input feature of the cth channel.

[0116] As a preferred embodiment of the present invention, the step (2.7) specifically comprises the following steps:

[0117] The final classification stage loss function is calculated in step (2.7), specifically:

[0118] The final classification stage loss function is calculated according to the following formula:

[0119]

[0120] Among them, J(,) is the cross entropy loss function, is the domain adaptation loss, n s is the number of source domain samples, Represents the model's response to the input sample The predicted output is represents the true label of the source domain sample, Measures the distribution difference between the source domain and the target domain. p and q represent the distribution of the source domain and the target domain respectively, and λ is the weight parameter.

[0121] As a preferred embodiment of the present invention, the step (3) is specifically as follows:

[0122] The trained model is stripped of all domain adaptation structures and used to perform defect detection on the target domain without label information.

[0123] The device for realizing inter-domain category adaptive industrial parts defect classification of the present invention, wherein the device comprises:

[0124] a processor configured to execute computer-executable instructions;

[0125] The memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the various steps of the above-mentioned industrial part defect classification method for realizing inter-domain category adaptation are implemented.

[0126] The processor of the present invention is used to implement inter-domain category adaptive industrial part defect classification, wherein the processor is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the various steps of the above-mentioned method for implementing inter-domain category adaptive industrial part defect classification are implemented.

[0127] The computer-readable storage medium of the present invention stores a computer program thereon, and the computer program can be executed by a processor to implement the various steps of the above-mentioned industrial part defect classification method for realizing inter-domain category adaptation.

[0128] In order to overcome the above-mentioned defects of the prior art, the present invention provides an industrial parts defect classification method based on inter-domain category adaptation. In order to solve the problem of poor domain generalization ability, most of the previous methods align the source domain and the target domain in the global domain through adversarial methods. Although great success has been achieved, the relatively category-independent alignment will inevitably lose category features and affect the classification results.

[0129] In a specific embodiment of the present invention, a method for industrial parts defect classification based on inter-domain category adaptation is provided. The method for industrial parts defect classification based on inter-domain category adaptation includes: obtaining a target domain sample to be detected; and inputting the target domain sample to be detected into a neural network detection model for defect detection; wherein, the step of training the neural network detection model includes: adding a domain adaptation structure to an initialized classification network to determine a domain adaptation detection network, and the domain adaptation structure includes a category feature enhancement module and a category domain alignment module; introducing a category domain alignment module in the classification stage, and performing inter-domain alignment based on the category as a condition to further improve the cross-domain classification capability; in addition, in actual industrial parts defect detection, many of the real defect categories are extremely small and feature extraction is difficult. In order to improve the detection capability of tiny defects, the ELA attention mechanism is introduced in the feature extraction part, so that the network model pays more attention to the feature information of local defects. Through the method of inter-domain category adaptation, the present invention achieves better detection effect on cross-domain classification tasks than other unsupervised domain adaptation methods, and has important application value for steel surface defect detection.

[0130] The application of unsupervised domain adaptation (UDA) in defect target detection has received increasing attention in recent years. With the advancement of Industry 4.0, automated and intelligent inspection systems play a vital role in the manufacturing industry. However, due to the high cost of data annotation and differences between domains, directly applying models from one domain (such as laboratory environment) to another domain (such as actual production environment) usually leads to a significant drop in performance. This challenge has prompted researchers to explore how to achieve efficient defect detection on unlabeled target domains.

[0131] In the task of defect target detection, domain differences are mainly manifested in differences in imaging conditions, object surface characteristics, defect types, etc. For example, high-quality images collected in a laboratory environment and images collected in an actual production environment may have significant differences in lighting, background, noise, etc. These differences will lead to poor generalization performance of the trained model in the target domain.

[0132] Unsupervised domain adaptation aims to train the model by utilizing the features of the source domain (labeled data) and the target domain (unlabeled data) so that it can also perform well in the target domain. The specific methods mainly include the following categories:

[0133] Adversarial training method: By introducing an adversarial training mechanism, the feature distribution of the source domain and the target domain is aligned as much as possible. Typical methods include Domain-Adversarial Neural Networks (DANN) and Adversarial Discriminative Domain Adaptation (ADDA).

[0134] Feature mapping method: By learning a common feature space, the data of the source domain and the target domain are mapped into the same feature space, thereby reducing the difference between domains. This type of method includes CORAL (Correlation Alignment) and DeepCORAL.

[0135] Self-supervised learning: Use unlabeled data in the target domain for self-supervised learning to improve the model's feature extraction capabilities in the target domain.

[0136] Since most defects in industrial parts defect detection are too small, feature extraction is difficult. After extracting features through multiple convolutional layers, defect information is often lost, which in turn affects the accuracy of classification. In addition, since the overall environment of the parts is single, when a defect occurs, its features will be significantly different from the surrounding features. For this reason, we add the ELA attention mechanism to the feature extraction network to pay more attention to defect information. ELA uses strip pooling in the spatial dimension to extract horizontal and vertical features, maintains an elongated kernel shape to obtain long-distance dependencies, and avoids interference from irrelevant areas on label prediction, so that features of the target position with rich information can be generated in each direction. ELA processes features in different directions independently for attention prediction, and then integrates them through multiplication, which can accurately capture the location of the region of interest, maintain the dimension of the input channel, and maintain its lightweight characteristics.

[0137] At present, the research on unsupervised domain adaptation defect object detection mainly focuses on the following aspects:

[0138] Adversarial domain alignment: Through adversarial training methods, the features of the source domain and the target domain are aligned as much as possible in the feature space, thereby improving the detection performance of the model in the target domain.

[0139] Self-supervision and pseudo-labeling: Generate pseudo-labels in the target domain and improve the utilization efficiency of target domain data through self-supervised learning.

[0140] Multi-level feature alignment: Not only global features but also local features and multi-level features are aligned to achieve more refined domain adaptation effects.

[0141] Although unsupervised domain adaptation technology has made some progress in defect target detection, it still faces many challenges, such as the complexity of domain differences, the scarcity and diversity of defect samples, etc. Future research can further explore more efficient feature alignment methods, more robust pseudo-label generation mechanisms, and multi-task learning frameworks to improve the performance of unsupervised domain adaptation technology in practical applications.

[0142] In summary, the application of unsupervised domain adaptation in defect target detection has important significance and broad prospects. Through continuous optimization and innovation, related technologies will provide more reliable and efficient solutions for industrial automation detection.

[0143] First, Figure 1 As shown in Figure 1, in order to obtain the ability to achieve efficient defect detection in the unlabeled target domain through the labeled source domain, multiple domain adaptation modules are added to the basic detection network. The overall structure of the domain adaptation detection network is shown in Figure 1. Figure 1 As shown, specifically:

[0144] First, we designed a feature extraction module. The feature extraction network of this module is ResNet50, which consists of four convolution modules stage1 to stage4. Each convolution module consists of a different number of bottleneck blocks. We divide the entire feature extraction network into two parts: and For stage1 and stage2 parts, It is for stage3 and stage4, and adds the ELA attention mechanism after each stage. and Structure Figure 2 .

[0145] Secondly, we designed a category feature enhancement module, such as Figure 3 As shown, this module consists of two feature extractors E di and E ds and a domain discriminator with a gradient reversal layer. dsCombined with the domain discriminator, it is used to extract the inter-domain consistency feature F for the global domain. ds , E di It is used to extract the feature information F between domains relative to the category di , and also includes a mutual information loss function to increase F di and F ds The difference between them is used to achieve feature decoupling capability, and instance-level invariant features from different domains are aligned to ensure inter-domain feature consistency and improve the cross-domain performance of the model. Finally, the extracted inter-domain category features F di and after Extracted features By adding them together, we can obtain features that still contain inter-domain category-specific information while keeping the inter-domain consistency features intact, thereby enhancing subsequent classification capabilities.

[0146] Furthermore, the ELA attention mechanism module is designed, such as Figure 4 As shown, specifically, consider the output of a stage, represented by R H×W×C , H, W and C represent width, height and number of channels respectively, then pooling is performed on each channel in two spatial ranges in the horizontal direction (H, 1) and vertical direction (1, W) to obtain the output of the cth channel at height h (h), and the output of the cth channel at width w; In order to effectively utilize the sum containing global information and precise position information, a one-dimensional convolution is applied to enhance the position information in the horizontal and vertical directions. Subsequently, group normalization is used to process the enhanced position information, thereby obtaining the position attention sum in the horizontal and vertical directions.

[0147] Finally, a category adaptation module is designed. In the category domain alignment module, the prediction output of the target domain in the network is used as the pseudo label of the target domain data. s and D t Divided into C subdomains and where c∈{1,2,…,C} represents the class label, and The distribution of p c and q c The purpose of inter-domain category adaptation is to keep the distribution of samples of the same category consistent and use the local maximum mean difference (LMMD) to estimate the distribution difference of inter-domain categories;

[0148] Please refer to Figure 5 , Figure 5 A system flow chart of a domain adaptive part surface defect detection method based on multi-dimensional decoupling confrontation provided according to some embodiments of the present invention is shown.

[0149] like Figure 5 As shown in step S0, first obtain the labeled source domain and unlabeled target domain datasets. Then, as Figure 5 As shown in step S1, the source domain dataset with label information is used to train the basic object detection network. Then, as Figure 5 As shown in step S2 of , a domain adaptation module is added to the basic object detection network, and the pre-adaptation detection network is trained using the labeled source domain and the unlabeled target domain. Figure 5 As shown in step S3, the domain adaptation module in the domain adaptation detection network is removed, and the trained basic target detection network is used to perform defect detection on the target domain data.

[0150] Furthermore, a public dataset is obtained, which is divided into a source domain and a target domain. The source domain is a dataset made of flat steel images taken by Severstal Steel Company using a high-frequency camera, which collects four typical flat steel surface defects, namely pitting (Ps), oil spots (Os), inclusions (In) and scratches (Sc); the target domain is the NEU surface defect dataset produced by Northeastern University, which collects six typical hot-rolled strip surface defects, namely oxide scale indentation (Rs), plaques (Pa), cracks (Cr), pitting (Ps), inclusions (In) and scratches (Sc); and the three defect categories of pitting (Ps), inclusions (In) and scratches (Sc) contained in the two datasets are extracted to produce source domain and target domain datasets of the same category but different domains; in addition, this method can be applied to cross-domain migration between different types of precision parts in the aerospace field, such as pre-adaptive migration of defect detection from impeller parts to casing parts, and can perform highly adaptive migration in defect detection of scratches, scratches, built-up edges, chatter marks, etc.

[0151] Furthermore, the defect detection accuracy of the model is determined based on the number of defects in the detected test data set and the number of all defects, and the model with the highest accuracy is determined as the required high-precision target detection model. The accuracy is the performance evaluation indicator. Specifically:

[0152] The performance evaluation index uses the average precision AP (average precision), which mainly includes two values: precision and recall. The precision represents the proportion of samples that actually belong to the category in the samples predicted to be positive in different categories. The formula is:

[0153]

[0154] Among them, TP (True Positive) represents the number of samples that correctly identify defective targets; FN (False Negative) represents the number of samples that do not identify defective targets; FP (False Positive) represents the number of samples that incorrectly identify defective targets, and recall is the recall rate, which represents the ratio of the number of correctly detected targets to the total number of targets in the test set. The formula is:

[0155]

[0156] The denominator of recall is true positives + false negatives, that is, the total number of defective samples, and the formula for average accuracy is:

[0157]

[0158] The model can be evaluated by the average precision mean MAP, which represents the average accuracy of all categories in the model. The calculation formula is:

[0159]

[0160] Please refer to Table 1, which shows the results of the defect detection accuracy of unlabeled target domain data by the domain adaptive part surface defect detection method based on multi-dimensional decoupling adversarial provided according to some embodiments of the present invention.

[0161] Table 1

[0162] Plaque Cracking Pitting Inclusion Avg Source only 85.6 87.3 91.2 86.4 87.6 DANN 88.3 90.1 93.3 89.5 90.1 CDAN 90.5 91.8 92.4 89.6 91.1 DSAN 90.2 93.5 95.3 92.7 93.0 Ours 91.4 95.6 94.5 95.8 94.3

[0163] Table 1 shows the results of the defect detection accuracy of the unlabeled target domain data of the present invention and the comparison with the results of some public methods; Source only is the result of the basic target detection network directly predicting the unlabeled target domain after being trained in the labeled source domain, DANN, CDAN and DSAN are some public existing methods for domain adaptive target detection. It can be seen that the present invention has obvious improvements, and the defect detection accuracy on the unlabeled target domain data set has been improved, which has practical value for cross-domain detection of part defects.

[0164] The specific implementation scheme of this embodiment can refer to the relevant description in the above embodiment, which will not be repeated here.

[0165] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0166] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0167] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0168] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0169] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the corresponding program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0170] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0171] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0172] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0173] The industrial parts defect classification method, device, processor and computer-readable storage medium for realizing inter-domain category adaptation of the present invention are adopted. The domain-invariant feature information for the category is decoupled in the middle layer of the feature extraction network and integrated into the original feature to enhance the classification ability, and the ELA attention mechanism is added to solve the key problem of small defect size and difficult feature extraction in parts detection. Finally, during classification, the category is aligned between domains based on the class label to further improve the cross-domain classification capability.

[0174] In this specification, the present invention has been described with reference to specific embodiments thereof. However, it is apparent that various modifications and variations may be made without departing from the spirit and scope of the present invention. Therefore, the specification and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A method for industrial parts defect classification that achieves inter-domain category adaptation, characterized in that: The method comprises the following steps: (1) Obtain a source domain dataset with labeled information and a target domain dataset without labeled information and perform preprocessing; (2) Input the target domain samples to be tested into the trained neural network detection model for defect detection; (3) Remove unnecessary domain adaptation structures in the domain adaptation detection network and perform defect detection on the target domain scene.

2. The industrial parts defect classification method for realizing inter-domain category adaptation according to claim 1 is characterized in that: The step (1) is specifically as follows: A public dataset is obtained, and the public dataset is divided into a source domain and a target domain. The source domain is a dataset made of flat steel images, and the target domain is a NEU surface defect dataset. The four defect categories of plaque, cracking, pitting and inclusions contained in the two datasets are extracted to produce source domain and target domain datasets of the same category but different domains.

3. The industrial parts defect classification method for realizing inter-domain category adaptation according to claim 1 is characterized in that: The step (2) specifically comprises the following steps: (2.1) Inputting the source domain dataset with label information into the basic target detection network for training, and obtaining a pre-trained parameter model to initialize the basic target detection network; (2.2) Design the category feature enhancement module, category domain alignment module and ELA attention mechanism module in sequence; (2.3) Adding multiple domain adaptation modules to the basic object detection network is called a domain adaptation detection network; (2.4) Input a set of source domain and target domain images into the domain adaptation detection network to extract inter-domain consistency features Input to the category feature enhancement module to extract the features of the inter-domain category And add it to the inter-domain consistency feature to get At the same time Input the basic domain discriminator to obtain the mutual information loss function of inter-domain consistency features and inter-domain category features, Focalloss loss function and basic domain discrimination loss function; (2.5) Input the subsequent feature extraction part to extract global features (2.6) Add the ELA attention mechanism to the feature extraction network; (2.7) The input category domain alignment module makes the distribution of samples of the same category consistent and calculates the loss function of the final classification stage.

4. The industrial parts defect classification method for realizing inter-domain category adaptation according to claim 3 is characterized in that: The step (2.6) uses the ResNet50 network as the feature extraction network.

5. The industrial parts defect classification method for realizing inter-domain category adaptation according to claim 3 is characterized in that: The category feature enhancement module in step (2.2) includes a feature extractor E di and feature extractor E ds , feature extractor E ds Cascade a basic domain adapter to extract the invariant features F between domains ds , feature extractor E di It is used to extract instance-specific distribution invariant features F between domains. di The category feature enhancement module also includes a mutual information loss function to increase F di and F ds The difference between them can achieve feature decoupling capability; In the category domain alignment module in step (2.2), the prediction output of the target domain in the network is used as a pseudo label of the target domain data; The ELA attention mechanism module in the step (2.2) is specifically as follows: in the ResNet50 feature extraction network, an ELA attention mechanism is added at the end of each stage to perform attention prediction on the features extracted at each stage.

6. The industrial parts defect classification method for realizing inter-domain category adaptation according to claim 3 is characterized in that: The step (2.3) specifically comprises the following steps: (2.3.1) Feature extraction part in Stage 1 of feature extraction network Then connect the category feature enhancement module to The extracted features F b 1 As input, output domain-specific features (2.3.2) In the feature extraction network Stage 2 feature extraction part After that, Extracted features As input to the category domain alignment module; (2.3.3) Add the ELA attention mechanism module directly to the end of each stage of the ResNet50 feature extraction network, and represent the output of a stage as R H×W×C , where H, W, and C represent width, height, and number of channels, respectively. Pooling is performed on each channel in the horizontal direction (H, 1) and vertical direction (1, W) to obtain the output of the cth channel at height h. and the output of the cth channel at width w; (2.3.4) Input the category domain alignment module to ensure that the distribution of samples of the same category remains consistent and calculate the loss function of the final classification stage.

7. The industrial parts defect classification method for realizing inter-domain category adaptation according to claim 3 is characterized in that: In the step (2.4), the mutual information loss function of the inter-domain consistency feature and the inter-domain category feature, the Focalloss loss function and the basic domain discrimination loss function are obtained, which are specifically: The mutual information loss function, Focalloss loss function and basic domain discrimination loss function of inter-domain consistency features and inter-domain category features are obtained according to the following formula: L FL =-α(1-p) γ log(p) L D =L FL (D b (F b ))+L FL (D ds (F ds )) Among them, F di represents the inter-domain category feature, F ds represents the inter-domain consistency feature, F d ' i represents the inter-domain category features after shuffling, T(θ) is a fully connected network, θ is a parameter, and D b and D ds is the domain discriminator, L FL is the Focal Loss loss, p is the predicted probability that the domain discriminator identifies the sample as the target domain, and α and γ are two adjustable hyperparameters.

8. The industrial parts defect classification method for realizing inter-domain category adaptation according to claim 3 is characterized in that: The step (2.6) specifically comprises the following steps: One-dimensional convolution is applied to enhance the position information in the horizontal and vertical directions, and the enhanced position information is processed by group normalization to obtain the sum of the position attention in the horizontal and vertical directions; the final output of ELA is obtained.

9. The industrial parts defect classification method for realizing inter-domain category adaptation according to claim 8 is characterized in that: The position attention and the vertical attention in the horizontal direction are obtained in the step (2.6), specifically: The horizontal and vertical position attention sums are obtained according to the following formula: y h =σ(G b (F h (z h ))) y w =σ(G b (F w (z w ))) Among them, F is a one-dimensional convolution, and its convolution kernel size is set to 7, G n is group normalization, σ is the sigmoid nonlinear activation function; The final output of ELA obtained in step (2.6) is specifically: The final output of ELA is obtained according to the following formula: F ELA =x c ×y h ×y w ; Among them, y h and w is the horizontal and vertical position attention sum, x c is the input feature of the cth channel.

10. The industrial parts defect classification method for realizing inter-domain category adaptation according to claim 3 is characterized in that: The step (2.7) specifically comprises the following steps: The final classification stage loss function is calculated in step (2.7), specifically: The final classification stage loss function is calculated according to the following formula: Among them, J(,) is the cross entropy loss function, is the domain adaptation loss, n s is the number of source domain samples, Represents the model's response to the input sample The predicted output is represents the true label of the source domain sample, Measures the distribution difference between the source domain and the target domain. p and q represent the distribution of the source domain and the target domain respectively, and λ is the weight parameter.

11. The industrial parts defect classification method for realizing inter-domain category adaptation according to claim 1 is characterized in that: The step (3) is specifically as follows: The trained model is stripped of all domain adaptation structures and used to perform defect detection on the target domain without label information.

12. A device for realizing inter-domain category adaptive industrial parts defect classification, characterized in that: The device comprises: a processor configured to execute computer executable instructions; A memory storing one or more computer executable instructions, wherein when the computer executable instructions are executed by the processor, the steps of the industrial part defect classification method for realizing inter-domain category adaptation described in any one of claims 1 to 11 are implemented.

13. A processor for realizing inter-domain category adaptive industrial parts defect classification, characterized in that: The processor is configured to execute computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the industrial part defect classification method for realizing inter-domain category adaptation described in any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the various steps of the industrial part defect classification method for realizing inter-domain category adaptation as described in any one of claims 1 to 11.

Images