A method and system for detecting surface defects in complex industrial components

Abstract

A method and system for detecting surface defects in complex industrial components are disclosed. The method includes: collecting a large number of real component defect samples and preprocessing them to obtain a training dataset covering multiple component types, multiple detection surfaces, and multiple defect morphologies; constructing a defect detection model, including a pre-trained visual encoder, a pre-trained text encoder, a three-part hierarchical hybrid expert model, a decoder, and a detection head; training the defect detection model using the training dataset to obtain a trained defect detection model; and deploying the trained defect detection model to an equipment to perform component defect detection and obtain the detection results. This invention is adaptable to various component and defect types, significantly improving its versatility and adaptability in complex production environments.

Description

Technical Field

[0001] This invention relates to the field of industrial defect detection technology, and in particular to a method and system for detecting surface defects in complex industrial components. Background Technology

[0002] Industrial defect detection is a crucial link in intelligent manufacturing and the safe operation of equipment, and is widely used in various fields such as automobile manufacturing and energy equipment. However, mechanical components are subjected to high loads and strong vibrations for extended periods. If undetected surface defects exist, they may lead to equipment downtime or even major safety accidents. Therefore, component defect detection is of great significance for the timely detection of safety hazards in industrial equipment, ensuring product quality and reliability, and achieving high-quality development in industrial manufacturing.

[0003] Deep learning methods have long dominated image recognition and detection tasks, achieving significant progress by learning defect features through large-scale training samples. However, in industrial defect detection scenarios, existing methods primarily design dedicated detection models for specific scenarios and tasks. When the types of objects to be detected increase and the defect morphologies become more diverse, traditional methods struggle to achieve unified detection across tasks, resulting in incomplete and unreliable detection results. Furthermore, the scarcity of defect samples makes it difficult for existing methods to adequately train models, leading to incomplete detection or false detections in practical applications.

[0004] Defects and shortcomings of existing technology: 1. Insufficient adaptability: Existing methods are mostly designed for specific tasks and lack a unified detection framework. They are difficult to deal with multiple component types and defect types at the same time, resulting in limited applicability in complex production environments.

[0005] 2. Lack of semantic guidance and dynamic adjustment capabilities: Most existing models rely solely on visual features and lack explicit guidance from external knowledge such as textual semantics. The models cannot dynamically adjust according to different detection needs, which limits detection accuracy and robustness. Summary of the Invention

[0006] This invention provides a method and system for detecting surface defects in complex industrial components to solve the technical problems mentioned in the background art.

[0007] To achieve the above objectives, the technical solution of the present invention is implemented as follows: This invention provides a method for detecting surface defects in complex industrial components, comprising the following steps: S1. Collect multiple real component defect samples and preprocess them to obtain a training dataset covering multiple component types, multiple detection surfaces and multiple defect morphologies; S2. Construct a defect detection model, including a pre-trained visual encoder, a pre-trained text encoder, a three-component hierarchical hybrid expert model, and a defect detector. The pre-trained visual encoder and pre-trained text encoder are connected in parallel, then connected in parallel with the three-component hierarchical hybrid expert model, and finally connected in series with the defect detector. S3. Train the defect detection model using the training dataset to obtain the trained defect detection model; S4. Deploy the trained defect detection model to the device and perform component defect detection to obtain the detection results.

[0008] Furthermore, the expression for the training dataset in S1 is as follows: ; in, Represents the training dataset. Indicates preprocessing operations; This represents multiple real component defect samples.

[0009] Furthermore, the three-component layered hybrid expert model includes component hybrid expert sub-models, detection surface hybrid expert sub-models, and defect hybrid expert sub-models connected in parallel; the defect detector includes a decoder and a detection head.

[0010] Furthermore, the component hybrid expert sub-model includes a first router network and multiple component expert units. The first router network is connected in parallel with the multiple component expert units. Each component expert unit includes a bottleneck layer, a first normalization layer, a first multi-head attention module, a second normalization layer, and a first feedforward neural network. The output of the bottleneck layer is designed with two branches. The first branch of the bottleneck layer is connected in series with the first normalization layer and the first multi-head attention module. The second branch of the bottleneck layer is connected to the output of the first multi-head attention module through a summation operation and then split into two branches, namely a summation first branch and a summation second branch. The summation first branch is connected in series with the second normalization layer and the first feedforward neural network. The summation second branch is connected to the output of the first feedforward neural network through a summation operation. The detection surface hybrid expert sub-model includes a second router network and multiple detection surface expert units. The second router network is connected in parallel with each of the multiple detection surface expert units. Each detection surface expert unit includes a first convolutional layer, a multi-scale attention layer, a pooling layer, a second multi-head attention module, a third normalization layer, and a second feedforward neural network. The output of the first convolutional layer is designed with two branches. The first branch of the first convolutional layer is connected in series with the multi-scale attention layer, the pooling layer, and the second multi-head attention module. The second branch of the first convolutional layer is connected to the output of the second multi-head attention module through a summation operation and then split into two branches: a summation third branch and a summation fourth branch. The summation third branch is connected in series with the third normalization layer and the second feedforward neural network. The summation fourth branch is connected to the output of the second feedforward neural network through a summation operation. The defect hybrid expert sub-model includes a third router network and multiple defect expert units. The third router network is connected in parallel with each of the multiple defect expert units. Each defect expert unit includes a second convolutional layer, a spatial selection module, a fourth normalization layer, a third multi-head attention module, a fifth normalization layer, and a third feedforward neural network. The output of the second convolutional layer is designed with two branches. The first branch of the second convolutional layer is connected in series with the spatial selection module, the fourth normalization layer, and the third multi-head attention module. The second branch of the second convolutional layer is connected to the output of the third multi-head attention module through a summation operation and then split into two branches: the fifth summation branch and the sixth summation branch. The fifth summation branch is connected in series with the fifth normalization layer and the third feedforward neural network. The sixth summation branch is connected to the output of the third feedforward neural network through a summation operation.

[0011] Furthermore, step S3 specifically includes the following steps: S31, Use the training dataset The defect image inside is input into the defect detection model. After passing through a pre-trained visual encoder and a pre-trained text encoder and undergoing multimodal feature fusion, a multimodal fusion feature x is obtained. S32. Input the multimodal fusion feature x into the three-component hierarchical hybrid expert model to obtain the global differential features. Differences and local areas The final fused features are generated through a self-attention mechanism. ; S33, Final fusion features The input is fed into the decoder to generate a multi-scale feature map, which is then fed into the detection head to obtain the center point location of the predicted defect bounding box. Length and width and categories; S34. Based on the predicted center point position of the defect bounding box Length and width The total loss function is constructed based on the categories, and the parameters of the defect detection model are adjusted according to the total loss function; S35, repeat S31 to S34 until the set iteration stopping condition is met, and the trained defect detection model is obtained.

[0012] Furthermore, step S31 specifically includes the following steps: S311, Transfer the training dataset The defect image is input into the defect detection model, and the visual features of the defect image are extracted by a pre-trained visual encoder. Visual features Includes general and task-specific features required for defect detection in different components: visual features The expression is as follows: ; in, This represents a pre-trained visual encoder; S312. Input the defect image into a pre-trained text encoder to obtain the semantic features of the component type corresponding to the defect image. and semantic features of text prompts on the defective image's inspected surface. The expressions are as follows: ; ; in, It is a text hint about the component type. This is a text prompt for the surface to be inspected; This represents a pre-trained text encoder; S313. Introduce a cross-modal attention mechanism and utilize it to focus on visual features. semantic features semantic features Multimodal feature fusion is performed to obtain the multimodal fused feature x, expressed as follows: ; in, This indicates a cross-modal feature fusion operation.

[0013] Furthermore, step S32 specifically includes the following steps: S321, Fusing features across multiple modalities The input is fed into the component hybrid expert sub-model within the three-component hierarchical hybrid expert model to extract the global dissimilarity features of different types of components, thus obtaining the global dissimilarity features. ; S322. The multimodal fusion feature x is input into the detection surface hybrid expert sub-model within the three-component hierarchical hybrid expert model. The detection surface hybrid expert sub-model processes all image patch features of the multimodal fusion feature x through an introduced multi-scale feature aggregation mechanism, and extracts the differential features brought about by textures and materials under various receptive fields for different detection surfaces. ; S323. Input the multimodal fusion feature x into the defect hybrid expert sub-model within the three-component hierarchical hybrid expert model. The defect hybrid expert sub-model captures the category semantic features after multimodal fusion through an image patch feature filtering mechanism. By establishing long-range dependencies between image patch features and spatial attention maps, a spatial attention map is constructed. Then, based on the image patch features with the highest scores in the spatial attention map, the most critical local regions of the defect hybrid expert sub-model are extracted. ; S324. Introduce a self-attention mechanism for globally differential features. Differences and local areas The fusion process is performed to generate the final fusion feature. The expression is as follows: ; in, , , These are the weights obtained through adaptive learning.

[0014] Furthermore, the expression for the total loss function in S34 is as follows: ; in, Represents the total loss function. Represents the cross-modal contrastive loss function; This represents the expert regularization loss function; This represents the defect location loss function; This represents the balanced cross-entropy loss function; , , , These represent the weights of the corresponding loss functions.

[0015] Furthermore, the expert regularization loss function The expression is as follows: + + ; ; in, , , Let i and j represent the regularization loss functions of the component hybrid expert sub-model, the detection surface hybrid expert sub-model, and the defect hybrid expert sub-model, respectively; and the calculation methods of these regularization loss functions are similar; i and j represent the indices of different component expert elements within the component hybrid expert sub-model, respectively; E represents the number of component expert elements in the component hybrid expert sub-model. , These represent the outputs of the i-th and j-th component expert units, respectively. Indicates cosine similarity; Defect localization loss function The expression is as follows: ; in, This represents a smooth L1 loss function; Indicates the location of the center point of the defect boundary box; This indicates the length and width of the bounding box of the defect annotation.

[0016] In another aspect, the present invention provides a surface defect detection system for complex industrial components, which is configured to perform the above-described surface defect detection method for complex industrial components.

[0017] The beneficial effects of this invention are: This invention discloses a method for detecting surface defects in complex industrial components, which effectively improves the accuracy and robustness of defect detection and enhances the scalability of the model. The main advantages are summarized as follows: 1. This invention designs a three-part hierarchical hybrid expert model, including parallel component hybrid expert sub-models, inspection surface hybrid expert sub-models, and defect hybrid expert sub-models. This model can extract differentiated features based on different component types, inspection surface characteristics, and defect morphologies. Each expert sub-model embeds a router network with a built-in gated routing mechanism. Under the dynamic selection of the gated routing mechanism, the model does not require customized design for a single task and can adapt to multiple component and defect types, significantly improving its versatility and adaptability in complex production environments.

[0018] 2. This invention introduces a pre-trained visual encoder and a pre-trained text encoder, and fuses visual features with semantic cues through a cross-modal attention mechanism, enabling the defect detection model to adaptively focus on key regions according to different detection tasks. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the operation of the defect detection model in this invention. Figure 2 This is a structural block diagram of the component hybrid expert sub-model in this invention; Figure 3 This is a structural block diagram of the detection surface mixing expert sub-model in this invention; Figure 4 This is a structural block diagram of the defect hybrid expert sub-model in this invention; Figure 5 This is a schematic diagram of the detection results in an embodiment of the present invention. Figure 1 ; Figure 6 This is a schematic diagram of the detection results in an embodiment of the present invention. Figure 2 . Detailed Implementation

[0020] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0021] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] This invention provides a method and system for detecting surface defects in complex industrial components, aiming to address the shortcomings of existing industrial defect detection methods in terms of insufficient adaptability and limited generalization across various component types and defect scenarios. By introducing a cross-modal attention mechanism and a three-component hierarchical hybrid expert model, this invention achieves dynamic selection and fusion of features at different levels, effectively improving the accuracy and robustness of defect detection. The following provides a detailed description of the various technical features and implementation schemes of this invention. The technical solution adopted by this invention is as follows: Reference Figure 1 This application provides a method for detecting surface defects in complex industrial components, comprising the following steps: S1. Collect multiple real component defect samples and preprocess them to obtain a training dataset covering multiple component types, multiple detection surfaces and multiple defect morphologies; S2. Construct a defect detection model, including a pre-trained visual encoder, a pre-trained text encoder, a three-component hierarchical hybrid expert model (MoE), and a defect detector. The pre-trained visual encoder and the pre-trained text encoder are connected in parallel, then connected in parallel with the three-component hierarchical hybrid expert model, and finally connected in series with the defect detector. In this embodiment, the visual encoder can be selected from the visual encoder CLIP-ViT, the visual encoder Swin-Transformer, or the visual encoder ConvNeXt, but the visual encoder CLIP-ViT is preferred. The pre-trained visual encoder CLIP-ViT is obtained by pre-training with large-scale image-text pairing data collected in advance. The text encoder can be BERT, RoBERTa, or ERNIE, but BERT is preferred. The pre-trained text encoder BERT is obtained by pre-training with large-scale image-text pairing data collected in advance. S3. Train the defect detection model using the training dataset to obtain the trained defect detection model; S4. Deploy the trained defect detection model to the device and perform component defect detection to obtain the detection results. See the results below. Figure 5 and Figure 6 As shown.

[0023] In some embodiments, the expression for the training dataset in S1 is as follows: ; in, Represents the training dataset. Indicates preprocessing operations; This represents multiple real component defect samples.

[0024] In some embodiments, the three-component layered hybrid expert model includes a component hybrid expert sub-model, a detection surface hybrid expert sub-model, and a defect hybrid expert sub-model connected in parallel; the defect detector includes a decoder and a detection head.

[0025] In some embodiments, refer to Figure 2 The component hybrid expert sub-model includes a first router network and multiple component expert units. The first router network is connected in parallel with the multiple component expert units. Each component expert unit includes a bottleneck layer, a first normalization layer, a first multi-head attention module, a second normalization layer, and a first feedforward neural network. The output of the bottleneck layer is designed with two branches. The first branch of the bottleneck layer is connected in series with the first normalization layer and the first multi-head attention module. The second branch of the bottleneck layer is connected to the output of the first multi-head attention module through a summation operation and then split into two branches, namely the summation first branch and the summation second branch. The summation first branch is connected in series with the second normalization layer and the first feedforward neural network, respectively. The summation second branch is connected to the output of the first feedforward neural network through a summation operation. Reference Figure 3The detection surface hybrid expert sub-model includes a second router network and multiple detection surface expert units. The second router network is connected in parallel with the multiple detection surface expert units. Each detection surface expert unit includes a first convolutional layer, a multi-scale attention layer, a pooling layer, a second multi-head attention module, a third normalization layer, and a second feedforward neural network. The output of the first convolutional layer is designed with two branches. The first branch of the first convolutional layer is connected in series with the multi-scale attention layer, the pooling layer, and the second multi-head attention module. The second branch of the first convolutional layer is connected to the output of the second multi-head attention module through a summation operation and then split into two branches, namely the summation third branch and the summation fourth branch. The summation third branch is connected in series with the third normalization layer and the second feedforward neural network. The summation fourth branch is connected to the output of the second feedforward neural network through a summation operation. Reference Figure 4 The defect hybrid expert sub-model includes a third router network and multiple defect expert units. The third router network is connected in parallel with the multiple defect expert units. Each defect expert unit includes a second convolutional layer, a spatial selection module, a fourth normalization layer, a third multi-head attention module, a fifth normalization layer, and a third feedforward neural network. The output of the second convolutional layer is designed with two branches. The first branch of the second convolutional layer is connected in series with the spatial selection module, the fourth normalization layer, and the third multi-head attention module. The second branch of the second convolutional layer is connected to the output of the third multi-head attention module through a summation operation and then split into two branches, namely the fifth summation branch and the sixth summation branch. The fifth summation branch is connected in series with the fifth normalization layer and the third feedforward neural network. The sixth summation branch is connected to the output of the third feedforward neural network through a summation operation.

[0026] The first router network, the second router network, and the third router network all have built-in gated routing mechanisms. Through the gated routing mechanism, the most relevant expert units (including component expert units, inspection surface expert units, and defect expert units) are dynamically selected to achieve hierarchical extraction of component difference features, multi-scale inspection surface features, and fine-grained defect features. Taking the component hybrid expert sub-model as an example, the calculation process of the gated routing mechanism is explained. The calculation process of the gated routing mechanism in the detection surface hybrid expert sub-model and the defect hybrid expert sub-model is similar. The gated routing mechanism within the component hybrid expert sub-model calculates the weight distribution of each component expert element and activates it. The component expert element with the highest weight is selected. The activated component expert elements are then summed according to their weight distribution. The expression for the weighted summation is as follows: ; in, This represents the weighted summation result. Represents the i-th component expert unit; This indicates the network score of the first router for the multimodal fusion feature x. The proposed hybrid expert sub-model for components, hybrid expert sub-model for detection surfaces, and hybrid expert sub-model for defects can extract knowledge from the most relevant expert models and also utilize the relevant knowledge of other experts to assist in reasoning, thus avoiding the simultaneous activation of all expert units and reducing computational complexity.

[0027] In some embodiments, S3 specifically includes the following steps: S31, Use the training dataset The defect image inside is input into the defect detection model. After passing through a pre-trained visual encoder and a pre-trained text encoder and undergoing multimodal feature fusion, a multimodal fusion feature x is obtained. S32. Input the multimodal fusion feature x into the three-component hierarchical hybrid expert model to obtain the global differential features. Differences and local areas The final fused features are generated through a self-attention mechanism. ; S33, Final fusion features The input is fed into the decoder, where it is restored to its original resolution through deconvolution layers and upsampling operations to generate a multi-scale feature map. This multi-scale feature map is then fed into the detection head to obtain the center point location of the predicted defect bounding box. Length and width and categories; S34. Based on the predicted center point position of the defect bounding box Length and width The total loss function is constructed based on the categories, and the parameters of the defect detection model are adjusted according to the total loss function; S35, repeat S31 to S34 until the set iteration stopping condition is met, and the trained defect detection model is obtained.

[0028] In some embodiments, S31 specifically includes the following steps: S311, Transfer the training dataset The internal defect image is input into the defect detection model, and its visual features are extracted by a pre-trained visual encoder. Visual features Includes general and task-specific features required for defect detection in different components: visual features The expression is as follows: ; in, This represents a pre-trained visual encoder; S312. Input the defect image into a pre-trained text encoder to obtain the semantic features of the component type corresponding to the defect image. and semantic features of text prompts on the defective image's inspected surface. The expressions are as follows: ; ; in, It is a text hint about the component type. This is a text prompt for the surface to be inspected; Indicates a text encoder; S313. Introduce a cross-modal attention mechanism and utilize it to focus on visual features. semantic features semantic features Multimodal feature fusion is performed to obtain the multimodal fused feature x, expressed as follows: ; in, This indicates a cross-modal feature fusion operation.

[0029] In some embodiments, S32 specifically includes the following steps: S321. Input the multimodal fusion feature x into the component hybrid expert sub-model within the three-component hierarchical hybrid expert model, extract the global differential features of different types of components, and obtain the global differential features. ; S322. The multimodal fusion feature x is input into the detection surface hybrid expert sub-model within the three-component hierarchical hybrid expert model. The detection surface hybrid expert sub-model processes all image patch features of the multimodal fusion feature x through an introduced multi-scale feature aggregation mechanism, and extracts the differential features brought about by textures and materials under various receptive fields for different detection surfaces. ; S323. Input the multimodal fusion feature x into the defect hybrid expert sub-model within the three-component hierarchical hybrid expert model. The defect hybrid expert sub-model captures the category semantic features after multimodal fusion through an image patch feature filtering mechanism. By establishing long-range dependencies between image patch features and spatial attention maps, a spatial attention map is constructed. Then, based on the image patch features with the highest scores in the spatial attention map, the most critical local regions of the defect hybrid expert sub-model are extracted. ; S324. Introduce a self-attention mechanism for globally differential features. Differences and local areas The fusion process is performed to generate the final fusion feature. The expression is as follows: ; in, , , These are the weights obtained through adaptive learning.

[0030] In some embodiments, the expression for the total loss function in S34 is as follows: ; in, Represents the total loss function. Represents the cross-modal contrastive loss function; This represents the expert regularization loss function; This represents the defect location loss function; This represents the balanced cross-entropy loss function; , , , These represent the weights of the corresponding loss functions, used to adjust the contribution of each loss term to the total loss.

[0031] Cross-modal contrastive loss function This technology is used to enhance the consistency between visual features and textual semantic features, solve the problem of cross-modal semantic alignment, maximize the similarity between correctly paired image-text prompt features, minimize the similarity of incorrect pairs, and enhance the cross-modal alignment capability of features. Balanced cross-entropy loss function Used to quantify the error in defect category identification and alleviate the problem of imbalanced samples; Defect localization loss function Used to quantify the error in locating the defect area; Expert regularization loss function This is used to constrain the output differences between different expert units, ensuring that each expert unit focuses on learning different features.

[0032] In some embodiments, the expert regularization loss function The expression is as follows: + + ; ; in, , , Let i and j represent the regularization loss functions of the component hybrid expert sub-model, the detection surface hybrid expert sub-model, and the defect hybrid expert sub-model, respectively; and the calculation methods of these regularization loss functions are similar; i and j represent the indices of different component expert elements within the component hybrid expert sub-model, respectively; E represents the number of component expert elements in the component hybrid expert sub-model. These represent the outputs of the i-th and j-th component expert units, respectively. Indicates cosine similarity; Defect localization loss function The expression is as follows: ; in, This represents a smooth L1 loss function; Indicates the location of the center point of the defect boundary box; This indicates the length and width of the bounding box of the defect annotation.

[0033] In another aspect, the present invention provides a surface defect detection system for complex industrial components, which is configured to perform the above-described surface defect detection method for complex industrial components.

[0034] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting surface defects in complex industrial components, characterized in that, Includes the following steps: S1. Collect multiple real component defect samples and preprocess them to obtain a training dataset covering multiple component types, multiple detection surfaces and multiple defect morphologies; S2. Construct a defect detection model, including a pre-trained visual encoder, a pre-trained text encoder, a three-component hierarchical hybrid expert model, and a defect detector. The pre-trained visual encoder and pre-trained text encoder are connected in parallel, then connected in parallel with the three-component hierarchical hybrid expert model, and finally connected in series with the defect detector. S3. Train the defect detection model using the training dataset to obtain the trained defect detection model; S4. Deploy the trained defect detection model to the device and perform component defect detection to obtain the detection results.

2. The method for detecting surface defects in complex industrial components according to claim 1, characterized in that, The expression for the training dataset in S1 is as follows: ； in, Represents the training dataset. Indicates preprocessing operations; This represents multiple real component defect samples.

3. The method for detecting surface defects in complex industrial components according to claim 2, characterized in that, The three-component layered hybrid expert model includes parallel component hybrid expert sub-models, detection surface hybrid expert sub-models, and defect hybrid expert sub-models; the defect detector includes a decoder and a detection head.

4. The method for detecting surface defects in complex industrial components according to claim 3, characterized in that, The component hybrid expert sub-model includes a first router network and multiple component expert units. The first router network is connected in parallel with the multiple component expert units. Each component expert unit includes a bottleneck layer, a first normalization layer, a first multi-head attention module, a second normalization layer, and a first feedforward neural network. The output of the bottleneck layer is designed with two branches. The first branch of the bottleneck layer is connected in series with the first normalization layer and the first multi-head attention module. The second branch of the bottleneck layer is connected to the output of the first multi-head attention module through a summation operation and then split into two branches, namely the summation first branch and the summation second branch. The summation first branch is connected in series with the second normalization layer and the first feedforward neural network, respectively. The summation second branch is connected to the output of the first feedforward neural network through a summation operation. The detection surface hybrid expert sub-model includes a second router network and multiple detection surface expert units. The second router network is connected in parallel with each of the multiple detection surface expert units. Each detection surface expert unit includes a first convolutional layer, a multi-scale attention layer, a pooling layer, a second multi-head attention module, a third normalization layer, and a second feedforward neural network. The output of the first convolutional layer is designed with two branches. The first branch of the first convolutional layer is connected in series with the multi-scale attention layer, the pooling layer, and the second multi-head attention module. The second branch of the first convolutional layer is connected to the output of the second multi-head attention module through a summation operation and then split into two branches: a summation third branch and a summation fourth branch. The summation third branch is connected in series with the third normalization layer and the second feedforward neural network. The summation fourth branch is connected to the output of the second feedforward neural network through a summation operation. The defect hybrid expert sub-model includes a third router network and multiple defect expert units. The third router network is connected in parallel with each of the multiple defect expert units. Each defect expert unit includes a second convolutional layer, a spatial selection module, a fourth normalization layer, a third multi-head attention module, a fifth normalization layer, and a third feedforward neural network. The output of the second convolutional layer is designed with two branches. The first branch of the second convolutional layer is connected in series with the spatial selection module, the fourth normalization layer, and the third multi-head attention module. The second branch of the second convolutional layer is connected to the output of the third multi-head attention module through a summation operation and then split into two branches: the fifth summation branch and the sixth summation branch. The fifth summation branch is connected in series with the fifth normalization layer and the third feedforward neural network. The sixth summation branch is connected to the output of the third feedforward neural network through a summation operation.

5. The method for detecting surface defects in complex industrial components according to claim 4, characterized in that, S3 specifically includes the following steps: S31, Use the training dataset The defect image inside is input into the defect detection model. After passing through a pre-trained visual encoder and a pre-trained text encoder and undergoing multimodal feature fusion, a multimodal fusion feature x is obtained. S32. Input the multimodal fusion feature x into the three-component hierarchical hybrid expert model to obtain the global differential features. Differences and local areas The final fused features are generated through a self-attention mechanism. ; S33, Final fusion features The input is fed into the decoder to generate a multi-scale feature map, which is then fed into the detection head to obtain the center point location of the predicted defect bounding box. Length and width and categories; S34. Based on the predicted center point position of the defect bounding box Length and width The total loss function is constructed based on the categories, and the parameters of the defect detection model are adjusted according to the total loss function; S35, repeat S31 to S34 until the set iteration stopping condition is met, and the trained defect detection model is obtained.

6. The method for detecting surface defects in complex industrial components according to claim 5, characterized in that, S31 specifically includes the following steps: S311, Transfer the training dataset The defect image is input into the defect detection model, and the visual features of the defect image are extracted by a pre-trained visual encoder. Visual features Includes general and task-specific features required for defect detection in different components: visual features The expression is as follows: ； in, This represents a pre-trained visual encoder; S312. Input the defect image into a pre-trained text encoder to obtain the semantic features of the component type corresponding to the defect image. and semantic features of text prompts on the defective image's inspected surface. The expressions are as follows: ； ； in, It is a text hint about the component type. This is a text prompt for the surface to be inspected; This represents a pre-trained text encoder; S313. Introduce a cross-modal attention mechanism and utilize it to focus on visual features. semantic features semantic features Multimodal feature fusion is performed to obtain the multimodal fused feature x, expressed as follows: ； in, This indicates a cross-modal feature fusion operation.

7. The method for detecting surface defects in complex industrial components according to claim 6, characterized in that, S32 specifically includes the following steps: S321, Fusing features across multiple modalities The input is fed into the component hybrid expert sub-model within the three-component hierarchical hybrid expert model to extract the global dissimilarity features of different types of components, thus obtaining the global dissimilarity features. ; S322. The multimodal fusion feature x is input into the detection surface hybrid expert sub-model within the three-component hierarchical hybrid expert model. The detection surface hybrid expert sub-model processes all image patch features of the multimodal fusion feature x through an introduced multi-scale feature aggregation mechanism, and extracts the differential features brought about by textures and materials under various receptive fields for different detection surfaces. ; S323. Input the multimodal fusion feature x into the defect hybrid expert sub-model within the three-component hierarchical hybrid expert model. The defect hybrid expert sub-model captures the category semantic features after multimodal fusion through an image patch feature filtering mechanism. By establishing long-range dependencies between image patch features and spatial attention maps, and then extracting the most critical local regions for the defect hybrid expert sub-model based on the image patch features with the highest scores in the spatial attention map. ; S324. Introduce a self-attention mechanism for globally differential features. Differences and local areas The fusion process is performed to generate the final fusion feature. The expression is as follows: ； in, , , These are the weights obtained through adaptive learning.

8. The method for detecting surface defects in complex industrial components according to claim 7, characterized in that, The expression for the total loss function in S34 is as follows: ； in, Represents the total loss function. Represents the cross-modal contrastive loss function; This represents the expert regularization loss function; This represents the defect location loss function; This represents the balanced cross-entropy loss function; , , , These represent the weights of the corresponding loss functions.

9. A method for detecting surface defects in complex industrial components according to claim 8, characterized in that, The expert regularization loss function The expression is as follows: + + ； ； in, , , Let i and j represent the regularization loss functions of the component hybrid expert sub-model, the detection surface hybrid expert sub-model, and the defect hybrid expert sub-model, respectively; and the calculation methods of these regularization loss functions are similar; i and j represent the indices of different component expert elements within the component hybrid expert sub-model, respectively; E represents the number of component expert elements in the component hybrid expert sub-model. , These represent the outputs of the i-th and j-th component expert units, respectively. Indicates cosine similarity; Defect localization loss function The expression is as follows: ； in, This represents a smooth L1 loss function; Indicates the location of the center point of the defect boundary box; This indicates the length and width of the bounding box of the defect annotation.

10. A surface defect detection system for complex industrial components, characterized in that, The method for detecting surface defects of complex industrial components as described in any one of claims 1 to 9 is configured or executed.

Images