A profile back-spraying defect detection method and system

Abstract

The application relates to the technical field of defect detection, and particularly discloses a profile back-spraying defect detection method and system, wherein the method comprises the following steps: S1, profile image acquisition and pretreatment; S2, generation of a profile region adaptive attention mask; S3, multi-scale feature extraction based on dynamic deformable convolution; S4, context-aware defect region positioning and preliminary segmentation; S5, multi-expert adaptive defect classification and quantification; and S6, output of a detection result. Through the construction of a closed-loop detection system integrating a triple adaptive mechanism of dynamic attention focusing, adaptive feature extraction and flexible decision fusion, the core defects of poor generalization ability of an existing static model and insufficient utilization of context information are solved.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a method and system for detecting defects caused by back spraying on profiles. Background Technology

[0002] In the industrial manufacturing sector, the quality of profile surface coating directly affects the product's appearance, corrosion resistance, and service life. Traditional manual visual inspection methods are inefficient, subjective, and prone to fatigue, making it difficult to meet the high precision, efficiency, and consistency requirements of modern intelligent manufacturing. Therefore, automated defect detection technology based on machine vision has become an inevitable trend in the industry.

[0003] In the existing technology, there are various spray coating defect detection solutions based on machine vision or deep learning. However, through in-depth analysis and research, the inventors of this application have found that the current existing technical solutions still have the following common technical limitations when dealing with diverse and complex industrial profile spray coating inspection scenarios. These limitations are interrelated and restrict the generalization ability, accuracy, and robustness of the detection system in actual production environments:

[0004] 1. The static nature of models and strategies lacks adaptability to the inherent characteristics of the detected objects. Existing methods mostly rely on fixed preprocessing procedures, pre-trained static models, or preset feature extraction and classification rules. The base materials, surface treatment processes (such as anodizing, powder coating, and wood grain transfer), colors, and textures of profiles vary greatly, resulting in significant differences in their optical properties (such as reflectivity and color space distribution). A static model trained and optimized on a specific type of profile often suffers model mismatch when facing new types of profiles with vastly different characteristics, leading to a sharp drop in detection accuracy, i.e., insufficient generalization ability of the model.

[0005] 2. The "context-independent" nature of feature extraction and defect judgment. Existing methods often focus on the absolute feature values ​​of local areas when extracting features and judging defects. This approach fails to fully consider the contextual information and local structural semantics of the target area on the entire profile surface. For example, a medium-sized dark spot may be judged as a stain defect on a flat area of ​​the profile, but it may be a normal shadow or structural feature in a complex structure where multiple edges intersect; a significant color difference in a uniform color area is a defect, but it may be a normal pattern in complex texture areas such as wood grain. Judgments that are detached from the global scene and local structural context are prone to false positives and false negatives, especially on profile surfaces with high texture or complex structures.

[0006] 3. Insufficient utilization of multi-scale information and fragmented fusion methods. Although some existing technologies attempt to acquire multi-scale information through different hardware lenses, their processing flow is essentially a sequential, stage-isolated model of "first coarse localization by a macroscopic network, then fine analysis by a microscopic network within a defined area." The information interaction between macroscopic and microscopic scales is unidirectional and limited, lacking a mechanism for synchronous, deep fusion, and complementary enhancement at the feature level. For defects such as the initial gradient region of sags and the extension morphology of fine cracks, the discrimination information is simultaneously contained in the macroscopic morphological contour and microscopic texture changes. Simple two-stage sequential processing cannot achieve collaborative perception of cross-scale features, limiting the system's ability to identify complex defects such as blurred morphology, weak contrast, or cross-scale manifestations. Summary of the Invention

[0007] To address the challenge of achieving a good balance between detection accuracy, model generalization ability, and adaptability to complex scenarios in existing technologies, this invention provides a method and system for detecting profile backspray defects.

[0008] The technical solution adopted in this invention is:

[0009] A method for detecting defects in profile spraying includes the following steps:

[0010] S1. Acquire RGB and depth images of the surface of the profile to be inspected; perform illumination normalization and color deviation correction on the RGB image to obtain a preprocessed RGB image; perform filtering and noise reduction on the depth image to obtain a preprocessed depth image.

[0011] S2. Load the nominal 3D model of the profile to be inspected, project the nominal 3D model onto the image plane, and generate an initial profile region mask; calculate the pixel-level difference map between the preprocessed RGB image and the reference template image in the initial profile region mask; combine the surface gradient magnitude map calculated by the preprocessed depth image, and generate an adaptive attention mask for the profile region through an adaptive weight fusion algorithm;

[0012] S3. Using the preprocessed RGB image and the adaptive attention mask of the profile region as input, extract multi-scale feature maps through a feature pyramid network; embed a dynamically deformable convolution module in the convolutional layer of the network so that the convolution kernel can adaptively align with the surface texture and defect edges of the profile; fuse the feature maps of different scales through a context-aware fusion module to output the fused multi-scale defect feature map.

[0013] S4. Input the fused multi-scale defect feature map into the region proposal network to generate defect candidate region proposals; use the profile region adaptive attention mask to perform weighted filtering on the proposals; for the retained proposal regions, use the mask branch based on the attention mechanism to perform preliminary segmentation to generate a binary mask of the defect region.

[0014] S5. Input the image patch of the defect area and its contextual features into a multi-expert classification network. The multi-expert classification network dynamically weights and aggregates the outputs of multiple expert sub-networks according to the input features to obtain the final defect type probability distribution and defect severity score.

[0015] S6. Integrate the location, category, severity score of defects and corresponding binary masks to generate a structured profile coating defect detection report.

[0016] Preferably, in step S2, an adaptive attention mask for the profile region is generated using an adaptive weight fusion algorithm, specifically by calculating the pixel points in the mask using the following formula. weight value :

[0017] ;

[0018] in, For pixel-level difference maps at location The value at that location, For the surface gradient magnitude map at location The value at that location, For the initial profile area mask at position The value at that location, and They are respectively The mean and standard deviation, and They are respectively The mean and standard deviation, , and These are the learnable fusion coefficients. This is the Sigmoid activation function.

[0019] Preferably, in step S3, the dynamically deformable convolution module can handle any position on the output feature map. The convolution kernel operation at point , its feature vector The calculation formula is:

[0020] ;

[0021] in, The sampling grid for the standard convolution kernel. For grid The sampling offset position in the middle, For the corresponding convolution kernel weights, For the input feature map, For the current location and sampling offset position The additional offsets learned dynamically are generated by parallel convolutional layers based on the input feature map.

[0022] Preferably, in step S3, the feature pyramid network includes a bottom-up backbone network and a top-down path enhancement network. The context-aware fusion module reweights feature maps from different scales through a channel attention mechanism and then performs element-wise addition and fusion.

[0023] Preferably, in step S5, the multi-expert classification network includes a shared feature extraction backbone, The system comprises a set of parallel-connected expert subnetworks and a routing network. The routing network extracts a global description vector from the backbone output based on the shared features and calculates the allocation of a feature to each expert subnetwork. dimensional weight vector :

[0024] ;

[0025] in, This is a global description vector. , , and For routing network parameters, the final defect classification probability distribution Calculated by the following formula:

[0026] ;

[0027] in, Weight vector The middle corresponds to the first A sub-network of experts The weighted components, For the first The prediction function of an expert subnetwork. This is an image block representing the defect area.

[0028] Preferably, the expert subnetwork is a fully connected neural network with the same structure but independent parameters. Each expert subnetwork contains two hidden layers, the activation function is ReLU, the output layer is a Softmax layer, and the number of neurons is equal to the number of defect categories.

[0029] Preferably, in step S5, the defect severity is scored. The calculation formula is:

[0030] ;

[0031] in, For the compactness of the defect area, and These are the defective pixel area and the candidate region proposal area, respectively. The average brightness of the defect area in the grayscale channels of the preprocessed RGB image. and These represent the mean and standard deviation of brightness of the background region adjacent to the defect area in the grayscale channels of the preprocessed RGB image, respectively. , and These are the weighting coefficients.

[0032] A profile backspray defect detection system, comprising:

[0033] The image acquisition and preprocessing module is configured to acquire RGB images and depth images of the surface of the profile to be inspected, perform illumination normalization and color deviation correction on the RGB images to obtain preprocessed RGB images, and perform filtering and noise reduction on the depth images to obtain preprocessed depth images.

[0034] An adaptive attention mask generation module is configured to load the nominal 3D model of the profile to be inspected and project it onto the image plane to generate an initial profile region mask. It calculates the pixel-level difference map between the preprocessed RGB image and the reference template image, and combines it with the surface gradient magnitude map calculated based on the preprocessed depth image to generate an adaptive attention mask for the profile region through an adaptive weight fusion algorithm.

[0035] The dynamic multi-scale feature extraction module is configured to take the preprocessed RGB image and the adaptive attention mask of the profile region as input, extract and fuse multi-scale features through a feature pyramid network with a dynamically deformable convolution module, and output the fused multi-scale defect feature map.

[0036] The defect localization and segmentation module is configured to input the multi-scale defect feature map into the region proposal network to generate defect candidate region proposals, use the profile region adaptive attention mask to perform weighted filtering of the proposals, and perform preliminary segmentation of the retained proposal regions through the mask branch based on the attention mechanism to generate a binary mask of the defect region.

[0037] A multi-expert adaptive classification and quantization module is configured to input image patches of the defect region and their contextual features into a multi-expert classification network. The multi-expert classification network includes a shared feature extraction backbone, multiple parallel-connected expert subnetworks, and a lightweight routing network. The routing network dynamically calculates the weights of each expert subnetwork based on the global description vector output by the shared feature extraction backbone, and weights and aggregates the outputs of all expert subnetworks to obtain the final defect type probability distribution and defect severity score.

[0038] The results generation and output module is configured to integrate the location, category, severity score and corresponding binarized mask of defects, and generate and output a structured profile spraying defect detection report.

[0039] The beneficial effects of this invention are:

[0040] By constructing a closed-loop detection system integrating dynamic attention focusing, adaptive feature extraction, and flexible decision fusion—a triple adaptive mechanism—this system addresses the core shortcomings of existing static models, such as poor generalization ability and insufficient utilization of contextual information. Through dynamically generated adaptive attention masks for profile regions, the system's computational resources can autonomously focus on key surfaces prone to defects based on real-time image differences and 3D geometric gradients, abandoning the inefficient mode of uniformly processing the entire image and improving the targeting and efficiency of detection. The introduced dynamically deformable convolution module enables the feature extractor to actively adjust the sampling grid based on local image content, achieving precise alignment between convolution operations and profile texture and defect morphology, thereby extracting features with stronger discriminative power and higher geometric robustness. The multi-expert adaptive classification mechanism dynamically weights and combines the opinions of multiple specialized sub-classifiers by analyzing global contextual features through a routing network, allowing the final decision to flexibly adapt to complex combinations of different profile substrates and defect types, rather than relying on a single fixed model. These three mechanisms work together to enable the detection system to adapt to the entire process from perception to analysis to decision-making, thereby achieving high precision, high robustness and good generalization performance when dealing with diverse profiles, complex surface conditions and new defects. Attached Figure Description

[0041] Figure 1 This is a schematic flowchart of the detection method in an embodiment of the present invention;

[0042] Figure 2 This is a structural block diagram of the detection system in an embodiment of the present invention;

[0043] Figure labels: 1. Image acquisition and preprocessing module; 2. Adaptive attention mask generation module; 3. Dynamic multi-scale feature extraction module; 4. Defect localization and segmentation module; 5. Multi-expert adaptive classification and quantization module; 6. Result generation and output module. Detailed Implementation

[0044] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0045] Example 1.

[0046] A method for detecting defects in profile spraying includes the following steps:

[0047] S1. Profile Image Acquisition and Preprocessing: A high-resolution RGB industrial camera and a 3D structured light depth camera are used to acquire RGB images and depth images of the surface of the profile to be inspected, respectively. The resolution of the depth image is aligned with that of the RGB image. Illumination normalization and color deviation correction are performed on the RGB image to obtain a preprocessed RGB image. The depth image is filtered and denoised to obtain a preprocessed depth image.

[0048] S2. Generate an adaptive attention mask for the profile region: Retrieve the CAD 3D model of the profile from the MES system, load the nominal 3D model of the profile to be inspected, and project the nominal 3D model onto the image plane using pre-calibrated camera intrinsic and extrinsic parameters to generate a binarized initial profile region mask; calculate the pixel-level difference map between the preprocessed RGB image and the reference template image within the initial profile region mask; combine the surface gradient magnitude map calculated from the preprocessed depth image, and generate an adaptive attention mask for the profile region using an adaptive weight fusion algorithm;

[0049] S3. Multi-scale feature extraction based on dynamic deformable convolution: Taking the preprocessed RGB image and the adaptive attention mask of the profile region as input, multi-scale feature maps are extracted through a feature pyramid network; a dynamic deformable convolution module is embedded in the convolutional layer of the network so that the convolution kernel can adaptively align with the surface texture and defect edges of the profile; the feature maps of different scales are fused through a context-aware fusion module to output the fused multi-scale defect feature map;

[0050] S4. Context-aware defect region localization and preliminary segmentation: The fused multi-scale defect feature map is input into the region proposal network to generate defect candidate region proposals; the proposals are weighted and filtered using the profile region adaptive attention mask; the retained proposal regions are preliminarily segmented using a mask branch based on an attention mechanism to generate a binary mask of the defect region.

[0051] S5. Multi-expert adaptive defect classification and quantization: The image patch of the defect region and its contextual features are input into a multi-expert classification network. The multi-expert classification network dynamically weights and aggregates the outputs of multiple expert sub-networks according to the input features to obtain the final defect type probability distribution and defect severity score.

[0052] S6. Output detection results: Integrate the location, category, severity score of defects and corresponding binary masks to generate a structured profile spraying defect detection report.

[0053] The above method features closed-loop adaptability. In step S2, instead of using a fixed mask, an attention mask is dynamically generated by combining the nominal model (prior knowledge) with real-time image differences and depth gradients (real-time perception), enabling it to autonomously focus on key areas of the current profile. In step S3, the dynamically deformable convolutional module learns the offset field, allowing the standard square convolutional kernel to "deform" to adapt to the actual content in the image (e.g., stretching along the texture direction), achieving dynamic alignment between the feature extractor and the input content. In step S5, the multi-expert network uses a routing mechanism to dynamically select and combine the most relevant expert knowledge based on the global features of the current profile, achieving dynamic adaptation of the classifier.

[0054] The triple adaptive mechanism (attention mask, feature extraction, and classifier) ​​can self-adjust for different inputs, overcoming the limitations of static models that apply only one approach. When faced with new types of profiles, the adaptive processes in steps S2 and S3 adjust the processing focus and feature extraction method, and the routing mechanism in step S5 calls the most relevant expert knowledge, thereby maintaining high detection accuracy.

[0055] Dynamic attention masks introduce spatial weights to guide the network to focus on valuable regions; dynamically deformable convolutions enable feature extraction to fit local structures; and multi-scale feature fusion considers information from different levels. These designs allow defect judgments to be based on richer contextual information, reducing the probability of misclassifying complex textures as defects or minor defects as background.

[0056] Adaptive attention masks can suppress computation on non-critical background regions, and dynamic routing mechanisms only need to activate a portion of the expert network. Compared to using a single large network or the entire expert network, this improves computational efficiency while maintaining high performance.

[0057] Example 2.

[0058] To scientifically and adaptively fuse multiple heterogeneous information sources (color differences, geometric gradients, and 3D model priors) to generate a dynamic attention map that accurately reflects the detection importance of each region on the profile surface, and to avoid suboptimal results caused by manually setting fixed fusion rules, this embodiment, based on embodiment 1, generates an adaptive attention mask for the profile region in step S2 using an adaptive weighted fusion algorithm. Specifically, the pixel points in the mask are calculated using the following formula. weight value :

[0059] ;

[0060] in, For pixel-level difference maps at location The value at that location, For the surface gradient magnitude map at location The value at that location, For the initial profile area mask at position The value at that location, and They are respectively The mean and standard deviation, and They are respectively The mean and standard deviation, , and These are the learnable fusion coefficients, with initial values ​​of 0.4, 0.4, and 0.2, respectively. This is the Sigmoid activation function.

[0061] This formula is an adaptive weighted fusion processor that normalizes color differences. Geometric gradient and deterministic model priors Linear combination is used; areas with large color differences may indicate contamination or color difference defects; areas with large geometric gradients (such as edges and welds) are usually high-incidence areas of defects; model priors ensure that the main body area of ​​the profile is considered. The key point lies in the coefficients. , and It is learnable; during training, the network automatically learns the optimal fusion ratio for the current task through a large amount of data. For example, for highly reflective profiles, the network may learn to assign color differences. Lower weights are used because reflections can cause color instability; for structurally complex profiles, geometric gradients may be imparted. Higher weight.

[0062] This step avoids the blindness and rigidity of manually setting fusion weights based on experience, enabling the generated attention mask to dynamically adjust the information sources it depends on according to different profile characteristics and imaging conditions, thereby generating more accurate and robust focus area indications.

[0063] Example 3.

[0064] To address the issue that standard convolutional kernels have fixed shapes and rigid receptive fields, making them unable to effectively adapt to diverse texture directions, irregular shapes of defects, and local image deformations on profile surfaces, and thus improve the ability of feature extraction to model target geometric transformations, this embodiment, based on embodiment 2, in step S3, allows the dynamically deformable convolutional module to handle any position on the output feature map. A convolution kernel operation at a certain point, whose feature vector The calculation formula is:

[0065] ;

[0066] in, The sampling grid for the standard convolution kernel. For grid The sampling offset position in the middle, For the corresponding convolution kernel weights, For the input feature map, For the current location and sampling offset position The additional offsets learned dynamically are generated by a parallel, lightweight convolutional layer based on the input feature map.

[0067] Traditional convolution at position Sampled input features At that time, the sampling point location is fixed. The dynamic deformable convolution method in this paper adds an adaptive offset to this. This offset is not fixed, but rather predicted in real-time by a small network ϕ based on the current input feature map. Therefore, the sampling grid of the convolutional kernel can "actively deform" according to the image content. For example, when the region corresponding to the feature map is a diagonal scratch, the network learns... This will cause the sampling points to be arranged along the direction of the scratch, thus allowing the convolution kernel to better capture edge features in that direction.

[0068] This step enables the network to adaptively align the sampling points of the convolutional kernels with the actual shape and local texture direction of the defect, greatly enhancing its feature representation capabilities. For irregular defects such as linear sags and irregular bubbles, the extracted features are more discriminative than those from standard convolution. Compared to increasing the receptive field by stacking more layers or larger convolutional kernels, dynamically deformable convolutions achieve a more flexible receptive field by fine-tuning the sampling positions, typically requiring only a small increase in parameters (for offset prediction networks), thus more efficiently improving model capabilities.

[0069] Example 4.

[0070] To address the issue that existing defect detection methods typically employ simple direct addition or channel splicing when performing multi-scale feature fusion, failing to distinguish the differences in the contribution of each channel in the feature maps at different scales to the final detection task, this embodiment, based on embodiment 3, includes a feature pyramid network comprising a bottom-up backbone network and a top-down path enhancement network in step S3. The context-aware fusion module reweights the feature maps from different scales through a channel attention mechanism and then performs element-wise addition fusion.

[0071] Through a channel attention mechanism, the network can adaptively learn and emphasize feature channels most relevant to defects in profile coating. For example, when detecting "bubble" defects, the network may automatically enhance feature channels sensitive to circular contours and high-gloss reflections; when detecting "sags," it may enhance feature channels sensitive to vertical linear textures. This dynamic channel selection produces more discriminative feature representations than fixed-weight fusion.

[0072] This mechanism can suppress interference from noise channels caused by complex background textures of the profiles and uneven lighting. It allows the network to focus on the defect signal itself, rather than being misled by irrelevant, changing environmental factors, thereby improving the stability of the system in different production line environments.

[0073] Before the final splicing and fusion, features at each scale are purified to ensure that the information input to subsequent detection heads (RPN, classification network) is optimized and filtered, providing a cleaner and more effective feature base for accurate localization and classification, and improving the overall detection accuracy from the source.

[0074] Example 5.

[0075] To design an efficient and effective mechanism that can automatically and reasonably allocate trust levels to different specialized sub-models (experts) based on the global characteristics of the current detection sample, and achieve dynamic combination of classification decisions to cope with the diversity of profiles and defect types, this embodiment, based on embodiment 4, adds a shared feature extraction backbone in step S5 to the multi-expert classification network. The system comprises a set of parallel-connected expert subnetworks and a lightweight routing network. The routing network extracts a global description vector from the backbone output based on the shared features and calculates the allocation of a vector to each expert subnetwork. dimensional weight vector :

[0076] ;

[0077] in, This is a global description vector. , , and For routing network parameters, the final defect classification probability distribution Calculated by the following formula:

[0078] ;

[0079] in, Weight vector The middle corresponds to the first A sub-network of experts The weighted components, For the first The prediction function of an expert subnetwork. This is an image block representing the defect area.

[0080] This is a soft weight allocation and decision fusion mechanism. The routing network acts as a scheduler; it does not directly perform classification but instead analyzes the global feature vector of the input samples. (This vector encodes contextual information such as profile type and approximate defect morphology) and outputs a weight distribution. Each expert Independently cut out defective areas Fine-grained analysis is performed. The final decision is a weighted average of all these expert opinions, with the weights dynamically determined by the routing network based on the global context. For example, when global features suggest that the current profile is "anodized aluminum" and the defect area is "dot-like," the routing network will assign higher weights to experts who are good at handling "aluminum pinholes" and "aluminum spots."

[0081] This step avoids the problems of internal knowledge conflicts or insufficient capacity caused by training a single large and comprehensive classifier. Each expert can achieve peak performance in their specialized sub-domain, while overall classification accuracy is guaranteed through weighted fusion. During inference, although all expert networks exist physically, through the soft selection of the routing network, for each sample, the decision is actually led by a few high-weighted experts, achieving an effect similar to conditional computation. This maintains a powerful model capacity while controlling computational overhead. When it is necessary to detect new materials or new defects, new expert sub-networks can be simply added and fine-tuned without redesigning or training the entire backbone network, resulting in strong system scalability.

[0082] Example 6.

[0083] In the multi-expert classification network framework described in Example 5, the implementation architecture of the expert sub-networks needs to be specifically defined to address the following two issues: First, how to design the expert network to ensure it has sufficient learning capacity to capture complex feature patterns in its specialized domain; second, how to maintain structural consistency of the expert network to achieve efficient and stable parallel computing and dynamic routing, and avoid training instability or ensemble bias caused by differences in network structure. Based on Example 5, this example uses fully connected neural networks with identical structures but independent parameters. Each expert sub-network contains two hidden layers with ReLU activation function, a Softmax output layer, and the number of neurons equal to the number of defect categories.

[0084] All expert subnetworks are restricted to using fully connected neural networks with the same structure. This means they have the same number of hidden layers, the same number of neurons per layer, and the same activation function (such as ReLU). This isomorphic design ensures that each expert processes the input vector. The initial capacity and computation path are the same, and the weights assigned by the routing network are the same. It directly reflects the relevance of an expert's "knowledge," rather than its structural advantages; batch processing can be used to compute the forward propagation of all experts in parallel, greatly improving computational efficiency; when adding a new expert, simply copy the same network structure and initialize it randomly for seamless integration into the existing system. A fully connected network with two hidden layers is a classic and powerful general-purpose function approximator structure, sufficient to learn from global description vectors. This involves a complex nonlinear mapping to the probability of specific defect categories. The ReLU activation function provides nonlinearity and alleviates the vanishing gradient problem. The Softmax activation function in the output layer transforms each expert's raw output into a probability distribution, intuitively representing the expert's confidence that the input sample belongs to each defect category.

[0085] Although structurally identical, the parameters of each expert subnetwork are trained and updated independently. During training, guided by the routing network and through backpropagation of the loss function, different experts naturally differentiate, learning to focus on different aspects or patterns of the input features. For example, one expert might focus on learning to distinguish between "pinholes" and "bubbles" (both may be circular but have different optical properties), while another expert might excel at distinguishing between "drips" and "scratches" (different forms of linear defects). This parameter diversity within structural consistency is the foundation for the effective operation of the "multi-expert" mechanism.

[0086] Homogeneous expert networks eliminate the problem of uneven optimization difficulty caused by structural differences, enabling all experts to perform collaborative optimization through gradient descent from a relatively fair starting point, simplifying the training process and improving the stability of model convergence.

[0087] The final classification result is a weighted vote by multiple experts with similar structures but differing viewpoints. This design makes the decision-making process interpretable to some extent: the routing weights can be analyzed. Understanding which type of experts a system relies on for decision-making, and analyzing the differences in the outputs of different experts on the same sample, can help to understand the ambiguity or complexity of the sample classification. This facilitates system debugging and trust building.

[0088] Example 7.

[0089] To quantify the detected defects and provide an objective and comprehensive severity score to guide maintenance prioritization and quality grading in the production process, and to avoid relying solely on classification results without a quantitative description of the defect's impact, this embodiment, based on embodiment 6, includes a defect severity score in step S5. The calculation formula is:

[0090] ;

[0091] in, For the compactness of the defect area, and These are the defective pixel area and the candidate region proposal area, respectively. The average brightness of the defect area in the grayscale channels of the preprocessed RGB image. and These are the mean and standard deviation of brightness of the background region adjacent to the defect region in the grayscale channel of the preprocessed RGB image, respectively. The background region can be understood as a ring-shaped background region adjacent to the defect region (for example, the new region obtained by expanding the defect mask outward by several pixels, and then subtracting the original defect region). , and These are the weighting coefficients.

[0092] The scoring formula quantifies the severity of defects from three dimensions: shape, size, and visual salience.

[0093] Morphological consistency item The more irregular the shape (the lower the compactness), the larger this value, which usually indicates serious process problems such as sagging and cracking.

[0094] Area percentage item The larger the proportion of a defect in its local area, the more significant its impact.

[0095] Optical contrast ratio The greater the brightness difference between the defect and the background (numerator), and the more uniform the background (smaller the denominator), the more obvious the defect and the greater its impact on visual quality. Normalization using the background standard deviation eliminates the influence of the complexity of the background texture in different areas.

[0096] Compared to simple binary classification (defective / no defect) or single area measurement, this comprehensive scoring system can more finely differentiate the severity of defects. For example, a large but regularly shaped defect with low contrast (such as a large area of ​​slight color difference) may receive a different score than a small but irregularly shaped defect with high contrast (such as a sharp spot of unpainted material), providing more accurate feedback for process improvement. The weighting coefficients can be adjusted according to different customers' quality standards. For example, in applications with extremely high requirements for appearance, the weighting can be increased. The weight of (optical contrast). This score can be used as an auxiliary output of the multi-expert network in step S5, or as additional input information for the routing network to calculate expert weights, making the decision-making of the entire system more comprehensive.

[0097] The specific structure and data flow of the multiple neural network modules involved in the above embodiments are as follows:

[0098] The structure of the Feature Pyramid Network (FPN) and the context-aware fusion module is as follows:

[0099] Overall architecture: A feedforward convolutional neural network is used, specifically a top-down architecture with lateral connections. The backbone network uses ResNet-50 pre-trained on ImageNet.

[0100] Input layer: The input is a tensor of size H×W×4, where H=3000, W=4096, and the four channels are the R, G, and B channels of the preprocessed RGB image and the adaptive attention mask for the profile region. The preprocessing method is to normalize the pixel values ​​to the [0,1] interval. The number of neurons in the input layer is H×W×4, determined by the size of the input image.

[0101] Hidden Layers: The backbone network ResNet-50 provides four feature maps at different scales (C2, C3, C4, C5). The top-down path of the FPN adjusts the channel count of the upper-layer features through a 1×1 convolutional layer, and then performs element-wise addition with the bottom-up feature map from the backbone through a 2x nearest neighbor upsampling. P5 is obtained by convolving C5 with a 1×1 convolution; P4 is obtained by adding the features of P5 upsampled and C4 convolved with a 1×1 convolution; P3 and P2 are obtained similarly.

[0102] The context-aware fusion module: For each scale feature map {P2, P3, P4, P5} output by the FPN, it first reweights them through a channel attention module (such as the Squeeze-and-Excitation block in SENet) to enhance the feature responses of important channels. Then, the reweighted feature maps P2 to P5 are uniformly upsampled to the scale of P2, channel concatenation is performed, and finally, a 3×3 convolutional layer is used for fusion to output the final multi-scale defect feature map F. This feature map F integrates multi-level contextual information from fine-grained texture to coarse-grained semantics.

[0103] Output layer: The output of this module is the feature map F, which is used for subsequent defect localization and segmentation. It does not involve a specific classification activation function.

[0104] The structure of the dynamically deformable convolutional module is as follows:

[0105] Overall architecture: This module is embedded as a plug-in component into some convolutional layers of the feature extraction network (such as ResNet's stage3 and stage4) to replace the standard 3×3 convolution.

[0106] Input and output: The input is the feature map output from the previous layer, and the output is the new feature map after deformable convolution. The input / output size remains unchanged.

[0107] Internal structure: For each convolutional kernel operation, two branches are run in parallel:

[0108] a) Main branch: Performs standard convolution operations, but its sampling position is variable.

[0109] b) Offset Prediction Branch: A lightweight subnetwork (e.g., consisting of two consecutive 3×3 convolutional layers, with the last convolutional layer having 2N output channels, where N = kernel size; for example, N = 9 for a 3×3 kernel) learns to output an offset field using the feature map from the previous layer as input. The size of this offset field is consistent with the spatial size of the input feature map, and each location contains 2N values ​​(corresponding to the x and y direction offsets of N sampling points).

[0110] Data Flow: The offset prediction branch dynamically generates offsets based on the content of the input feature map. The convolutional kernels of the main branch are positioned at irregular locations on the feature map based on these offsets. Sampling and weighted summation are performed to obtain the output features. This allows the convolution kernel to adaptively focus on the edges or texture directions of defects.

[0111] Training: The weights of this module (including the parameters of the main branch convolution kernel and the offset prediction branch) are jointly trained end-to-end. The loss function is backpropagated through the main branch, which also trains the offset prediction branch to predict sampling locations that are beneficial to the task (defect detection).

[0112] The structure of a multi-expert classification network is as follows:

[0113] Overall architecture: It is a feedforward neural network based on dynamic routing.

[0114] Input layer: The input consists of two parts:

[0115] 1) The defect area image patch cropped from the original image is scaled to 224×224×3;

[0116] 2) The context feature vectors of the corresponding regions extracted from feature map F are subjected to the same normalization process for the image blocks.

[0117] Shared feature extraction backbone: a lightweight convolutional neural network (such as MobileNetV2), taking a 224×224×3 image patch as input and outputting a 1024-dimensional global description vector. .

[0118] Lightweight routing network: a two-layer fully connected neural network.

[0119] First layer: Input dimension d=1024 (i.e., vector) The output dimension is h=128, using the ReLU activation function. The parameters are... (128×1024), (128×1).

[0120] Second layer: Input dimension 128, output dimension N (number of experts, N=5 in this example), no activation function. Parameters are... (5×128), (5×1).

[0121] Output layer: Apply the Softmax function to the output of the second layer to obtain the weight vector. (Dimension 5×1), which sums to 1, representing the weights assigned to the 5 experts.

[0122] Expert subnetwork: 5 fully connected neural networks with identical structures and independent parameters.

[0123] The input to each expert network is a global description vector output from the shared backbone. (1024 dimensions).

[0124] First hidden layer: 1024 neurons, activation function is ReLU.

[0125] The second hidden layer has 512 neurons and uses ReLU as the activation function.

[0126] Output layer: C neurons (C is the number of defect categories, such as 6 categories: pinhole, drip, bubble, particle, orange peel, normal), activation function is Softmax, output a category probability distribution.

[0127] Data flow and training:

[0128] Forward propagation: Input image patches are processed through a shared backbone to obtain vectors. . On the one hand, the input routing network obtains the weights. On the other hand, all five expert subnetworks are input in parallel, resulting in five prediction probability distributions {E1, E2, ..., E5}. The final classification probability distribution... .

[0129] Training process:

[0130] Loss function: The error between the final prediction P and the true label is calculated using the cross-entropy loss function.

[0131] Optimizer: The Adam optimizer is used, with an initial learning rate of 0.0001. The learning rate is reduced to half its original value every 30 training epochs.

[0132] Training strategy: First, fix the parameters of the routing network and expert networks, and train the shared feature extraction backbone separately for 100 epochs. Then, unfreeze all parameters and perform end-to-end joint training for 150 epochs. During joint training, the loss gradient will simultaneously update the parameters of the shared backbone, the routing network, and each expert network, enabling the routing network to learn to allocate weights according to the input features, and each expert network to specialize in the feature regions it is good at.

[0133] Example 8.

[0134] A profile backspray defect detection system, comprising:

[0135] Image acquisition and preprocessing module 1 is configured to acquire RGB images and depth images of the surface of the profile to be inspected, perform illumination normalization and color deviation correction on the RGB images to obtain preprocessed RGB images, and perform filtering and noise reduction on the depth images to obtain preprocessed depth images.

[0136] The adaptive attention mask generation module 2 is configured to load the nominal three-dimensional model of the profile to be inspected and project it onto the image plane to generate an initial profile region mask. It calculates the pixel-level difference map between the preprocessed RGB image and the reference template image, and combines it with the surface gradient magnitude map calculated based on the preprocessed depth image to generate an adaptive attention mask for the profile region through an adaptive weight fusion algorithm.

[0137] The dynamic multi-scale feature extraction module 3 is configured to take the preprocessed RGB image and the adaptive attention mask of the profile region as input, extract and fuse multi-scale features through a feature pyramid network with a dynamically deformable convolution module, and output the fused multi-scale defect feature map.

[0138] The defect localization and segmentation module 4 is configured to input the multi-scale defect feature map into the region proposal network to generate defect candidate region proposals, use the profile region adaptive attention mask to perform weighted filtering of the proposals, and perform preliminary segmentation of the retained proposal regions through the mask branch based on the attention mechanism to generate a binary mask of the defect region.

[0139] The multi-expert adaptive classification and quantization module 5 is configured to input the image patch of the defect region and its contextual features into the multi-expert classification network. The multi-expert classification network includes a shared feature extraction backbone, multiple parallel-connected expert sub-networks, and a lightweight routing network. The routing network dynamically calculates the weights of each expert sub-network based on the global description vector output by the shared feature extraction backbone, and weights and aggregates the outputs of all expert sub-networks to obtain the final defect type probability distribution and defect severity score.

[0140] Result generation and output module 6 is configured to integrate the location, category, severity score and corresponding binarized mask of defects, and generate and output a structured profile spraying defect detection report.

[0141] The corresponding steps in the implementation method of this system are not repeated here.

[0142] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for detecting backspray defects in profiles, characterized in that, Includes the following steps: S1. Acquire RGB and depth images of the surface of the profile to be inspected; perform illumination normalization and color deviation correction on the RGB image to obtain a preprocessed RGB image; perform filtering and noise reduction on the depth image to obtain a preprocessed depth image. S2. Load the nominal three-dimensional model of the profile to be inspected, project the nominal three-dimensional model onto the image plane, and generate an initial profile area mask; Calculate the pixel-level difference map between the preprocessed RGB image and the reference template image within the initial profile region mask; Based on the surface gradient magnitude map calculated from the preprocessed depth image, an adaptive attention mask for the profile region is generated using an adaptive weight fusion algorithm; S3. Using the preprocessed RGB image and the adaptive attention mask of the profile region as input, extract multi-scale feature maps through a feature pyramid network; embed a dynamically deformable convolution module in the convolutional layer of the network so that the convolution kernel can adaptively align with the surface texture and defect edges of the profile; fuse the feature maps of different scales through a context-aware fusion module to output the fused multi-scale defect feature map. S4. Input the fused multi-scale defect feature map into the region proposal network to generate defect candidate region proposals; use the profile region adaptive attention mask to perform weighted filtering on the proposals. For the retained proposal regions, an attention-based mask branch is used for preliminary segmentation to generate a binary mask for the defective regions. S5. Input the image patch of the defect area and its contextual features into a multi-expert classification network. The multi-expert classification network dynamically weights and aggregates the outputs of multiple expert sub-networks according to the input features to obtain the final defect type probability distribution and defect severity score. S6. Integrate the location, category, severity score of defects and corresponding binary masks to generate a structured profile coating defect detection report.

2. The method for detecting profile backspray defects according to claim 1, characterized in that, In step S2, an adaptive attention mask for the profile region is generated using an adaptive weight fusion algorithm. Specifically, the pixel points in the mask are calculated using the following formula. weight value : ； in, For pixel-level difference maps at location The value at that location, For the surface gradient magnitude map at location The value at that location, For the initial profile area mask at position The value at that location, and They are respectively The mean and standard deviation, and They are respectively The mean and standard deviation, , and These are the learnable fusion coefficients. This is the Sigmoid activation function.

3. The method for detecting profile backspray defects according to claim 2, characterized in that, In step S3, the dynamically deformable convolution module can handle any position on the output feature map. Convolution kernel operation at the location, Its eigenvectors The calculation formula is: ； in, The sampling grid for the standard convolution kernel. For grid The sampling offset position in the middle, For the corresponding convolution kernel weights, For the input feature map, For the current location and sampling offset position The additional offsets learned dynamically are generated by parallel convolutional layers based on the input feature map.

4. The method for detecting profile backspray defects according to claim 3, characterized in that, In step S3, the feature pyramid network includes a bottom-up backbone network and a top-down path enhancement network. The context-aware fusion module reweights feature maps from different scales through a channel attention mechanism and then performs element-wise addition and fusion.

5. The method for detecting profile backspray defects according to claim 4, characterized in that, In step S5, the multi-expert classification network includes a shared feature extraction backbone, The system comprises a set of parallel-connected expert subnetworks and a routing network. The routing network extracts a global description vector from the backbone output based on the shared features and calculates the allocation of a feature to each expert subnetwork. dimensional weight vector : ； in, This is a global description vector. , , and For routing network parameters, the final defect classification probability distribution Calculated by the following formula: ； in, Weight vector The middle corresponds to the first A sub-network of experts The weighted components, For the first The prediction function of an expert subnetwork. This is an image block representing the defect area.

6. The method for detecting profile backspray defects according to claim 5, characterized in that, The expert subnetworks are fully connected neural networks with identical structures but independent parameters. Each expert subnetwork contains two hidden layers with ReLU activation function and a Softmax output layer. The number of neurons is equal to the number of defect categories.

7. The method for detecting profile backspray defects according to claim 6, characterized in that, In step S5, the defect severity is scored. The calculation formula is: ； in, For the compactness of the defect area, and These are the defective pixel area and the candidate region proposal area, respectively. The average brightness of the defect area in the grayscale channels of the preprocessed RGB image. and These represent the mean and standard deviation of brightness of the background region adjacent to the defect area in the grayscale channels of the preprocessed RGB image, respectively. , and These are the weighting coefficients.

8. A profile backspray defect detection system, characterized in that, include: The image acquisition and preprocessing module is configured to acquire RGB images and depth images of the surface of the profile to be inspected, perform illumination normalization and color deviation correction on the RGB images to obtain preprocessed RGB images, and perform filtering and noise reduction on the depth images to obtain preprocessed depth images. An adaptive attention mask generation module is configured to load the nominal 3D model of the profile to be inspected and project it onto the image plane to generate an initial profile region mask. It calculates the pixel-level difference map between the preprocessed RGB image and the reference template image, and combines it with the surface gradient magnitude map calculated based on the preprocessed depth image to generate an adaptive attention mask for the profile region through an adaptive weight fusion algorithm. The dynamic multi-scale feature extraction module is configured to take the preprocessed RGB image and the adaptive attention mask of the profile region as input, extract and fuse multi-scale features through a feature pyramid network with a dynamically deformable convolution module, and output the fused multi-scale defect feature map. The defect localization and segmentation module is configured to input the multi-scale defect feature map into the region proposal network to generate defect candidate region proposals, use the profile region adaptive attention mask to perform weighted filtering of the proposals, and perform preliminary segmentation of the retained proposal regions through the mask branch based on the attention mechanism to generate a binary mask of the defect region. A multi-expert adaptive classification and quantization module is configured to input image patches of the defective region and their contextual features into a multi-expert classification network. The multi-expert classification network includes a shared feature extraction backbone, multiple parallel-connected expert subnetworks, and a lightweight routing network. The routing network dynamically calculates the weights of each expert subnetwork based on the global description vector output by the shared feature extraction backbone, and aggregates the outputs of all expert subnetworks in a weighted manner to obtain the final defect type probability distribution and defect severity score. The results generation and output module is configured to integrate the location, category, severity score and corresponding binarized mask of defects, and generate and output a structured profile spraying defect detection report.

Images