Part surface defect detection method based on image processing

Abstract

The invention relates to the technical field of machine vision image processing, and discloses a part surface defect detection method based on image processing. The method comprises the following steps: acquiring a part surface image through image acquisition, extracting defect sensitive characteristics by using a multi-scale Gaussian difference operator, and filtering through a threshold value to obtain a candidate defect point set; and inputting the candidate point set into a serialized network containing an attention mechanism and a memory unit to generate an enhanced feature sequence. And carrying out space alignment on the sequence and an original topological association graph, and generating an enhanced feature graph containing node attributes and edge weights through graph convolution. And on the basis, performing region growing and merging operation to generate a connected candidate defect region. And extracting geometric and texture descriptors of each region to form a feature vector, matching the feature vector with a standard vector in a defect feature library so as to identify the defect category and position, and finally outputting a defect detection map. According to the method, the detection rate and the segmentation precision of the complex distribution defects are improved.

Description

Technical Field

[0001] This invention relates to the field of machine vision image processing technology, specifically to a method for detecting surface defects in parts based on image processing. Background Technology

[0002] In automatic surface defect detection of parts based on image processing, conventional methods typically rely on feature extraction and pattern classification of images. These methods first extract local features of suspected defect areas through edge detection, texture analysis, or deep learning models, and then use classifiers to discriminate these isolated features to identify defects. Real defects on part surfaces often consist of a series of spatially discrete but intrinsically related anomalies or tiny regions. Existing technologies treat these feature points as independent samples, ignoring the potential spatial context and sequential relationships between them. This approach is prone to missed detections or false positives for sparsely distributed, complex-shaped defects or defects similar to the background texture, making it difficult to accurately delineate the overall outline and internal structure of the defects.

[0003] Most detection methods directly perform region segmentation or classification after feature extraction, lacking explicit modeling and utilization of the inherent spatial topological relationships between defect points. Although convolutional neural networks can capture local neighborhood information, their fixed gridded convolutional kernels are difficult to flexibly adapt to the irregular spatial distribution relationships of defect point groups. After generating candidate regions, traditional methods fail to place features in a graph structure that reflects the original connectivity relationships of pixels or superpixels for propagation and optimization. This results in the feature representation failing to fully integrate geometric structural information, limiting the ability to accurately locate defect boundaries and determine the merging of adjacent defect regions. Summary of the Invention

[0004] The purpose of this invention is to provide a method for detecting surface defects in parts based on image processing, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides a method for detecting surface defects in parts based on image processing, the method comprising:

[0006] Standardized part surface images are obtained through an image acquisition system, and defect-sensitive features are extracted by combining multi-scale Gaussian difference operators.

[0007] By using a preset feature filtering threshold, the defect-sensitive features are filtered, and the features that meet the feature filtering threshold are mapped to a set of candidate defect points.

[0008] A serialized feature processing network containing an attention mechanism and a memory unit is constructed. The candidate defect point set is input into the serialized feature processing network to generate a reinforced feature sequence with temporal correlation.

[0009] The enhanced feature sequence is spatially aligned with the original topological association graph, and the features are propagated through graph convolution operations to generate an enhanced feature graph containing node attributes and edge weights.

[0010] The enhanced feature map is subjected to region growing and merging operations, and several connected candidate defect regions are generated based on node attribute similarity and edge connection strength.

[0011] Extract the geometric morphology descriptor and texture statistical descriptor of each candidate defect region, and combine them to form a region feature description vector;

[0012] The region feature description vector is matched and measured with the standard defect feature vector pre-stored in the defect feature library, and the defect category of the candidate defect region is identified according to the matching result.

[0013] Output a surface defect detection map of the part with defect category and location information.

[0014] Preferably, obtaining a standardized part surface image through the image acquisition system includes:

[0015] Under controlled lighting conditions, a sequence of original multi-view images of the surface of a part is acquired using a multi-angle linear scanning camera array;

[0016] Adaptive illumination compensation and shadow removal processing are performed on each image in the original multi-view image sequence;

[0017] The processed multi-view image sequence is input into the dense 3D reconstruction engine to generate a 3D model of the part surface with 3D point cloud coordinates and surface normal information.

[0018] The three-dimensional model of the part surface is parametrically unfolded in two dimensions to obtain a part surface image that eliminates perspective distortion and scale differences.

[0019] Preferably, the step of extracting defect-sensitive features by combining multi-scale Gaussian difference operators includes:

[0020] The surface image of the part is decomposed into a set of visual primitives with spatial coordinates, directional components, and texture attributes;

[0021] Based on the set of visual primitives, each visual primitive is encoded using a preset primitive dictionary to generate a high-dimensional sparse encoding vector, and a topological association graph of the part surface is constructed from the high-dimensional sparse encoding vector.

[0022] A multi-scale Gaussian difference operator is applied to the topological correlation graph to generate a multi-scale feature map pyramid, and defect-sensitive features are extracted from the multi-scale feature map pyramid.

[0023] The step of decomposing the surface image of the part into a set of visual primitives with spatial coordinates, directional components, and texture attributes includes:

[0024] Local contrast normalization and Gaussian difference filtering are performed on the part surface image to obtain the preprocessed enhanced image;

[0025] On the enhanced image, a series of connected regions with stable boundaries are extracted using the maximum stable extremum region detection method;

[0026] For each connected region, its spatial centroid coordinates, principal axis direction, region area, and texture attributes based on the gray-level co-occurrence matrix are calculated to form the visual primitive.

[0027] All visual primitives are sorted according to their spatial centroid coordinates to form the visual primitive set.

[0028] Preferably, the step of encoding each visual primitive based on the set of visual primitives using a preset primitive dictionary to generate a high-dimensional sparse coding vector includes:

[0029] Invoke the pre-trained primitive dictionary, which contains multiple visual primitive templates;

[0030] The texture attributes and orientation components of each visual primitive in the visual primitive set are compared with the similarity of each visual primitive template in the primitive dictionary.

[0031] For each visual primitive, the indices of the top K primitive templates with the highest similarity metric values ​​are selected, and in a zero vector with the same length as the primitive dictionary capacity, the value at the corresponding index position is set to the corresponding similarity metric value, thereby generating the high-dimensional sparse coding vector.

[0032] Preferably, the construction of the topological correlation graph of the part surface from the high-dimensional sparse coding vector includes:

[0033] Each visual primitive is treated as a node in the topological association graph, and the corresponding high-dimensional sparse coding vector is used as the initial attribute of the node.

[0034] Calculate the spatial distance and texture difference between any two visual primitive nodes. When the spatial distance is less than the spatial neighborhood radius threshold and the texture difference is less than the texture difference threshold, establish an edge between the corresponding two nodes.

[0035] The edges are assigned weights that are inversely proportional to the weighted sum of the spatial distance and texture difference, thereby constructing the topological association graph containing nodes and weighted edges.

[0036] Preferably, the step of applying a multi-scale Gaussian difference operator to the topological correlation graph to generate a multi-scale feature map pyramid includes:

[0037] The topological association graph is represented as an adjacency matrix and a node feature matrix;

[0038] Gaussian smoothing is applied to the adjacency matrix under different scale parameters to obtain smoothed adjacency matrices at different scales;

[0039] Graph convolution is performed between the smoothed adjacency matrix and the node feature matrix at each scale to obtain the node feature response map at the specified scale.

[0040] Stack the node feature response maps at all scales to form the multi-scale feature map pyramid.

[0041] Preferably, the step of fusing and extracting defect-sensitive features from the multi-scale feature map pyramid includes:

[0042] For the feature response values ​​of the same spatial location in the multi-scale feature map pyramid at different scales, cross-scale max pooling is performed to generate a preliminary fused feature map.

[0043] The orientation gradient histogram is calculated on the preliminary fused feature map to obtain the orientation gradient statistical features at each location;

[0044] The directional gradient statistical features are concatenated with the feature values ​​at corresponding positions in the preliminary fusion feature map to form the defect-sensitive features.

[0045] Preferably, the construction of a serialized feature processing network including an attention mechanism and memory units, and the input of the candidate defect set into the serialized feature processing network to generate a reinforcement feature sequence with temporal correlation, includes:

[0046] Each point in the candidate defect point set is arranged in scan line order according to its spatial location to form an initial feature sequence;

[0047] The initial feature sequence is sequentially input into the long short-term memory unit in the serialization feature processing network to learn the long-range dependencies of features in the sequence.

[0048] An attention mechanism is introduced at the output of the long short-term memory unit to dynamically calculate the attention weight of each position feature in the sequence;

[0049] The weighted feature sequence is residually concatenated with the original feature sequence to output the enhanced feature sequence.

[0050] Preferably, the step of performing region growing and merging operations on the enhanced feature map, generating several connected candidate defect regions based on node attribute similarity and edge connection strength, includes:

[0051] In the enhanced feature map, an unlabeled node is randomly selected as a seed point;

[0052] Using the seed point as the core, traverse all adjacent nodes directly connected to it by edges. If the attribute similarity between the adjacent node and the seed point is greater than the attribute similarity threshold, and the strength of the connecting edge is greater than the edge strength threshold, then merge the adjacent node and the seed point into the same region.

[0053] Recursively perform traversal and judgment operations on the nodes of the newly merged region until no new node meets the merging condition, thus completing the growth of a candidate defect region;

[0054] The region growth process, starting from the selected seed point, is repeated until all nodes in the enhanced feature map are marked, ultimately generating several connected candidate defect regions.

[0055] Preferably, the step of performing a matching metric calculation between the region feature description vector and the standard defect feature vector pre-stored in the defect feature library, and identifying the defect category of the candidate defect region based on the matching result, includes:

[0056] Calculate the cosine similarity between the region feature description vector and each standard defect feature vector in the defect feature library;

[0057] Select the standard defect feature vector with the highest cosine similarity and determine whether its corresponding similarity value exceeds the predetermined category determination threshold;

[0058] If the number exceeds the limit, the defect category corresponding to the standard defect feature vector is assigned to the candidate defect region from which the regional feature description vector originates.

[0059] If the limit is not exceeded, the defect category of the candidate defect region will be marked as an unknown category.

[0060] Compared with the prior art, the beneficial effects of the present invention are:

[0061] A sequential feature processing network incorporating an attention mechanism and memory units is constructed, taking a discrete set of candidate defect points as sequence input. The attention mechanism dynamically evaluates the importance of each node in the sequence, focusing on more representative defect features; the memory units maintain and transmit historical state information during sequence processing. This enables the network to learn and retain temporal dependencies and long-range contextual associations between defect points, transforming isolated point features into enhanced feature sequences with coherent semantic information. This improves the overall perception of defects with complex distributions such as chain-like and scattering patterns, and reduces the false negative rate caused by weak isolated point features.

[0062] The enhanced feature sequence is spatially aligned with the original topological association graph reflecting the underlying pixel / superpixel connectivity of the image, and graph convolution is then performed on this basis. Through edge connections in the graph structure, the enhanced features of each node can propagate and iteratively aggregate to its topological neighbors, while edge weights are updated based on feature similarity. This achieves collaborative optimization of node features and graph structure, ultimately generating an enhanced feature graph that simultaneously contains node attributes and edge connection strengths. This deeply integrates spatial topological constraints into the feature representation, enabling a more accurate reflection of the true geometry and internal connectivity of defect regions. This provides a basis for subsequent region growing and merging, improving the boundary accuracy and region integrity of defect segmentation. Attached Figure Description

[0063] Figure 1 This is a schematic diagram illustrating the working principle of the image processing-based part surface defect detection method described in this invention.

[0064] Figure 2 Flowchart for acquiring standardized part surface images;

[0065] Figure 3 A flowchart for encoding visual primitives into high-dimensional sparse vectors;

[0066] Figure 4 The curves show the evolution of the LSTM hidden state with the scan line sequence during the serialization feature processing stage.

[0067] Figure 5 This is a graph showing the correlation between the accuracy of multi-scale feature extraction and the response intensity. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] Please see Figure 1 This invention provides a method for detecting surface defects in parts based on image processing. The method includes: obtaining a standardized surface image of the part through an image acquisition system; extracting defect-sensitive features by combining a multi-scale Gaussian difference operator; filtering the defect-sensitive features through a preset feature selection threshold, mapping features that meet the threshold to a set of candidate defect points; constructing a serialized feature processing network containing an attention mechanism and a memory unit; inputting the set of candidate defect points into the serialized feature processing network to generate an enhanced feature sequence with temporal correlation; aligning the enhanced feature sequence with the original topological graph in spatial position, propagating features through graph convolution operations to generate an enhanced feature map containing node attributes and edge weights; performing region growing and merging operations on the enhanced feature map, generating several connected candidate defect regions based on node attribute similarity and edge connection strength; extracting the geometric morphology descriptor and texture statistical descriptor of each candidate defect region, combining them to form a region feature description vector; performing a matching metric calculation between the region feature description vector and a standard defect feature vector pre-stored in a defect feature library, identifying the defect category of the candidate defect region based on the matching result; and outputting a surface defect detection map of the part with defect category and location information.

[0070] Example 1: See Figure 2 Under controlled lighting conditions, a sequence of original multi-view images of the part surface is acquired by a multi-angle linear scanning camera array. Adaptive lighting compensation and shadow removal are performed on each image in the original multi-view image sequence. The processed multi-view image sequence is then input into a dense 3D reconstruction engine to generate a 3D model of the part surface with 3D point cloud coordinates and surface normal information. The 3D model of the part surface is then parametrically unfolded in two dimensions to obtain a part surface image that eliminates perspective distortion and scale differences.

[0071] The surface image of a part is decomposed into a set of visual primitives with spatial coordinates, directional components, and texture attributes. This decomposition process includes: performing local contrast normalization and Gaussian difference filtering on the part surface image to obtain a preprocessed enhanced image; extracting a series of connected regions with stable boundaries on the enhanced image using the maximum stable extremum region detection method; calculating the spatial centroid coordinates, principal axis direction, region area, and texture attributes based on the gray-level co-occurrence matrix for each connected region to form visual primitives; and sorting all visual primitives according to their spatial centroid coordinates to form a visual primitive set. Based on the visual primitive set, each visual primitive is encoded using a pre-defined primitive dictionary to generate a high-dimensional sparse encoding vector, which is then used to construct a topological correlation graph of the part surface. A multi-scale Gaussian difference operator is applied to the topological correlation graph to generate a multi-scale feature map pyramid, and defect-sensitive features are extracted from the multi-scale feature map pyramid.

[0072] In specific implementations, under controlled lighting conditions, a multi-angle linear scanning camera array arranged on both sides and above the part's transport line synchronously acquires a sequence of original multi-view images of the part's surface, covering the entire exposed surface of the part. In some embodiments, each image in the original multi-view image sequence undergoes adaptive illumination compensation and shadow removal processing. This processing, based on Retinex theory, dynamically adjusts the pixel values ​​of each frame to balance uneven illumination, while morphological opening operations are used to eliminate local shadow areas caused by surface undulations or external occlusion. The processed multi-view image sequence is then input into a dense 3D reconstruction engine based on the fusion of stereo vision and structured light. This engine generates a 3D model of the part's surface with 3D point cloud coordinates and surface normal information through feature matching and triangulation calculations. In some embodiments, the 3D model of the part's surface is parametrically unfolded using a conformal mapping algorithm, mapping the 3D model surface onto a 2D plane without overlap, resulting in a part surface image that eliminates perspective distortion and scale differences while maintaining consistent geometric relationships. In its specific implementation, the conformal mapping algorithm involves projecting each triangular facet or surface element on the 3D model of the part surface onto a 2D parameter domain through mathematical transformation. Based on complex variable function theory, the algorithm ensures the invariance of local angles during the mapping process, thereby maintaining the shape features and relative geometric relationships of the original 3D surface on the 2D unfolded diagram. This process is achieved by solving a system of partial differential equations, such as using harmonic mapping or minimizing the angular distortion energy function, so that any neighboring points on the 3D model surface still maintain their topological adjacency on the 2D plane, avoiding overlapping or tearing phenomena during the mapping process, and finally generating a scale-uniform image of the part surface that can be used for subsequent image processing.

[0073] It is understandable that decomposing the part surface image into a set of visual primitives with spatial coordinates, directional components, and texture attributes is a key step. This decomposition process involves performing local contrast normalization and Gaussian difference filtering on the part surface image to obtain a preprocessed enhanced image. The scale parameter of the Gaussian difference filtering is set according to the typical texture period of the part surface. On the enhanced image, a series of connected regions with stable boundaries are extracted using a maximum stable extremum region detection method. The stability criterion threshold of this method is adjusted according to the image noise level. For each connected region, its spatial centroid coordinates, principal axis direction, region area, and texture attributes based on the gray-level co-occurrence matrix are calculated. Texture attributes include contrast, correlation, energy, and homogeneity, thus forming visual primitives. All visual primitives are sorted according to their spatial centroid coordinates from left to right and from top to bottom to form a visual primitive set.

[0074] In practical implementation, based on the visual primitive set, each visual primitive is encoded using a pre-defined primitive dictionary to generate a high-dimensional sparse encoding vector, and a topological correlation map of the part surface is constructed from the high-dimensional sparse encoding vector. A multi-scale Gaussian difference operator is applied to the topological correlation map to generate a multi-scale feature map pyramid, and defect-sensitive features are extracted and fused from the multi-scale feature map pyramid. Optionally, the contrast parameter in the texture attributes of the visual primitives is calculated using the following formula:

[0075]

[0076] in: This indicates the contrast value. and These are the row and column indices in the gray-level co-occurrence matrix. This represents the gray level at a given distance and direction. and The joint probability density of co-occurrences in the image. This calculation process can be understood as providing quantized texture features for each visual primitive. Applying a multi-scale Gaussian difference operator to the topological association graph involves representing the graph as an adjacency matrix and a node feature matrix, smoothing the adjacency matrix using a Gaussian kernel function under multiple preset scale parameters, and then performing graph convolution on the smoothed adjacency matrix and the node feature matrix to generate a series of node feature response maps under different receptive fields. These response maps are stacked to form a multi-scale feature map pyramid. Defect-sensitive features are extracted from the multi-scale feature map pyramid by performing cross-scale max pooling on the feature response values ​​of the same image location at all scales within the pyramid to generate a preliminary fused feature map. Then, the histogram of oriented gradients (OGG) is calculated on this preliminary fused feature map. Finally, the statistical features of the OGG at each location are concatenated with the original feature values ​​of that location in the preliminary fused feature map, forming a defect-sensitive feature that integrates multi-scale information and gradient structure information.

[0077] Example 2: See Figure 3 The pre-trained primitive dictionary is invoked. This primitive dictionary contains multiple visual primitive templates. The texture attributes and orientation components of each visual primitive in the set of visual primitives are compared with the visual primitive templates in the primitive dictionary. For each visual primitive, the indices of the top K primitive templates with the highest similarity values ​​are selected, and the values ​​at the corresponding index positions are set to the corresponding similarity values ​​in a zero vector with the same length as the capacity of the primitive dictionary, thereby generating a high-dimensional sparse coding vector.

[0078] A topological graph of the part surface is constructed using high-dimensional sparse coding vectors. Each visual primitive is treated as a node in the topological graph, and the corresponding high-dimensional sparse coding vector is used as the initial attribute of the node. The spatial distance and texture difference between any two visual primitive nodes are calculated. When the spatial distance is less than the spatial neighborhood radius threshold and the texture difference is less than the texture difference threshold, an edge is established between the corresponding two nodes. The edge is assigned a weight, which is inversely proportional to the weighted sum of the spatial distance and the texture difference. Thus, a topological graph containing nodes and weighted edges is constructed.

[0079] In the specific implementation, a pre-trained primitive dictionary is invoked. This dictionary contains multiple visual primitive templates clustered from a large number of normal part surface image patches using unsupervised learning methods. Each visual primitive template is a feature vector representing a specific local texture pattern and directional features. The texture attributes and directional components of each visual primitive in the set are compared with each visual primitive template in the dictionary using vector dot product operations for similarity. For each visual primitive in the set, the indices of the top K primitive templates with the highest similarity metrics in the primitive dictionary are selected, where K is a preset positive integer. In a zero-based vector of the same length as the primitive dictionary, the value at the corresponding index is set to the corresponding similarity metric, thus generating a high-dimensional sparse coding vector for each visual primitive. The number of non-zero elements in the high-dimensional sparse coding vector is K.

[0080] In some embodiments, the process of constructing a topological graph of a part surface from high-dimensional sparse encoded vectors is specifically described as follows: each visual primitive is treated as a node in the topological graph to be constructed, and the corresponding high-dimensional sparse encoded vector is used as the initial attribute of the node. The spatial distance and texture difference between any two visual primitive nodes are calculated. The spatial distance is calculated using the Euclidean distance formula based on the spatial centroid coordinates of the visual primitive, and the texture difference is calculated based on the high-dimensional sparse encoded vector of the visual primitive. In a specific implementation, when the calculated spatial distance is less than a preset spatial neighborhood radius threshold and the texture difference is less than a preset texture difference threshold, an undirected edge is established between the corresponding two visual primitive nodes. A weight is assigned to the established edge, and the weight of the edge is inversely proportional to the weighted sum of the spatial distance and the texture difference. The coefficient of the weighted sum is set according to the relative importance of spatial proximity and texture consistency in the specific application scenario, thereby constructing a topological graph containing nodes and weighted edges. Optionally, the texture difference can be calculated using the following formula:

[0081]

[0082] in: Indicates texture difference. and These represent the high-dimensional sparse coding vectors corresponding to the two visual primitives to be compared. This formula measures the cosine distance between the vectors. and Let a and b represent the Euclidean norms of vectors a and b, respectively, which are the magnitudes of the vectors.

[0083] In some embodiments, the spatial neighborhood radius threshold is determined based on the resolution of the part surface image and the expected minimum defect size to be detected, while the texture difference threshold is set by statistically analyzing the difference distribution among visual primitives of normal samples. It can be understood that the topological association graph established through the above steps not only encodes the local features of the part surface but also explicitly models the spatial adjacency and similarity relationships between features through weighted edges between nodes.

[0084] Example 3: Apply a multi-scale Gaussian difference operator to the topological association graph to generate a multi-scale feature map pyramid. The topological association graph is represented as an adjacency matrix and a node feature matrix. Under different scale parameters, the adjacency matrix is ​​Gaussian smoothed to obtain smoothed adjacency matrices at different scales. Graph convolution is performed on the smoothed adjacency matrix and the node feature matrix at each scale to obtain the node feature response map at that scale. The node feature response maps at all scales are stacked to form a multi-scale feature map pyramid.

[0085] Defect-sensitive features are extracted from the multi-scale feature map pyramid. The feature response values ​​of the same spatial location in the multi-scale feature map pyramid at different scales are subjected to cross-scale max pooling to generate a preliminary fused feature map. The directional gradient histogram is calculated on the preliminary fused feature map to obtain the directional gradient statistical features of each location. The directional gradient statistical features are concatenated with the feature values ​​of the corresponding locations in the preliminary fused feature map to form the defect-sensitive features.

[0086] In practice, a multi-scale Gaussian difference operator is applied to the topological graph to generate a multi-scale feature map pyramid. This operation begins by representing the topological graph as an adjacency matrix and a node feature matrix. The elements of the adjacency matrix are filled with the weights of the edges in the topological graph, and the rows of the node feature matrix are composed of the high-dimensional sparse coding vectors corresponding to each visual primitive node. In practice, a set of scale parameters arranged in ascending order is selected. At different scale parameters, the adjacency matrix is ​​Gaussian smoothed. Specifically, this is achieved by applying a Gaussian kernel convolution operation based on the scale parameter to the adjacency matrix, resulting in smoothed adjacency matrices at different scales. A graph convolution operation is then performed between the smoothed adjacency matrix at each scale and the node feature matrix. The graph convolution operation follows the propagation rules defined by the formula:

[0087]

[0088] in: Indicates the current Gaussian scale parameter The smoothed adjacency matrix yes The degree matrix, It is the first The node feature matrix of the layer It is a trainable parameter matrix. This represents a non-linear activation function, through which the node feature response map at this scale is obtained. Stacking the node feature response maps at all scales forms a multi-scale feature map pyramid arranged in an ordered manner along the scale dimension.

[0089] In some embodiments, the specific operation of fusing and extracting defect-sensitive features from a multi-scale feature map pyramid is as follows: For the feature response values ​​of the same spatial location at different scales in the multi-scale feature map pyramid, a cross-scale max pooling operation is performed. This operation compares the feature vectors of the same node in all scale feature response maps and selects the maximum value element-wise to generate a preliminary fused feature map that integrates multi-scale information. An oriented gradient histogram is calculated on the preliminary fused feature map. The oriented gradient histogram calculation divides the local image region mapped by the feature vector of each node into cell units, statistically analyzes the distribution of gradient directions within each cell unit, and connects the statistical histograms of all cell units to obtain the statistical features of the oriented gradient at each location.

[0090] In practical implementation, the scale parameter sequence is set based on the expected range of variation in the surface texture of the part. Smaller scale parameters are used to capture subtle local changes, while larger scale parameters are used to perceive more macroscopic regional patterns. Optionally, the cross-scale max pooling operation can be performed in parallel, simultaneously reading data from corresponding positions in all scale feature response maps to improve processing speed. In some embodiments, the cell size and number of orientation intervals used in the oriented gradient histogram calculation are preset fixed parameters. It can be understood that processing the topological correlation graph using the multi-scale Gaussian difference operator can simulate the perception of graph structure information under different receptive fields, and the subsequent fusion operation effectively aggregates the feature information from these different perspectives, providing a robust feature representation for subsequent defect discrimination.

[0091] Example 4: Construct a serialized feature processing network that includes an attention mechanism and a memory unit. Input the candidate defect point set into the serialized feature processing network to generate a reinforced feature sequence with temporal correlation. Arrange each point in the candidate defect point set according to its spatial location in scan line order to form an initial feature sequence. Input the initial feature sequence into the long short-term memory unit in the serialized feature processing network to learn the long-range dependencies of features in the sequence. Introduce an attention mechanism at the output of the long short-term memory unit to dynamically calculate the attention weight of each feature in the sequence. Perform a residual connection between the weighted feature sequence and the original feature sequence to output the reinforced feature sequence.

[0092] The enhanced feature map is subjected to region growing and merging operations. Based on the similarity of node attributes and the strength of edge connections, several connected candidate defect regions are generated. In the enhanced feature map, an unlabeled node is randomly selected as a seed point. With the seed point as the core, all adjacent nodes directly connected to it by edges are traversed. If the attribute similarity between the adjacent node and the seed point is greater than the attribute similarity threshold, and the strength of the connecting edge is greater than the edge strength threshold, then the adjacent node and the seed point are merged into the same region. The traversal and judgment operations are recursively performed on the nodes of the newly merged region until no new node meets the merging conditions, thus completing the growth of a candidate defect region. The region growing process starting from the selection of the seed point is repeated until all nodes in the enhanced feature map are labeled, and finally several connected candidate defect regions are generated.

[0093] In practice, each point in the candidate defect point set is arranged according to its spatial location using scan lines, traversing the image plane row by row from left to right and top to bottom, forming an initial feature sequence. This initial feature sequence is then sequentially input into the Long Short-Term Memory (LSTM) units of the serialization feature processing network. The LSM units learn the long-range dependencies of features in the sequence through their internal gating mechanism. An attention mechanism is introduced at the output of the LSM units to dynamically calculate the attention weight for each feature in the sequence. The attention weight is calculated as follows:

[0094]

[0095] in: Indicates the first in the sequence Attention weights for each position, It is an attention score calculated using a feedforward neural network, which takes the hidden state of the long short-term memory unit at the current position as input. It is the total length of the initial feature sequence. It is an exponential function used to map the attention score to the positive domain, ensuring the mathematical validity of softmax normalization. The weighted feature sequence is residually concatenated with the original feature sequence, i.e., the feature vectors at corresponding positions are added together to output the enhanced feature sequence.

[0096] In some embodiments, region growing and merging operations are performed on the enhanced feature map. Based on node attribute similarity and edge connection strength, several connected candidate defect regions are generated. Each node in the enhanced feature map contains spatial coordinates and feature attributes processed by graph convolution propagation and serialization. In the enhanced feature map, a node not labeled by any existing region is randomly selected as a seed point. Using the seed point as the core, all adjacent nodes directly connected to the seed point through edges are traversed. If the attribute similarity between the adjacent node and the seed point is greater than the attribute similarity threshold, and the strength of the edge connecting the seed point and the adjacent node is greater than the edge strength threshold, then the adjacent node and the seed point are merged into the same region. The traversal and judgment operations are recursively performed on the nodes of the newly merged region. The nodes of the newly merged region are regarded as new growth cores, and their adjacent nodes are checked to see if they meet the merging conditions, until no new nodes meet the merging conditions, completing the growth of a candidate defect region. The region growing process starting from the seed point is repeated, selecting new seed points from the remaining unlabeled nodes in the enhanced feature map, until all nodes in the enhanced feature map are labeled, finally generating several connected candidate defect regions.

[0097] In practice, attribute similarity is obtained by calculating the cosine similarity between the feature vectors of two nodes, and edge strength is the weight of the corresponding edge in the topological graph. It can be understood that the recursive process of region growth and merging operations can aggregate nodes with similar attributes and close connections together, thereby forming a coherent suspected defect region. The attribute similarity threshold and edge strength threshold used to control the growth process can be configured according to the characteristics of different part surfaces. See Table 1.

[0098] Table 1: A set of parameters for scratch detection on metal surfaces

[0099] Parameter name Parameter value illustrate Attribute similarity threshold 0.85 Least cosine similarity between node feature vectors Edge strength threshold 0.70 Minimum value of edge weight

[0100] Optionally, the random selection of seed points can be replaced by starting with the node whose attribute value differs most from the global average among unlabeled nodes, prioritizing growth from the most anomalous regions. In some embodiments, the region growth process is implemented iteratively rather than recursively, managing the growth front by maintaining a queue of nodes to be inspected. It can be understood that the sequential feature processing network models the contextual relationships of candidate points on the scanning path through long short-term memory units, the attention mechanism enhances features at key locations, and the subsequent region growth algorithm, based on the enhanced node attributes and the original graph connectivity, aggregates discrete anomalous points into meaningful candidate region objects, providing explicit processing units for subsequent feature extraction and classification steps.

[0101] See Figure 4 The LSTM hidden state change curves during the sequential feature processing stage illustrate the dynamic evolution of the three hidden state dimensions after candidate defect points are input into the Long Short-Term Memory (LSTM) unit in scanline order. Specifically, the scanline order (horizontal axis) corresponds to the spatial traversal order of candidate points from left to right and from top to bottom, while the vertical axis represents the hidden state values: Hidden state dimension 3 (blue curve) initially appears at a high level, showing a continuous downward trend as the sequence progresses, reflecting the long-range dependency modeling process of the LSTM on the initial input features; Hidden state dimensions 1 (red) and 2 (green) exhibit a fluctuating decrease followed by a synchronous increase and convergence, with the two intersecting in the middle of the sequence (approximately positions 40-60), demonstrating the selective memorization and forgetting of different temporal features by the LSTM gating mechanism. In the later stages of the sequence (after position 80), the state values ​​of the three dimensions gradually converge, and combined with the weighting effect of the attention mechanism, this indicates that the model has completed the feature enhancement of key candidate points, and the convergence of the hidden states verifies the effective learning of the long short-term dependency relationship.

[0102] Example 5: The region feature description vector is matched and measured with the standard defect feature vectors pre-stored in the defect feature library. The defect category of the candidate defect region is identified according to the matching result. The cosine similarity between the region feature description vector and each standard defect feature vector in the defect feature library is calculated. The standard defect feature vector with the highest cosine similarity is selected. It is determined whether its corresponding similarity value exceeds the predetermined category determination threshold. If it exceeds the threshold, the defect category corresponding to the standard defect feature vector is assigned to the candidate defect region from which the region feature description vector originates. If it does not exceed the threshold, the defect category of the candidate defect region is identified as an unknown category.

[0103] In practice, the regional feature description vector of the candidate defect region is matched and measured against the standard defect feature vector pre-stored in the defect feature library. The defect feature library stores standard feature representations of various typical defects after annotation and feature extraction. The regional feature description vector is formed by combining the geometric morphology descriptor and texture statistical descriptor of the candidate defect region. In practice, the cosine similarity between the regional feature description vector and each standard defect feature vector in the defect feature library is calculated. The cosine similarity calculation follows the formula defined below:

[0104]

[0105] in: Represents the similarity value. This represents the feature description vector of the region to be determined. This represents a standard defect feature vector in the defect feature library. This represents the dot product operation of vectors. Let represent the Euclidean norm of the vector. Among all calculated cosine similarity values, the standard defect feature vector with the highest value and its corresponding similarity value are selected as the best matching result.

[0106] In some embodiments, it is determined whether the similarity value corresponding to the best matching result exceeds a predetermined category determination threshold. The category determination threshold is a scalar value between 0 and 1, and its setting is based on statistical analysis of the matching results of known defect samples. If the similarity value corresponding to the best matching result exceeds the category determination threshold, the defect category represented by the standard defect feature vector corresponding to the best matching result is assigned to the candidate defect region from which the region feature description vector originates. If the similarity value corresponding to the best matching result does not exceed the category determination threshold, the defect category of this candidate defect region is marked as an unknown category. It can be understood that the setting of the category determination threshold directly affects the strictness of classification. A higher threshold increases the confidence of category determination but increases the number of unknown categories, while a lower threshold will classify more regions but is accompanied by the risk of misjudgment. In a specific implementation, after completing the category identification or unknown category identification of all candidate defect regions, the system integrates the spatial location information of each region with its corresponding category label to generate and output a part surface defect detection map with defect category and location information. Optionally, for candidate defect regions marked as unknown categories, they can be highlighted in the detection map with special colors or marks to prompt operators to perform manual review. In some embodiments, the defect feature library supports dynamic updates, and new standard defect feature vectors can be added to the library after verification, thereby enabling the system's recognition capabilities to continuously evolve.

[0107] See Figure 5In the analysis of multi-scale feature extraction performance, the correlation between feature extraction scale and feature extraction accuracy and feature response intensity was demonstrated. Specifically, a dual-axis correlation curve was constructed with feature extraction scale as the horizontal axis and feature extraction accuracy (left vertical axis) and feature response intensity (right vertical axis) as evaluation indicators. When the feature extraction scale increased from 1.0 to 3.0, both feature extraction accuracy and feature response intensity showed a significant upward trend, reaching their peak values ​​simultaneously at scale 3.0 (accuracy 0.88, response intensity 0.90). Beyond scale 3.0, both showed a simultaneous downward trend, reflecting the synergistic influence of feature extraction scale on accuracy and response intensity. This result provides a quantitative basis for selecting the optimal scale in multi-scale feature extraction.

[0108] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0109] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting surface defects in parts based on image processing, characterized in that, Includes the following steps: Standardized part surface images are obtained through an image acquisition system, and defect-sensitive features are extracted by combining multi-scale Gaussian difference operators. By using a preset feature filtering threshold, the defect-sensitive features are filtered, and the features that meet the feature filtering threshold are mapped to a set of candidate defect points. A serialized feature processing network containing an attention mechanism and a memory unit is constructed. The candidate defect point set is input into the serialized feature processing network to generate a reinforced feature sequence with temporal correlation. The enhanced feature sequence is spatially aligned with the original topological association graph, and the features are propagated through graph convolution operations to generate an enhanced feature graph containing node attributes and edge weights. The enhanced feature map is subjected to region growing and merging operations, and several connected candidate defect regions are generated based on node attribute similarity and edge connection strength. Extract the geometric morphology descriptor and texture statistical descriptor of each candidate defect region, and combine them to form a region feature description vector; The region feature description vector is matched and measured with the standard defect feature vector pre-stored in the defect feature library, and the defect category of the candidate defect region is identified according to the matching result. Output a surface defect detection map of the part with defect category and location information.

2. The method for detecting surface defects of parts based on image processing according to claim 1, characterized in that, The process of obtaining standardized part surface images through an image acquisition system includes: Under controlled lighting conditions, a sequence of original multi-view images of the surface of a part is acquired using a multi-angle linear scanning camera array; Adaptive illumination compensation and shadow removal processing are performed on each image in the original multi-view image sequence; The processed multi-view image sequence is input into the dense 3D reconstruction engine to generate a 3D model of the part surface with 3D point cloud coordinates and surface normal information. The three-dimensional model of the part surface is parametrically unfolded in two dimensions to obtain a part surface image that eliminates perspective distortion and scale differences.

3. The method for detecting surface defects of parts based on image processing according to claim 1, characterized in that, The extraction of defect-sensitive features by combining multi-scale Gaussian difference operators includes: The surface image of the part is decomposed into a set of visual primitives with spatial coordinates, directional components, and texture attributes; Based on the set of visual primitives, each visual primitive is encoded using a preset primitive dictionary to generate a high-dimensional sparse encoding vector, and a topological association graph of the part surface is constructed from the high-dimensional sparse encoding vector. A multi-scale Gaussian difference operator is applied to the topological correlation graph to generate a multi-scale feature map pyramid, and defect-sensitive features are extracted from the multi-scale feature map pyramid. The step of decomposing the surface image of the part into a set of visual primitives with spatial coordinates, directional components, and texture attributes includes: Local contrast normalization and Gaussian difference filtering are performed on the part surface image to obtain the preprocessed enhanced image; On the enhanced image, a series of connected regions with stable boundaries are extracted using the maximum stable extremum region detection method; For each connected region, its spatial centroid coordinates, principal axis direction, region area, and texture attributes based on the gray-level co-occurrence matrix are calculated to form the visual primitive. All visual primitives are sorted according to their spatial centroid coordinates to form the visual primitive set.

4. The method for detecting surface defects of parts based on image processing according to claim 3, characterized in that, The step of encoding each visual primitive based on the set of visual primitives using a preset primitive dictionary to generate a high-dimensional sparse coding vector includes: Invoke the pre-trained primitive dictionary, which contains multiple visual primitive templates; The texture attributes and orientation components of each visual primitive in the visual primitive set are compared with the similarity of each visual primitive template in the primitive dictionary. For each visual primitive, the indices of the top K primitive templates with the highest similarity metric values ​​are selected, and in a zero vector with the same length as the primitive dictionary capacity, the value at the corresponding index position is set to the corresponding similarity metric value, thereby generating the high-dimensional sparse coding vector.

5. The method for detecting surface defects of parts based on image processing according to claim 3, characterized in that, The construction of the topological correlation graph of the part surface from the high-dimensional sparse coding vector includes: Each visual primitive is treated as a node in the topological association graph, and the corresponding high-dimensional sparse coding vector is used as the initial attribute of the node. Calculate the spatial distance and texture difference between any two visual primitive nodes. When the spatial distance is less than the spatial neighborhood radius threshold and the texture difference is less than the texture difference threshold, establish an edge between the corresponding two nodes. The edges are assigned weights that are inversely proportional to the weighted sum of the spatial distance and texture difference, thereby constructing the topological association graph containing nodes and weighted edges.

6. The method for detecting surface defects of parts based on image processing according to claim 3, characterized in that, The step of applying a multi-scale Gaussian difference operator to the topological correlation graph to generate a multi-scale feature map pyramid includes: The topological association graph is represented as an adjacency matrix and a node feature matrix; Gaussian smoothing is applied to the adjacency matrix under different scale parameters to obtain smoothed adjacency matrices at different scales; Graph convolution is performed between the smoothed adjacency matrix and the node feature matrix at each scale to obtain the node feature response map at the specified scale. Stack the node feature response maps at all scales to form the multi-scale feature map pyramid.

7. The method for detecting surface defects of parts based on image processing according to claim 3, characterized in that, The step of fusing and extracting defect-sensitive features from the multi-scale feature map pyramid includes: For the feature response values ​​of the same spatial location in the multi-scale feature map pyramid at different scales, cross-scale max pooling is performed to generate a preliminary fused feature map. The orientation gradient histogram is calculated on the preliminary fused feature map to obtain the orientation gradient statistical features at each location; The directional gradient statistical features are concatenated with the feature values ​​at corresponding positions in the preliminary fusion feature map to form the defect-sensitive features.

8. The method for detecting surface defects of parts based on image processing according to claim 1, characterized in that, The process involves constructing a sequential feature processing network that includes an attention mechanism and memory units. The candidate defect set is input into the sequential feature processing network to generate a reinforcement feature sequence with temporal correlation, including: Each point in the candidate defect point set is arranged in scan line order according to its spatial location to form an initial feature sequence; The initial feature sequence is sequentially input into the long short-term memory unit in the serialization feature processing network to learn the long-range dependencies of features in the sequence. An attention mechanism is introduced at the output of the long short-term memory unit to dynamically calculate the attention weight of each position feature in the sequence; The weighted feature sequence is residually concatenated with the original feature sequence to output the enhanced feature sequence.

9. The method for detecting surface defects of parts based on image processing according to claim 1, characterized in that, The enhanced feature map is subjected to region growing and merging operations, which generate several connected candidate defect regions based on node attribute similarity and edge connection strength, including: In the enhanced feature map, an unlabeled node is randomly selected as a seed point; Using the seed point as the core, traverse all adjacent nodes directly connected to it by edges. If the attribute similarity between the adjacent node and the seed point is greater than the attribute similarity threshold, and the strength of the connecting edge is greater than the edge strength threshold, then merge the adjacent node and the seed point into the same region. Recursively perform traversal and judgment operations on the nodes of the newly merged region until no new node meets the merging condition, thus completing the growth of a candidate defect region; The region growth process, starting from the selected seed point, is repeated until all nodes in the enhanced feature map are marked, ultimately generating several connected candidate defect regions.

10. The method for detecting surface defects of parts based on image processing according to claim 1, characterized in that, The step of performing a matching metric calculation between the region feature description vector and the standard defect feature vector pre-stored in the defect feature library, and identifying the defect category of the candidate defect region based on the matching result, includes: Calculate the cosine similarity between the region feature description vector and each standard defect feature vector in the defect feature library; Select the standard defect feature vector with the highest cosine similarity and determine whether its corresponding similarity value exceeds the predetermined category determination threshold; If the number exceeds the limit, the defect category corresponding to the standard defect feature vector is assigned to the candidate defect region from which the regional feature description vector originates. If the limit is not exceeded, the defect category of the candidate defect region will be marked as an unknown category.

Images