Real-time detection and classification method and system for machine vision defects based on deep learning

Abstract

The application provides a kind of machine vision defect real-time detection and classification method and system based on deep learning, it is related to machine vision detection field, including by calculating local entropy value and gradient direction consistency to determine regional enhancement weight, and carry out regional adaptive enhancement;Establish feature transmission sequence and progressive fusion feature;Generate and correct defect area probability distribution map;Build dynamic decision matrix to calculate comprehensive score for defect grading.The method can improve the accuracy of defect detection under complex background, reduce false detection and missed detection, realize real-time positioning and accurate classification of defects.

Description

TECHNICAL FIELD

[0001] The present application relates to machine vision detection technology, and in particular to a machine vision defect real-time detection and classification method and system based on deep learning. BACKGROUND

[0002] Industrial product surface defect detection is a key link of product quality control. Traditional manual detection methods have low efficiency and poor consistency. With the development of deep learning technology, automated detection methods based on machine vision have gradually become a research hotspot. However, under complex background and lighting conditions, detection accuracy and real-time performance still face challenges.

[0003] The existing deep learning detection method mainly has the following problems: the image feature extraction is not sufficient, which leads to low detection accuracy; the defect area positioning is not accurate, which affects the classification effect; the defect evaluation standard is single, which is difficult to meet the quality control needs of different products; and the detection process is complex, which is difficult to realize real-time requirements.

[0004] There is an urgent need for a detection method that can adaptively enhance image features, optimize feature fusion paths, accurately locate defect areas, and realize hierarchical evaluation, in order to improve detection accuracy and real-time performance, and realize intelligent detection and classification of industrial product surface defects. SUMMARY

[0005] The embodiment of the present application provides a machine vision defect real-time detection and classification method and system based on deep learning, which can solve the problems in the prior art.

[0006] In a first aspect, the embodiment of the present application provides a machine vision defect real-time detection and classification method based on deep learning, comprising:

[0007] Obtain image data of the surface of an industrial product, calculate the local entropy value and gradient direction consistency of each pixel point in the image data, determine the region enhancement weight according to the local entropy value and the gradient direction consistency, perform region adaptive enhancement on the image data based on the region enhancement weight, and obtain enhanced image data;

[0008] Extract the structural features of the enhanced image data, establish a feature transmission sequence, calculate the correlation matrix between features according to the feature transmission sequence, establish a feature optimization path based on the correlation matrix, and perform progressive fusion of the features along the feature optimization path to obtain fused feature data;

[0009] Generate a defect region probability distribution map according to the fused feature data, correct the defect region probability distribution map in combination with the region enhancement weight, determine the defect region, extract the topological structure features of the defect region, establish a defect feature description, classify the defects according to the defect feature description, and output the defect type and confidence score;

[0010] A dynamic decision matrix is constructed based on the region enhancement weight and the confidence score, a comprehensive score of each defect region is calculated in the dynamic decision matrix, the defects are graded according to the comprehensive score, a quantitative index of defect severity is generated, and a defect detection result is output.

[0011] In an alternative embodiment,

[0012] Image data of the surface of the industrial product is acquired, local entropy values and gradient direction consistencies of each pixel point in the image data are calculated, region enhancement weights are determined according to the local entropy values and the gradient direction consistencies, and the image data is regionally and adaptively enhanced based on the region enhancement weights to obtain enhanced image data, including:

[0013] Image data of the surface of the industrial product is acquired, and the image data is regionally segmented along a preset direction sequence to obtain a plurality of region sub-images;

[0014] In each region sub-image, a correlation strength matrix of a pixel point and its neighborhood is constructed, a local entropy value of each pixel point is calculated based on the correlation strength matrix, a gradient vector of each pixel point is extracted and projected in different directions, a gradient direction consistency of each pixel point is calculated according to the projection components, and the local entropy values and the gradient direction consistencies are combined to generate region characteristic data;

[0015] Extreme value points of the local entropy values and the gradient direction consistencies of each pixel point in the region characteristic data are extracted, region seed points are clustered from the extreme value points that meet a preset extreme value range, a topological connection relationship between the region seed points is established, a characteristic transfer coefficient is calculated, and a region growth path is generated;

[0016] The local entropy values and the gradient direction consistencies of the region seed points are transferred outward along the region growth path, a decay factor is calculated according to a distance of a pixel point to a region seed point, a product of the local entropy value, the gradient direction consistency, and the decay factor is taken as a region enhancement weight, the region characteristic data is weighted based on the region enhancement weight, and an enhancement coefficient is obtained;

[0017] A non-linear mapping function is constructed according to the enhancement coefficient, each region sub-image is enhanced, the enhancement intensity is adjusted by introducing the characteristic transfer coefficient at the region boundary, the enhanced region sub-image is reorganized, and enhanced image data is obtained.

[0018] In an alternative embodiment,

[0019] Structure features of the enhanced image data are extracted, and a feature transfer sequence is established, including:

[0020] constructing a pyramid structure on the enhanced image data, dividing overlapping sub-regions in each layer of the pyramid structure, extracting local gradient distribution of the overlapping sub-regions, adjusting filter direction parameters based on the local gradient distribution;

[0021] extracting structural features of the overlapping sub-regions using the adjusted filter, establishing corresponding mapping of the structural features in overlapping regions of adjacent layers, determining a feature fusion order according to the corresponding mapping, combining structural features of each layer according to the feature fusion order to obtain multi-layer structural features;

[0022] constructing a hierarchical transfer tree based on the feature fusion order, establishing a feature transfer link in the hierarchical transfer tree, the feature transfer link including transfer nodes between adjacent features, analyzing spatial structure distribution and direction consistency between adjacent transfer nodes, determining node transfer priority according to the spatial structure distribution and direction consistency, and generating a feature transfer sequence based on the node transfer priority.

[0023] In an optional embodiment,

[0024] calculating a correlation degree matrix between features according to the feature transfer sequence, establishing a feature optimization path based on the correlation degree matrix, and progressively fusing features along the feature optimization path to obtain fused feature data, including:

[0025] calculating feature correlation strength between adjacent transfer nodes in the feature transfer sequence, and transferring the feature correlation strength level by level in the feature transfer sequence to construct a correlation degree matrix;

[0026] determining an initial transfer node in the correlation degree matrix, expanding a next transfer node from the initial transfer node level by level based on the feature correlation strength, establishing a transfer connection between nodes, and forming a feature optimization path by connecting the transfer connections in order from high to low according to the feature correlation strength;

[0027] along the feature optimization path, starting from the initial transfer node, fusing features of a current transfer node and features of a next transfer node by weighting, the weight of the weighted fusion being determined by the feature correlation strength of the corresponding transfer connection, taking the fused features as new current node features, and continuing to fuse with features of a next transfer node, completing fusion of features of all transfer nodes by progressive level by level to obtain final fused feature data.

[0028] In an optional embodiment,

[0029] generating a defect region probability distribution map according to the fused feature data, correcting the defect region probability distribution map in combination with a region enhancement weight, and determining a defect region, including:

[0030] calculating local statistics on the fusion feature data, constructing an adaptive kernel function based on the local statistics, decomposing the adaptive kernel function by using an orthogonal transformation to obtain a basis function set, and performing convolution calculation on the basis function set and the fusion feature data to obtain a feature response map;

[0031] performing gradient diffusion on the feature response map to obtain a density flow field, extracting gradient trajectories in the density flow field, calculating regional clustering features according to converging points and converging directions of the gradient trajectories, and generating a defect region probability distribution map based on the regional clustering features;

[0032] performing adaptive weighted fusion on the defect region probability distribution map and a region enhancement weight to obtain a corrected probability map, extracting a boundary point set of the corrected probability map, calculating multi-order moment features of the boundary point set, constructing a level set function on the corrected probability map, adaptively adjusting an evolution speed of the level set function based on the multi-order moment features, and performing segmentation on a zero level surface of the level set function to obtain a defect region.

[0033] In an optional embodiment,

[0034] extracting topological structure features of the defect region, establishing a defect feature description, performing defect classification according to the defect feature description, and outputting a defect type and a confidence score, including:

[0035] extracting a closed contour of the defect region by using an adaptive threshold boundary tracking algorithm, performing noise reduction and smoothing processing on the closed contour to obtain a continuous boundary, establishing a contour point sequence, constructing a distance transformation field based on the contour point sequence, extracting a center line from the distance transformation field by using a gradient descent method, and constructing a center point sequence;

[0036] mapping the contour point sequence and the center point sequence to the same coordinate space to construct a topological structure of the defect region, extracting branch nodes and intersection points in the topological structure, calculating connection relationships between adjacent nodes, constructing a topological tree according to the connection relationships, and extracting structure features of the topological tree as topological structure features of the defect region;

[0037] performing normalization processing on each feature component of the topological structure features, calculating correlation coefficients between the normalized feature components, constructing a feature correlation matrix, calculating a discrimination ability of each feature component based on the feature correlation matrix, determining a feature weight according to the discrimination ability, performing weighted combination on the feature components by using the feature weight, and establishing a defect feature description;

[0038] The defect feature descriptions are matched with preset multi-type defect samples, the distance and similarity between the feature descriptions are calculated, a feature matching matrix is ​​constructed, the matching degree between the test sample and each type of defect sample in the feature matching matrix is ​​analyzed, the type with the highest matching degree is determined as the defect type, and its corresponding matching degree is used as the confidence score.

[0039] In one alternative embodiment,

[0040] A dynamic decision matrix is ​​constructed based on region enhancement weights and confidence scores. A comprehensive score is calculated for each defect region within this matrix. Defects are then classified according to their comprehensive scores, generating a quantitative index of defect severity. The defect detection results are then output, including:

[0041] The uniformity of the distribution of the region enhancement weights and the fluctuation range of the confidence scores are calculated. The uniformity of the distribution and the fluctuation range are used as scoring weight coefficients to perform weighted fusion of the region enhancement weights and the confidence scores to generate a scoring matrix.

[0042] The variation pattern of the region enhancement weight with location is extracted to obtain the region variation feature, and the stability of the confidence score is extracted to obtain the reliability feature. The dimension weights of the scoring matrix are calculated based on the region variation feature and the reliability feature, and a dynamic decision matrix is ​​constructed based on the dimension weights.

[0043] The relative importance between dimensions is calculated in the dynamic decision matrix to generate an importance weight vector. The importance weight vector is then multiplied by the scoring matrix to obtain a comprehensive score for each defect area.

[0044] A graded evaluation standard is constructed based on the comprehensive score. The severity distribution is calculated based on the graded evaluation standard. The severity distribution is transformed into a severity quantification index using a membership function. The defect level is classified according to the severity quantification index and the detection results are output.

[0045] A second aspect of this invention provides a real-time machine vision defect detection and classification system based on deep learning, comprising:

[0046] The first unit is used to acquire image data of the surface of industrial products, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the region enhancement weight based on the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the region enhancement weight to obtain the enhanced image data.

[0047] The second unit is used to extract the structural features of the enhanced image data, establish a feature transfer sequence, calculate the correlation matrix between features based on the feature transfer sequence, establish a feature optimization path based on the correlation matrix, and perform progressive fusion of features along the feature optimization path to obtain fused feature data.

[0048] The third unit is used to generate a probability distribution map of defect areas based on the fused feature data, correct the probability distribution map of defect areas by combining the region enhancement weight, determine the defect areas, extract the topological structure features of the defect areas, establish defect feature descriptions, classify defects based on the defect feature descriptions, and output defect types and confidence scores.

[0049] The fourth unit is used to construct a dynamic decision matrix based on regional enhancement weights and confidence scores. It calculates the comprehensive score for each defect region in the dynamic decision matrix, classifies the defects according to the comprehensive score, generates a quantitative index of defect severity, and outputs the defect detection results.

[0050] A third aspect of the present invention provides an electronic device, comprising:

[0051] processor;

[0052] Memory used to store processor-executable instructions;

[0053] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0054] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0055] In this embodiment, region enhancement weights are calculated using local entropy and gradient direction consistency, achieving regional adaptive enhancement. This improves the contrast and clarity of defect areas, solving the problem of traditional methods struggling to identify low-contrast and blurry defects, and significantly enhancing the accuracy and sensitivity of defect detection. A feature transfer sequence and correlation matrix are established to achieve progressive feature fusion, effectively preserving the multi-scale structural information of defects. Through topological feature extraction and defect feature description, accurate classification of different types of defects is achieved, improving the system's robustness to defect identification in complex backgrounds. A dynamic decision matrix is ​​constructed, and a comprehensive defect area score is calculated, enabling quantitative grading of defect severity. This provides a precise reference for product quality control. Simultaneously, the feature extraction and fusion process is optimized, reducing computational complexity and ensuring the system's real-time performance, making it suitable for the online inspection needs of high-speed industrial production lines. Attached Figure Description

[0056] Figure 1 This is a flowchart illustrating the real-time defect detection and classification method for machine vision based on deep learning, as described in an embodiment of the present invention.

[0057] Figure 2 This is a schematic diagram comparing the performance of different defect detection methods in embodiments of the present invention;

[0058] Figure 3 This is a schematic diagram of the intelligent defect detection and comprehensive rating process in this embodiment. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0061] Figure 1 This is a flowchart illustrating the real-time defect detection and classification method based on deep learning for machine vision according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0062] Acquire image data of the surface of industrial products, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the region enhancement weight based on the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the region enhancement weight to obtain the enhanced image data.

[0063] Structural features of the enhanced image data are extracted, a feature transfer sequence is established, the correlation matrix between features is calculated based on the feature transfer sequence, a feature optimization path is established based on the correlation matrix, and features are progressively fused along the feature optimization path to obtain fused feature data.

[0064] A probability distribution map of defect areas is generated based on the fused feature data. The probability distribution map of defect areas is then corrected by combining the region enhancement weights to identify defect areas. The topological features of the defect areas are extracted to establish defect feature descriptions. Defects are classified based on the defect feature descriptions, and the defect type and confidence score are output.

[0065] A dynamic decision matrix is ​​constructed based on regional enhancement weights and confidence scores. A comprehensive score for each defect region is calculated in the dynamic decision matrix. Defects are classified according to the comprehensive score, a quantitative index of defect severity is generated, and the defect detection results are output.

[0066] In one optional implementation, image data of the surface of an industrial product is acquired, the local entropy value and gradient direction consistency of each pixel in the image data are calculated, region enhancement weights are determined based on the local entropy value and gradient direction consistency, and the image data is adaptively enhanced by region based on the region enhancement weights to obtain enhanced image data including:

[0067] Acquire image data of the surface of industrial products, and perform region segmentation on the image data along a preset direction sequence to obtain multiple region sub-images;

[0068] Within each sub-image region, a correlation strength matrix between a pixel and its neighborhood is constructed. Based on the correlation strength matrix, the local entropy value of each pixel is calculated. The gradient vector of each pixel is extracted and projected in different directions. The gradient direction consistency of each pixel is calculated based on the projection components. The local entropy value and gradient direction consistency are combined to generate regional characteristic data.

[0069] Extract the extreme points in the regional characteristic data where the local entropy value of each pixel is consistent with the gradient direction, cluster the extreme points that meet the preset extreme value range to obtain regional seed points, establish the topological connection relationship between regional seed points, calculate the characteristic transfer coefficient, and generate regional growth path;

[0070] The local entropy value and gradient direction consistency of the region seed point are propagated outward along the region growth path. The attenuation factor is calculated based on the distance from the pixel to the region seed point. The product of the local entropy value, gradient direction consistency and attenuation factor is used as the region enhancement weight. The region characteristic data is weighted based on the region enhancement weight to obtain the enhancement coefficient.

[0071] A nonlinear mapping function is constructed based on the enhancement coefficient to enhance each region sub-image; a characteristic transfer coefficient is introduced at the region boundary to adjust the enhancement intensity, and the enhanced region sub-images are recombined to obtain the enhanced image data.

[0072] In this embodiment, image data of the industrial product surface is acquired by imaging the product surface using a high-resolution industrial camera. During acquisition, the camera is perpendicular to the product surface to maintain uniform illumination and avoid strong light reflection and shadow interference. The image acquisition resolution is set to 2048×1536 pixels, and the image is saved as an 8-bit grayscale image. Image preprocessing includes noise reduction and contrast correction to ensure the quality of the original image.

[0073] When segmenting the acquired image data along a preset directional sequence, the preset directional sequence includes horizontal, vertical, and diagonal directions. A multi-scale method is employed, setting sliding windows of different sizes in each direction, with window sizes of 16×16, 32×32, and 64×64 pixels respectively. The step size of the sliding window is set to 1 / 4 of the window size to ensure sufficient overlap between adjacent regions. The entire image is scanned through the sliding window to obtain multiple sub-images of different regions.

[0074] When constructing the association strength matrix between a pixel and its neighborhood within each sub-image region, a 5×5 pixel area is defined as the neighborhood centered on the current pixel. The grayscale difference between the center pixel and each pixel in its neighborhood is calculated; the smaller the difference, the stronger the association strength. The association strength is determined by an inverse proportional function of the grayscale difference. When the grayscale difference is 0, the association strength reaches its maximum value of 1; when the grayscale difference exceeds a threshold of 20, the association strength drops below 0.1.

[0075] When calculating the local entropy value of each pixel based on the correlation strength matrix, the correlation strength matrix is ​​normalized so that the sum of all its elements is 1, forming a probability distribution. For a pixel with coordinates (i, j) in the image, the distribution of gray values ​​within its 5×5 neighborhood is statistically analyzed, and the entropy value of this distribution is calculated as the local entropy value of the pixel. The higher the local entropy value, the higher the texture complexity of the region; the lower the local entropy value, the higher the smoothness of the region.

[0076] When extracting the gradient vector for each pixel, the Sobel operator is used to calculate the gradient components in the horizontal and vertical directions respectively. For a pixel with coordinates (i, j) in the image, the horizontal gradient is obtained by calculating the gray-level difference between pixels (i, j+1) and (i, j-1), and the vertical gradient is obtained by calculating the gray-level difference between pixels (i+1, j) and (i-1, j). Combining the horizontal and vertical gradient components, the magnitude and direction of the gradient vector are obtained.

[0077] The gradient vector is projected in eight different directions, with 45-degree intervals between directions. For each pixel, the distribution of gradient directions among pixels in its 11×11 neighborhood is statistically analyzed. If the gradient component in a certain direction accounts for more than 60%, the region is considered to have high directional consistency; if the gradient components are evenly distributed in all directions, the directional consistency is low. The gradient direction consistency value range is [0, 1], with values ​​closer to 1 indicating greater consistency in gradient directions within the region.

[0078] When combining local entropy and gradient direction consistency to generate regional characteristic data, a weighted average is applied with a weight ratio of 6:4. For regions rich in detail, the weight of local entropy is increased to 0.7; for edge and texture regions, the weight of gradient direction consistency is increased to 0.6. The combined characteristic data reflects the complexity and structural features of the region. When extracting the extreme points of local entropy and gradient direction consistency for each pixel in the regional characteristic data, a threshold of 0.75 for local entropy and a threshold of 0.8 for gradient direction consistency are set. Local maximum or minimum values ​​are searched within a 7×7 window, and points that meet the conditions are marked as candidate extreme points. The preset extreme value range is points with local entropy values ​​between [0.7, 0.9] or gradient direction consistency between [0.75, 0.95].

[0079] When clustering extreme points that meet the preset extreme value range, a density-based clustering algorithm is used, with a cluster radius of 15 pixels. When the Euclidean distance between two extreme points is less than the cluster radius and their characteristic difference is less than 0.2, they are grouped into the same class. Each cluster center is determined as a region seed point. For a 2048×1536 image, typically 30-50 region seed points can be obtained.

[0080] When establishing topological connections between seed points in a region, connections are established between seed points less than 50 pixels apart. Connection strength is directly proportional to the similarity of characteristics between seed points and inversely proportional to distance. A characteristic transfer coefficient is calculated based on connection strength, ranging from [0.3, 0.9], with a transfer coefficient close to 0.9 between highly similar seed points. When generating a region growth path, it starts from the seed point with the highest characteristic value and gradually expands along the direction of greatest connection strength. During growth, the rate of change of characteristics from the current point to the next does not exceed 0.15, ensuring the smoothness of the growth path. The growth path is recorded as a sequence of coordinate points for subsequent characteristic transfer. When consistently transferring the local entropy value and gradient direction of the region seed points outward along the region growth path, a maximum transfer radius of 100 pixels is set. The transfer strength gradually weakens as distance increases. An exponential decay function is used for the decay factor: 0.9 for a distance of 10 pixels; 0.5 for a distance of 50 pixels; and 0.2 for a distance of 100 pixels.

[0081] The region enhancement weight is calculated by multiplying the local entropy, gradient direction consistency, and decay factor. For high-texture regions, the local entropy contributes a weight of 0.7; for regions with obvious directionality, the gradient direction consistency contributes a weight of 0.6. The enhancement weight ranges from [0.1, 1.0], with a larger weight resulting in a more significant enhancement effect. When enhancing sub-image regions using a nonlinear mapping function constructed based on the enhancement coefficient, an S-shaped curve is used. For regions with high enhancement coefficients, the curve slope is larger, enhancing contrast; for regions with low enhancement coefficients, the curve slope is smaller, preserving the original characteristics. The parameters of the mapping function are dynamically adjusted according to the region characteristics to ensure that the enhancement effect adapts to different regions. When introducing a feature transfer coefficient at the region boundary to adjust the enhancement intensity, the boundary width is set to 10 pixels. The enhancement intensity of the boundary region is a weighted average of the enhancement intensities of adjacent regions, with the weight determined by the feature transfer coefficient. When the feature transfer coefficient is 0.7, the enhancement intensity of the boundary region inherits 70% of the characteristics of the higher-enhanced region and 30% of the characteristics of the lower-enhanced region.

[0082] When reconstructing the enhanced sub-image regions, a weighted average method is used to fuse overlapping areas. The weights are inversely proportional to the distance from the pixel to the center of the region, ensuring a natural transition in the reconstructed image. The final output enhanced image has the same resolution as the input image, but with clearer details, more reasonable contrast, and adaptability to the characteristics of different regions.

[0083] The above technical solution introduces joint modeling of local entropy and gradient direction consistency to accurately characterize fine-grained texture and edge direction features in the image. It combines extreme point clustering and topological connectivity to generate region seed points, and achieves diffusion-style enhancement of information to region boundaries through feature transfer and attenuation control, effectively improving the contrast and structural clarity of the image in defective areas. Adaptive enhancement is achieved by constructing a nonlinear mapping function, avoiding the limitations of traditional image enhancement methods. Figure 1 The problem of knife cutting improves the sensitivity to identify defects with low contrast, small area or blurred boundaries, lays a high-quality data foundation for subsequent feature extraction and defect localization, and improves detection accuracy and robustness.

[0084] In one optional implementation, extracting structural features from the enhanced image data and establishing a feature transfer sequence includes:

[0085] A pyramid hierarchical structure is constructed for the enhanced image data. Overlapping sub-regions are divided in each layer. The local gradient distribution of the overlapping sub-regions is extracted, and the filter direction parameters are adjusted based on the local gradient distribution.

[0086] The structural features of the overlapping sub-regions are extracted using an adjusted filter. Corresponding mappings are established between the structural features in the overlapping regions of adjacent layers. The feature fusion order is determined based on the corresponding mappings. The structural features of each layer are combined according to the feature fusion order to obtain multi-level structural features.

[0087] A hierarchical transmission tree is constructed based on the feature fusion order. Feature transmission links are established in the hierarchical transmission tree. The feature transmission links include transmission nodes between adjacent features. The spatial structure distribution and directional consistency between adjacent transmission nodes are analyzed. The node transmission priority is determined based on the spatial structure distribution and directional consistency. A feature transmission sequence is generated based on the node transmission priority.

[0088] For example, when constructing a pyramid structure for enhanced image data, the original image is downsampled according to its size ratio, typically using a 1 / 2 scaling ratio, forming a pyramid structure that gradually increases in size from the top to the bottom. For instance, for a 1024×1024 pixel image, a pyramid structure with 5 layers can be constructed, where layer 0 is the original size of 1024×1024, layer 1 is 512×512, layer 2 is 256×256, layer 3 is 128×128, and layer 4 is 64×64. When dividing overlapping sub-regions within each layer, an overlap ratio of 30% to 50% is typically used. For example, for the 512×512 image in layer 1, it can be divided into multiple 256×256 sub-regions, with adjacent sub-regions overlapping by 128 pixels to ensure feature continuity and smooth transition.

[0089] When extracting the local gradient distribution of overlapping sub-regions, the gradient magnitude and direction of each pixel are calculated for each sub-region. Specifically, horizontal and vertical difference operators can be used to calculate the gradient of each pixel. For example, for a pixel at position (x, y), its horizontal gradient Gx and vertical gradient Gy are calculated, and then the gradient magnitude G and direction θ are synthesized. The gradient magnitude G is equal to the square root of the sum of the squares of the horizontal and vertical gradients, and the gradient direction θ is equal to the arctangent of the vertical and horizontal gradients. The distribution of gradient directions within each sub-region is statistically analyzed, typically by dividing 360 degrees into 8 or 16 directional intervals. The cumulative intensity of the gradient within each directional interval is calculated to obtain the gradient direction histogram for that sub-region.

[0090] When adjusting filter directional parameters based on local gradient distribution, the filter's directional sensitivity is adjusted according to the dominant directions in the gradient direction histogram. For example, if the gradient in a sub-region is mainly distributed in the 45-degree and 225-degree directions, the filter parameters are adjusted to be more sensitive to these directions. In practice, the center direction of the directional filter can be adjusted to θmax based on the direction angle θmax corresponding to the highest peak value in the gradient direction histogram, and the filter's directional bandwidth can be adjusted according to the dispersion of the histogram. When the gradient direction distribution is relatively concentrated, a narrower directional bandwidth (e.g., 15 degrees) is set; when the direction distribution is relatively dispersed, a wider directional bandwidth (e.g., 45 degrees) is set.

[0091] When extracting structural features from overlapping sub-regions using the adjusted filter, the adjusted directional filter is applied to the corresponding sub-region to extract structural information such as edges and textures. For example, for a sub-region with 45 degrees as the dominant direction, a directional filter with a center direction of 45 degrees and a bandwidth of 30 degrees is used for convolution to extract structural features in that direction. For each sub-region, structural features in multiple directions can be extracted to form a feature vector. In practical applications, features in 8 or 16 directions can be extracted, with each direction corresponding to a feature channel.

[0092] When establishing a mapping between structural features in overlapping regions of adjacent layers, scale variations between different layers need to be considered. For example, a 256×256 sub-region in layer 1 corresponds to a 512×512 region in layer 0; the mapping is established through positional correspondence. In practice, bilinear interpolation can be used to upsample features from smaller scales to larger scales, or downsample features from larger scales to smaller scales. Then, a similarity matrix between the two feature maps is calculated; cosine similarity or Euclidean distance can be used for similarity calculation. For feature point pairs with similarity exceeding a threshold (e.g., 0.7), a corresponding mapping relationship is established.

[0093] When determining the feature fusion order based on the corresponding mapping, the correlation strength between features at different levels is analyzed. Specifically, the average similarity of adjacent layer feature mapping pairs can be calculated; the higher the similarity, the higher the fusion priority. For example, if the average mapping similarity between layer 1 and layer 0 is 0.85, and the average mapping similarity between layer 2 and layer 1 is 0.76, then the feature fusion order is to fuse layer 1 and layer 0 first, and then fuse layer 2.

[0094] When combining structural features from each layer according to the feature fusion order to obtain multi-level structural features, a weighted fusion method is used. For example, if the determined fusion order is to fuse layer 1 and layer 0 first, with weights of 0.4 and 0.6 respectively, then the fused feature F01 is equal to 0.4 multiplied by layer 1 feature F1 plus 0.6 multiplied by layer 0 feature F0. Then, F01 is fused with layer 2 feature F2 with weights of 0.7 and 0.3 to obtain F012, and so on to complete the feature fusion of all levels.

[0095] When constructing a hierarchical transit tree based on the feature fusion order, the feature nodes of each level are connected according to the fusion order to form a tree structure. For example, if the fusion order is to fuse layers 0 and 1 first, then fuse layer 2, the root node of the tree is the fused F012, the lower-level nodes are F01 and F2, and the lower-level nodes are F0 and F1. When establishing feature transit links in the hierarchical transit tree, feature nodes of adjacent levels are connected by edges to form a transit path.

[0096] When analyzing the spatial structure distribution and directional consistency between adjacent transmission nodes, the spatial distribution similarity and directional consistency scores of node features are calculated. Spatial distribution similarity can be measured by the absolute value of the difference in spatial distribution entropy between feature maps; the closer the entropy values, the more similar the distributions. Directional consistency is measured by calculating the angle between the main directions of two nodes; the smaller the angle, the higher the consistency. Node transmission priority is determined based on spatial distribution similarity and directional consistency, using a weighted sum method; for example, the priority score equals 0.6 multiplied by the spatial distribution similarity plus 0.4 multiplied by the directional consistency score.

[0097] When generating a feature transfer sequence based on node transfer priority, all transfer nodes are sorted from highest to lowest priority to form the transfer sequence. For example, if the priority score of node pair (F0, F1) is 0.92 and the priority score of node pair (F01, F2) is 0.85, then the transfer sequence is to transfer F0 to F1 first, and then transfer F01 to F2. The final generated feature transfer sequence can guide the effective fusion and transfer of multi-level features, improving the accuracy and robustness of image structural feature extraction.

[0098] In this embodiment, by constructing a pyramid-shaped hierarchical structure and introducing overlapping sub-region division, and dynamically adjusting the filter direction parameters in conjunction with local gradient distribution, the structural feature extraction becomes more directionally adaptable and multi-scale perceptual. By mapping and fusing structural features of adjacent levels, multi-level structural features are constructed, enhancing the ability to express complex textures and subtle deformations. The introduction of a hierarchical transfer tree and spatial structural distribution and directional consistency analysis between nodes ensures that the feature transfer process has structural continuity and semantic relevance, thereby generating a feature transfer sequence with hierarchical and priority control, providing stable and high-quality structural information support for subsequent feature fusion and defect region identification.

[0099] In one optional implementation, a correlation matrix between features is calculated based on the feature transfer sequence, a feature optimization path is established based on the correlation matrix, and features are progressively fused along the feature optimization path to obtain fused feature data, including:

[0100] Calculate the feature association strength between adjacent transmission nodes in the feature transmission sequence, and transmit the feature association strength step by step in the feature transmission sequence to construct the association degree matrix;

[0101] In the correlation matrix, an initial transmission node is determined. Based on the feature correlation strength, the next transmission node is expanded step by step from the initial transmission node to establish transmission connections between nodes. The transmission connections are connected in series in order of feature correlation strength from high to low to form a feature optimization path.

[0102] Along the feature optimization path, starting from the initial transmission node, the features of the current transmission node are weighted and fused with the features of the next transmission node. The weight of the weighted fusion is determined by the feature association strength of the corresponding transmission connection. The fused feature is used as the new current node feature and continues to be fused with the features of the next transmission node. By progressively fusing the features of all transmission nodes, the final fused feature data is obtained.

[0103] For example, when calculating the feature association strength between adjacent transit nodes in a feature transit sequence, the feature association strength for any two adjacent nodes is determined by calculating the similarity of their feature vectors. The similarity calculation uses a cosine distance metric, taking the dot product of the two feature vectors divided by the product of their respective magnitudes. When the extracted features of the two nodes are both 128-dimensional vectors, if the angle between the two vectors is 30 degrees, the calculated association strength is approximately 0.866; if the angle is 60 degrees, the association strength is approximately 0.5. The association strength threshold is typically set to 0.75; node pairs below this threshold are considered weakly associated.

[0104] The progressive transmission of feature association strength in the feature transmission sequence employs a cumulative attenuation mechanism. Starting from the initial node, the association strength decreases by a certain percentage each time it passes through a node in the transmission sequence. The attenuation coefficient is dynamically adjusted based on the distance between nodes. Typically, the attenuation coefficient is 0.9 for adjacent nodes with a distance of 1, 0.8 for nodes with a distance of 2, and 0.7 for nodes with a distance of 3. Specifically, if the association strength between nodes A and B is 0.85, and the association strength between B and C is 0.8, then the association strength transmitted from A to C through B is 0.85 × 0.8 × 0.9 = 0.612.

[0105] When constructing the correlation matrix, the direct correlation strength and indirect transmission correlation strength are calculated for each pair of nodes in the feature transmission sequence, and the maximum of the two is taken as the final correlation strength. Each element in the matrix represents the correlation strength between different nodes, forming a symmetric matrix. A sequence containing 5 transmission nodes may generate the following correlation matrices: 0.89 for the first and second nodes, 0.76 for the second and third nodes, 0.82 for the third and fourth nodes, and 0.65 for the fourth and fifth nodes; 0.68 for the first and third nodes, 0.62 for the second and fourth nodes, and 0.53 for the third and fifth nodes; 0.56 for the first and fourth nodes, 0.48 for the second and fifth nodes; and 0.36 for the first and fifth nodes.

[0106] When determining the initial transmission node in the correlation matrix, the node with the highest average correlation strength is selected. Specifically, the elements in each row of the matrix are summed, and the node corresponding to the row with the largest sum is selected. If the sum of the correlation strengths of the second node and all other nodes is 2.75, which is higher than the sum of the correlation strengths of all other nodes, then the second node is selected as the initial transmission node.

[0107] When expanding from the initial node to the next node based on feature association strength, a greedy strategy is used to select the unvisited node with the highest association strength with the current node. If the initial node is the second node, and its association strengths with the first, third, fourth, and fifth nodes are 0.89, 0.76, 0.62, and 0.48 respectively, then the first node is selected as the next node. Each time a new node is selected, the set of visited nodes is updated, and the process of selecting the unvisited node with the highest association strength continues until all nodes have been visited.

[0108] When establishing transitive connections between nodes, the selected node sequence is converted into directed connections. If the selected node sequence is [second, first, third, fourth, fifth], then the established transitive connections are second to first, first to third, third to fourth, and fourth to fifth, with each connection marked with its corresponding association strength. The transitive connections are concatenated in descending order of feature association strength to form a feature optimization path.

[0109] When performing weighted fusion along the feature optimization path, fusion begins from the initial transit node and proceeds progressively with the next node. The fusion weight is determined by the feature correlation strength of the corresponding transit connection. If the correlation strength between the second node and the first node is 0.89, the fusion weight is set to 0.89 to 0.11, meaning the second node's features account for 89% and the first node's features account for 11%. The fusion operation uses a weighted average; for each feature dimension, the fused value is equal to the weighted average of the feature values ​​of the two nodes.

[0110] In practice, for two 128-dimensional feature vectors with a correlation strength of 0.89, the fusion result is calculated as follows: Fusion Feature = 0.89 × Second Node Feature + 0.11 × First Node Feature. The fused feature is then used as the new current node feature and fused with the feature of the next passing node, the third node. If the correlation strength between the fused feature and the third node is 0.76, the fusion weight is 0.76 to 0.24, and the fusion result = 0.76 × Fusion Feature + 0.24 × Third Node Feature.

[0111] The progressive fusion process includes the following steps: First, the feature of the second node of the initial transfer node is taken as the current feature; the correlation strength between the current feature and the feature of the first node of the next transfer node is calculated, which is 0.89; based on the correlation strength, a fusion weight is set, and a weighted average is performed on the current feature and the feature of the next node to obtain a new current feature; the correlation strength between the new current feature and the feature of the third node of the next transfer node is calculated, which is 0.76; the fusion weight is set again based on the correlation strength, a weighted average is performed, and the current feature is updated; this process is repeated until all transfer nodes participate in the fusion, resulting in the final fused feature data. Through this progressive fusion method, the features of all nodes are gradually fused along the feature optimization path, ultimately obtaining fused feature data containing complete information. The final fused feature can effectively express the features of various types of defects on the product surface. The dimensions of the fused feature are the same as the original feature, but it contains comprehensive information across multiple scales and directions, significantly improving the accuracy of defect detection and classification.

[0112] In this embodiment, by quantifying the feature association strength between adjacent nodes in the feature transfer sequence, a correlation degree matrix that accurately reflects the similarity and dependency between features is constructed, avoiding the problems of disordered or equal-weighted superposition in traditional feature fusion. Based on the correlation degree matrix, an ordered and priority-defined feature optimization path is constructed, so that the feature fusion process follows the principle of strong association priority, realizing a progressive fusion strategy from local to global and from shallow to deep. By dynamically weighting the features of each transfer node, key structural information is retained and redundant interference is suppressed, improving the discriminative power and stability of the fused features, and providing higher quality feature representation for defect identification and classification.

[0113] In one optional implementation, a probability distribution map of defect regions is generated based on the fused feature data, and the probability distribution map of defect regions is corrected by incorporating region enhancement weights to determine that the defect regions include:

[0114] Calculate local statistics on the fused feature data, construct an adaptive kernel function based on the local statistics, decompose the adaptive kernel function using orthogonal transformation to obtain a basis function set, and convolve the basis function set with the fused feature data to obtain a feature response map;

[0115] Gradient diffusion is performed on the feature response map to obtain the density flow field. Gradient trajectories in the density flow field are extracted. Regional clustering features are calculated based on the convergence points and convergence directions of the gradient trajectories. A probability distribution map of defect areas is generated based on the regional clustering features.

[0116] The region enhancement weights are adaptively weighted and fused with the probability distribution map of the defect region to obtain a modified probability map. The boundary point set of the modified probability map is extracted, the multi-moment features of the boundary point set are calculated, a level set function is constructed on the modified probability map, and the evolution speed of the level set function is adaptively adjusted based on the multi-moment features. The defect region is obtained by segmenting the zero level plane of the level set function.

[0117] In implementing this invention, the local statistics of the fused feature data are first calculated, including calculating the mean, variance, skewness, and kurtosis of each local region in the feature space using a sliding window approach. In this embodiment, a 7×7 pixel local window is used with a sliding step size of 1, and the above statistics are calculated for the feature values ​​within each window. For example, for a certain region, the local statistical features may be obtained with a mean of 0.65, a variance of 0.23, a skewness of 0.08, and a kurtosis of 3.12.

[0118] An adaptive kernel function is constructed based on the calculated local statistics. This kernel function can adaptively adjust its shape and scale according to the local feature distribution. In this embodiment, the bandwidth matrix of the kernel function is determined by the covariance matrix of the local region. When the feature variance is large, the kernel function has stronger extensibility in that direction. For example, for regions with complex textures, the kernel function exhibits anisotropy, with its major axis aligned with the texture direction and its minor axis to major axis ratio of 1:3. An orthogonal transformation is used to decompose the adaptive kernel function, resulting in a set of basis functions. In this embodiment, singular value decomposition (SVD) is used to decompose the kernel function into five orthogonal basis functions, each capturing feature information at different directions and scales. The weights of these basis functions are 0.45, 0.25, 0.15, 0.10, and 0.05, representing their respective importance in feature representation.

[0119] The basis function set is convolved with the fused feature data to obtain the feature response map. In this embodiment, the convolution operation is accelerated using Fast Fourier Transform. The response map corresponding to each basis function is calculated, and the data is weighted and combined according to the basis function weights to form the final feature response map. For example, for a 512×512 pixel image, a feature response map of the corresponding size is generated. The response value of the defect area is significantly higher than that of the background area. The response value range of the typical defect area is [0.75, 0.95], while the response value range of the background area is [0.05, 0.30].

[0120] Gradient diffusion is performed on the feature response map to obtain the density flow field. In this embodiment, an anisotropic diffusion method is used, with a diffusion time of 10 iterations. The diffusion coefficient is adaptively adjusted according to the gradient magnitude; diffusion is weak in regions with large gradients and strong in regions with small gradients. For example, the gradient threshold is set to 0.2. When the gradient magnitude is less than the threshold, the diffusion coefficient is 0.8; otherwise, it is 0.2 × (1 - gradient magnitude). Gradient trajectories are extracted from the density flow field by randomly selecting starting points in the flow field and tracing along the gradient direction. In this embodiment, 1000 random starting points are selected, and the maximum number of tracing steps for each trajectory is 100, with a step size of 0.5 pixels. Tracing stops when the distance between trajectory points is less than 0.1 pixels or exceeds the image boundary.

[0121] Region clustering features are calculated based on the convergence points and convergence directions of gradient trajectories. In this embodiment, a density-based clustering algorithm is used to group convergence points less than 5 pixels apart into one cluster, calculating the center position, size, shape, and orientation of each cluster. For example, a typical defect may form 3-5 convergence regions, each containing 50-200 convergence points. A probability distribution map of the defect region is generated based on the region clustering features. For each pixel, its distance to the nearest convergence region and directional similarity are calculated and converted into a probability value. For example, the distance function uses a Gaussian decay model with a standard deviation of 15 pixels; when the distance from the point to the nearest convergence region is 0, the probability is 1, and when the distance is 30 pixels, the probability drops to 0.1.

[0122] A region enhancement weight map is constructed, which considers the edge information, texture complexity, and local contrast of the original image. In this embodiment, edge weights are extracted using the Canny operator, with thresholds set to 50 and 150; texture complexity is calculated using local entropy with a window size of 9×9; and contrast is calculated by dividing the local standard deviation by the local mean. The final weights are the weighted sum of these three factors, with weights of 0.4, 0.35, and 0.25, respectively.

[0123] The region enhancement weights and the defect region probability distribution map are adaptively weighted and fused to obtain a corrected probability map. In this embodiment, the fusion formula is: Corrected probability = Original probability × (1 + α × Enhancement weight), where α is an adaptive coefficient, determined statistically based on the global probability distribution, with a typical value of 0.4. Boundary point sets are extracted from the corrected probability map and binarized using a probability threshold (e.g., 0.65). The multi-moment features of the boundary point sets are calculated, including centroid, principal axis direction, and eccentricity. For example, for a typical crack defect, its second-order central moment might show a principal axis direction of 32 degrees and a major-to-minor axis ratio of 5:1. A level set function is constructed for the corrected probability map, with the initial level set function being the corrected probability map minus a threshold (e.g., 0.5). The evolution speed of the level set function is adaptively adjusted based on the multi-moment features, with a faster evolution speed along the principal axis direction and a slower speed perpendicular to the principal axis direction. In this embodiment, the speed coefficient along the principal axis direction is 1.5, and the speed coefficient in the perpendicular direction is 0.8. The final defect region is obtained by segmenting the level set function using the zero-level plane. The level set evolution employs an iterative approach, with a maximum of 100 iterations. It stops when the boundary change is less than 0.5 pixels for five consecutive iterations. For example, for a defect with an area of ​​approximately 500 square pixels, it converges after about 35 iterations, yielding an accurate defect region contour.

[0124] The above technical solution improves the sensitivity of the feature response map to defects of different scales and shapes by calculating local statistics and constructing an adaptive kernel function from the fused feature data; it introduces gradient diffusion to generate a density flow field and extract gradient trajectories, which can effectively identify the spatial clustering trend of defect features and improve the accuracy of defect region localization; it combines region enhancement weights for weighted correction to achieve adaptive enhancement of defects with blurred boundaries or low contrast; and it guides the evolution process of the level set function through multi-order moment features, making the defect segmentation boundary more refined and smooth, ultimately achieving high-precision extraction of defect regions and enhancing the robustness and segmentation accuracy of detection.

[0125] Figure 2 This is a schematic diagram comparing the performance of different defect detection methods according to embodiments of the present invention, such as... Figure 2 As shown in the figure, this graph illustrates the detection accuracy of three different defect detection algorithms (adaptive kernel function method, traditional convolutional neural network, and level set segmentation method) in four different defect types (crack detection, surface depression, texture anomaly, and complex background). The data clearly shows that the adaptive kernel function method performs best in all test scenarios, with an accuracy ranging from 85.2% to 92.7%; the traditional convolutional neural network is second best, while the level set segmentation method performs weaker overall. Particularly under complex background conditions, the accuracy of the adaptive kernel function method (85.2%) is significantly higher than that of the traditional method (71.8% and 68.5%), demonstrating the algorithm's advantage in handling complex scenes.

[0126] In one optional implementation, the topological features of the defective region are extracted to establish a defect feature description. Based on the defect feature description, defects are classified, and the defect type and confidence score are output, including:

[0127] An adaptive threshold boundary tracking algorithm is used to extract the closed contour of the defect region. The closed contour is then denoised and smoothed to obtain a continuous boundary. A contour point sequence is established, and a distance transformation field is constructed based on the contour point sequence. The center line is extracted from the distance transformation field using the gradient descent method, and a center point sequence is constructed.

[0128] The contour point sequence and the center point sequence are mapped to the same coordinate space to construct the topology of the defect region. The branch nodes and intersections in the topology are extracted, the connection relationship between adjacent nodes is calculated, a topology tree is constructed based on the connection relationship, and the structural features of the topology tree are extracted as the topology features of the defect region.

[0129] Normalize each feature component of the topological structure feature, calculate the correlation coefficient between the normalized feature components, construct a feature correlation matrix, calculate the discrimination ability of each feature component based on the feature correlation matrix, determine the feature weight according to the discrimination ability, and use the feature weight to weight and combine the feature components to establish a defect feature description.

[0130] The defect feature descriptions are matched with preset multi-type defect samples, the distance and similarity between the feature descriptions are calculated, a feature matching matrix is ​​constructed, the matching degree between the test sample and each type of defect sample in the feature matching matrix is ​​analyzed, the type with the highest matching degree is determined as the defect type, and its corresponding matching degree is used as the confidence score.

[0131] For example, an adaptive threshold boundary tracking algorithm is used when extracting the closed contour of the defect region. This algorithm first performs grayscale processing on the image, determining the threshold by calculating the mean and standard deviation of the grayscale values ​​of local regions. The threshold is set to the mean minus 0.5 times the standard deviation. For example, for an 8-bit grayscale image, if the mean grayscale value of a certain region is 120 and the standard deviation is 40, then the threshold for that region is 120 - 0.5 × 40 = 100. Based on this threshold, the image is binarized to obtain the initial boundary of the defect region. The boundary tracking process uses the 8-connectivity criterion, starting from the first boundary point in the upper left corner, exploring adjacent points in a clockwise direction, recording the coordinates of all boundary points, until returning to the starting point to complete the extraction of the closed contour.

[0132] When performing noise reduction and smoothing on the extracted closed contours, a sliding window averaging method is used, with the window size set to 7 pixels. For each contour point, the coordinates of the three points before and after it are taken, and the average of these seven points is calculated as the new position of the current point. For example, if the original contour point coordinates are [(10, 15), (12, 15), (14, 16), (16, 17), (18, 18), (20, 20), (22, 21)], after smoothing, the new coordinates of the middle point (16, 17) are (16, 17.4). After processing all points, a smooth and continuous boundary and contour point sequence is obtained.

[0133] When constructing the distance transformation field, the shortest distance from each point in the image to the boundary is calculated. Specifically, a two-dimensional array is used to store the distance values, initializing the distance to boundary points to 0 and the distance to non-boundary points to infinity. The distance value of each point is calculated through two scans (forward and backward). The forward scan goes from the top left corner to the bottom right corner, and the backward scan goes from the bottom right corner to the top left corner. Each scan updates the point's distance value to the minimum of the current value and the distance to its adjacent point plus 1. For example, if a point's current distance is 5 and its adjacent point above it is 3, then the distance to that point is updated to min(5, 3+1) = 4.

[0134] When extracting the centerline from a distance-transformed field using gradient descent, the process starts from a local maximum point in the distance field and iteratively moves along the gradient direction. For each point, the gradient of the distance values ​​of its 8 neighboring points is calculated, and the direction with the largest gradient is chosen to move forward until a local extremum or a visited centerline point is reached. For example, if the 8 neighboring distance values ​​of a point are [5, 6, 5, 4, 3, 4, 5, 6], then the gradient direction is towards the second value (6). During the iteration process, the coordinates of all visited points are recorded, forming a sequence of centerlines. When multiple starting points exist, the connections between the sequences are analyzed, and they are merged into a complete centerline.

[0135] When mapping the contour point sequence and the center point sequence to the same coordinate space, the original image coordinate system is kept unchanged, and the points of the two sequences are merged. The merged sequence forms a topological representation of the defect region. When extracting branch nodes and intersections from the topological structure, the number of centerline points in the neighborhood of each point on the centerline is calculated. The neighborhood radius is set to 3 pixels. If there are 3 or more centerline segments in different directions within the neighborhood, the point is an intersection; if there are more than 2 centerline points but only 2 different directions, it is a branch node. For example, if a point has 5 centerline points in its neighborhood, each coming from a line segment in 3 different directions, then that point is marked as an intersection.

[0136] When calculating the connections between adjacent nodes, start from each node and trace along the centerline until reaching another node, recording the distance, angle, and centerline width changes between the two nodes. For example, the distance between nodes A and B is 30 pixels, the average width is 5 pixels, and the standard deviation of the width is 0.8 pixels. Based on these connections, a topology tree is constructed, where each node represents a branch node or intersection, and edges represent the connections between nodes.

[0137] When extracting the structural features of a topology tree, metrics such as the number of nodes, the number of leaf nodes, the tree depth, and the branch density are calculated. For example, a defective topology tree has 15 nodes, including 8 leaf nodes, a tree depth of 4, and a branch density of 0.6. These metrics constitute the topological feature vector of the defective region.

[0138] The topological features are normalized using a maximum-minimum normalization method. For example, the minimum number of nodes in a sample is 5, and the maximum is 25. If a sample has 15 nodes, the normalized value is (15-5) / (25-5) = 0.5. A similar process is applied to all feature components to obtain normalized feature vectors. When calculating the correlation coefficients between feature components, the Pearson correlation coefficient is used between each pair of features. For example, the correlation coefficient between the number of nodes and the number of leaf nodes is 0.85, indicating a high correlation. All correlation coefficients form a feature association matrix. Based on the feature association matrix, the discriminative power of each feature component is calculated using the information gain method. For each feature, its information gain value in the classification task is calculated; a higher value indicates stronger discriminative power. For example, the information gain of the branch density feature is 0.75, indicating that this feature has strong discriminative power.

[0139] The information gain values ​​of each feature are normalized and used as the weights of the corresponding features. For example, in a defect classification task, the topological features include four features: number of nodes, number of leaf nodes, tree depth, and branch density, with weights of 0.2, 0.15, 0.25, and 0.4, respectively. These weights are used to weight and combine the feature components to form a defect feature description. The defect feature description is then matched with pre-defined multi-type defect samples, calculating the Euclidean distance and cosine similarity between the test sample and the feature description of each type of defect sample. For example, the Euclidean distance between the test sample and type A defects is 0.25, and the cosine similarity is 0.92; the Euclidean distance with type B defects is 0.47, and the cosine similarity is 0.81. These distance and similarity values ​​form the feature matching matrix.

[0140] When analyzing the matching degree between the test sample and defect samples of each type in the feature matching matrix, distance and similarity are considered together to calculate a comprehensive matching score. For example, the comprehensive score for type A defects is 0.85, and for type B defects it is 0.72. The type with the highest score is selected as the defect type, and its corresponding matching score is used as the confidence score. In the above example, the system will output the recognition result as "Defect type: Type A; Confidence: 0.85".

[0141] Based on the above technical solutions, structured modeling and accurate classification of complex defect regions can be achieved. By extracting closed contours and centerlines to construct a topological structure, the geometric features and spatial connectivity of the defect region are preserved, improving the comprehensiveness and separability of defect representation. Extracting key node relationships and quantifying their features based on the topological tree structure helps distinguish defects with similar shapes but significant structural differences. Introducing correlation analysis and weighted combination among feature components constructs a defect feature description with high discriminative power, improving the accuracy and robustness of classification in feature matching. The final output defect type and confidence score support intelligent and interpretable defect identification and subsequent quality control.

[0142] In one optional implementation, a dynamic decision matrix is ​​constructed based on region enhancement weights and confidence scores. A comprehensive score is calculated for each defect region within the dynamic decision matrix. Defects are then classified according to the comprehensive score, generating a quantitative index of defect severity. The defect detection results are then output, including:

[0143] The uniformity of the distribution of the region enhancement weights and the fluctuation range of the confidence scores are calculated. The uniformity of the distribution and the fluctuation range are used as scoring weight coefficients to perform weighted fusion of the region enhancement weights and the confidence scores to generate a scoring matrix.

[0144] The variation pattern of the region enhancement weight with location is extracted to obtain the region variation feature, and the stability of the confidence score is extracted to obtain the reliability feature. The dimension weights of the scoring matrix are calculated based on the region variation feature and the reliability feature, and a dynamic decision matrix is ​​constructed based on the dimension weights.

[0145] The relative importance between dimensions is calculated in the dynamic decision matrix to generate an importance weight vector. The importance weight vector is then multiplied by the scoring matrix to obtain a comprehensive score for each defect area.

[0146] A graded evaluation standard is constructed based on the comprehensive score. The severity distribution is calculated based on the graded evaluation standard. The severity distribution is transformed into a severity quantification index using a membership function. The defect level is classified according to the severity quantification index and the detection results are output.

[0147] like Figure 3The diagram shown illustrates the intelligent defect detection and comprehensive rating process of this embodiment.

[0148] In this embodiment, when calculating the uniformity of the distribution of enhancement weights in a region, it is necessary to analyze the distribution of enhancement weights within the defect region. Distribution uniformity is represented by the coefficient of variation (COP) of the enhancement weights within the region, which is the standard deviation divided by the mean. For a detected electronic component solder joint defect region, if the mean enhancement weight is 0.75 and the standard deviation is 0.15, the COP is 0.2, indicating a relatively uniform distribution; if the standard deviation is 0.3 and the COP is 0.4, it indicates an uneven distribution. Higher distribution uniformity corresponds to a larger scoring weight coefficient, typically ranging from 0.6 to 0.9.

[0149] The fluctuation range of the confidence score is determined by analyzing the difference between the maximum and minimum confidence scores of all pixels within the defect area. For a detected microcrack defect, if the maximum built-in confidence score of the area is 0.92 and the minimum is 0.78, the fluctuation range is 0.14; if the maximum is 0.95 and the minimum is 0.55, the fluctuation range is 0.4. The smaller the fluctuation range, the more stable the confidence score, and the larger the corresponding scoring weight coefficient, which typically ranges from 0.65 to 0.95.

[0150] Using uniformity of distribution and range of fluctuation as scoring weight coefficients, a weighted fusion of regional enhancement weights and confidence scores is generated to produce a scoring matrix. Specifically, for each defect region, the product of the uniformity of distribution weight and the regional enhancement weight is calculated, and then added to the product of the range of fluctuation weight and the confidence score to form the element values ​​of the scoring matrix. For a surface scratch defect on a product, if the uniformity of distribution weight is 0.85, the regional enhancement weight is 0.78, the range of fluctuation weight is 0.75, and the confidence score is 0.83, then the corresponding element value of the scoring matrix is ​​0.85 × 0.78 + 0.75 × 0.83 = 1.29.

[0151] When extracting the variation pattern of enhancement weights with location to obtain regional variation characteristics, gradient analysis is used to calculate the rate of change of enhancement weights in the horizontal and vertical directions. For a metal surface depression defect, if the enhancement weights in the horizontal direction change from left to right to 0.65, 0.72, 0.85, 0.79, and 0.68, the differences between adjacent positions can be calculated to be 0.07, 0.13, -0.06, and -0.11, respectively. The variation pattern shows an initial increase followed by a decrease, forming a peak characteristic. If a similar trend is observed in the vertical direction, the regional variation characteristic is considered "center-prominent," and a high dimensional weight of 0.8 is assigned. If the variation is monotonically changed from one side to the other, the regional variation characteristic is considered "gradual," and a medium dimensional weight of 0.6 is assigned.

[0152] When extracting the stability of confidence scores to obtain reliability features, the local consistency and cross-regional consistency of the confidence scores are analyzed. Local consistency refers to the difference in confidence scores between adjacent pixels, usually measured by local variance; cross-regional consistency refers to the difference in confidence scores between different sub-regions, usually measured by inter-regional variance. If the local variance is 0.02 and the inter-regional variance is 0.05, the reliability feature value is 0.93, indicating high confidence stability; if the local variance is 0.08 and the inter-regional variance is 0.12, the reliability feature value is 0.8, indicating moderate confidence stability.

[0153] The dimensional weights of the scoring matrix are calculated based on regional change characteristics and reliability characteristics, and a dynamic decision matrix is ​​constructed according to these weights. A weighted average method is used to calculate the dimensional weights, with the ratio of regional change characteristic weights to reliability characteristic weights typically being 7:3. If the regional change characteristic weight is 0.75 and the reliability characteristic weight is 0.88, then the dimensional weight is 0.75 × 0.7 + 0.88 × 0.3 = 0.789. The dynamic decision matrix is ​​constructed by multiplying each element of the scoring matrix by its corresponding dimensional weight.

[0154] The relative importance between dimensions is calculated in the dynamic decision matrix to generate an importance weight vector. The relative importance between dimensions is determined based on a defect type feature library, with different importance weights assigned to different defect types. The importance of region enhancement weights is usually higher than the confidence score, and the importance weight vector may be [0.65, 0.35]. For solder joint defects on the surface of electronic components, the importance of the confidence score is usually higher than the region enhancement weight, and the importance weight vector may be [0.4, 0.6]. The importance weight vector is multiplied by the scoring matrix to obtain the comprehensive score for each defect region. For a small crack defect on a glass surface, if the corresponding element of the scoring matrix is ​​[1.25, 1.35] and the importance weight vector is [0.55, 0.45], then the comprehensive score is 1.25 × 0.55 + 1.35 × 0.45 = 1.29.

[0155] A grading assessment standard is constructed based on the comprehensive score, and the severity distribution is calculated according to the grading assessment standard. The grading assessment standard is usually determined based on historical data and expert experience. For example, for surface defects of automotive parts, a comprehensive score of less than 0.8 is a minor defect, 0.8 to 1.2 is a general defect, 1.2 to 1.6 is a serious defect, and greater than 1.6 is a fatal defect. The severity distribution is obtained by calculating the proportion of defective areas within each level range.

[0156] The severity distribution is transformed into a severity quantification index using a membership function. The membership function is a piecewise linear function that maps the comprehensive score to a severity quantification index range of 0 to 10. For example, for a pitting defect on the surface of a precision bearing, if the comprehensive score is 1.29, falling into the severe defect range, the severity quantification index calculated using the membership function is 7.2 points. Defect levels are classified according to the severity quantification index, and the detection results are output. For example, the defect level classification criteria are: 0 to 3 points for Level 1 defects, 3 to 6 points for Level 2 defects, 6 to 8 points for Level 3 defects, and 8 to 10 points for Level 4 defects. For a defect with a severity quantification index of 7.2 points, it is classified as a Level 3 defect, marked as a red warning in the detection results, and detailed information such as the defect area coordinates, area, and shape characteristics is provided to provide a basis for subsequent processing decisions.

[0157] Based on the above technical solution, quantitative evaluation and intelligent grading of defect detection results can be achieved. By integrating regional enhancement weights and confidence scores, a scoring matrix and a dynamic decision matrix are constructed to effectively characterize the importance and detection stability of defect regions. Furthermore, dimensional weights are introduced by combining regional variation characteristics and reliability characteristics to improve the accuracy and interpretability of the scoring results. Relative importance analysis and weighted fusion calculation are used to calculate the comprehensive score, enhancing the sensitivity to distinguish different types of defects. Finally, continuous quantitative expression of defect levels is achieved through severity distribution and membership functions, providing precise support for defect management and quality assessment.

[0158] A second aspect of this invention provides a real-time machine vision defect detection and classification system based on deep learning, the system comprising:

[0159] The first unit is used to acquire image data of the surface of industrial products, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the region enhancement weight based on the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the region enhancement weight to obtain the enhanced image data.

[0160] The second unit is used to extract the structural features of the enhanced image data, establish a feature transfer sequence, calculate the correlation matrix between features based on the feature transfer sequence, establish a feature optimization path based on the correlation matrix, and perform progressive fusion of features along the feature optimization path to obtain fused feature data.

[0161] The third unit is used to generate a probability distribution map of defect areas based on the fused feature data, correct the probability distribution map of defect areas by combining the region enhancement weight, determine the defect areas, extract the topological structure features of the defect areas, establish defect feature descriptions, classify defects based on the defect feature descriptions, and output defect types and confidence scores.

[0162] The fourth unit is used to construct a dynamic decision matrix based on regional enhancement weights and confidence scores. It calculates the comprehensive score for each defect region in the dynamic decision matrix, classifies the defects according to the comprehensive score, generates a quantitative index of defect severity, and outputs the defect detection results.

[0163] A third aspect of the present invention provides an electronic device, comprising:

[0164] processor;

[0165] Memory used to store processor-executable instructions;

[0166] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0167] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0168] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A machine vision defect real-time detection and classification method based on deep learning, characterized in that, include: Acquire image data of the surface of industrial products, and perform region segmentation on the image data along a preset direction sequence to obtain multiple region sub-images; Within each sub-image region, a correlation strength matrix between a pixel and its neighborhood is constructed. Based on the correlation strength matrix, the local entropy value of each pixel is calculated. The gradient vector of each pixel is extracted and projected in different directions. The gradient direction consistency of each pixel is calculated based on the projection components. The local entropy value and gradient direction consistency are combined to generate regional characteristic data. Extract the extreme points in the regional characteristic data where the local entropy value of each pixel is consistent with the gradient direction, cluster the extreme points that meet the preset extreme value range to obtain regional seed points, establish the topological connection relationship between regional seed points, calculate the characteristic transfer coefficient, and generate regional growth path; The local entropy value and gradient direction consistency of the region seed point are propagated outward along the region growth path. The attenuation factor is calculated based on the distance from the pixel to the region seed point. The product of the local entropy value, gradient direction consistency and attenuation factor is used as the region enhancement weight. Based on the region enhancement weight, the image data is subjected to regional adaptive enhancement to obtain the enhanced image data. Structural features of the enhanced image data are extracted, a feature transfer sequence is established, the correlation matrix between features is calculated based on the feature transfer sequence, a feature optimization path is established based on the correlation matrix, and features are progressively fused along the feature optimization path to obtain fused feature data. A probability distribution map of defect areas is generated based on the fused feature data. The probability distribution map of defect areas is then corrected by combining the region enhancement weights to identify defect areas. The topological features of the defect areas are extracted to establish defect feature descriptions. Defects are classified based on the defect feature descriptions, and the defect type and confidence score are output. A dynamic decision matrix is ​​constructed based on regional enhancement weights and confidence scores. A comprehensive score for each defect region is calculated in the dynamic decision matrix. Defects are classified according to the comprehensive score, a quantitative index of defect severity is generated, and the defect detection results are output.

2. The method according to claim 1, characterized in that, Image data is adaptively enhanced by region-based enhancement weights, resulting in enhanced image data including: The enhancement coefficient is obtained by weighting the regional characteristic data based on the regional enhancement weight. A nonlinear mapping function is constructed based on the enhancement coefficient to enhance each region sub-image; a characteristic transfer coefficient is introduced at the region boundary to adjust the enhancement intensity, and the enhanced region sub-images are recombined to obtain the enhanced image data.

3. The method according to claim 1, characterized in that, Extracting structural features from the enhanced image data and establishing a feature transfer sequence includes: A pyramid hierarchical structure is constructed for the enhanced image data. Overlapping sub-regions are divided in each layer. The local gradient distribution of the overlapping sub-regions is extracted, and the filter direction parameters are adjusted based on the local gradient distribution. The structural features of the overlapping sub-regions are extracted using an adjusted filter. Corresponding mappings are established between the structural features in the overlapping regions of adjacent layers. The feature fusion order is determined based on the corresponding mappings. The structural features of each layer are combined according to the feature fusion order to obtain multi-level structural features. A hierarchical transmission tree is constructed based on the feature fusion order. Feature transmission links are established in the hierarchical transmission tree. The feature transmission links include transmission nodes between adjacent features. The spatial structure distribution and directional consistency between adjacent transmission nodes are analyzed. The node transmission priority is determined based on the spatial structure distribution and directional consistency. A feature transmission sequence is generated based on the node transmission priority.

4. The method according to claim 1, characterized in that, The correlation matrix between features is calculated based on the feature transfer sequence. A feature optimization path is established based on the correlation matrix. Features are progressively fused along the feature optimization path to obtain fused feature data, including: Calculate the feature association strength between adjacent transmission nodes in the feature transmission sequence, and transmit the feature association strength step by step in the feature transmission sequence to construct the association degree matrix; In the correlation matrix, an initial transmission node is determined. Based on the feature correlation strength, the next transmission node is expanded step by step from the initial transmission node to establish transmission connections between nodes. The transmission connections are connected in series in order of feature correlation strength from high to low to form a feature optimization path. Along the feature optimization path, starting from the initial transmission node, the features of the current transmission node are weighted and fused with the features of the next transmission node. The weight of the weighted fusion is determined by the feature association strength of the corresponding transmission connection. The fused feature is used as the new current node feature and continues to be fused with the features of the next transmission node. By progressively fusing the features of all transmission nodes, the final fused feature data is obtained.

5. The method according to claim 1, characterized in that, A probability distribution map of defect areas is generated based on the fused feature data. This map is then corrected using region enhancement weights to determine the defect areas, which include: Calculate local statistics on the fused feature data, construct an adaptive kernel function based on the local statistics, decompose the adaptive kernel function using orthogonal transformation to obtain a basis function set, and convolve the basis function set with the fused feature data to obtain a feature response map; Gradient diffusion is performed on the feature response map to obtain the density flow field. Gradient trajectories in the density flow field are extracted. Regional clustering features are calculated based on the convergence points and convergence directions of the gradient trajectories. A probability distribution map of defect areas is generated based on the regional clustering features. The region enhancement weights are adaptively weighted and fused with the probability distribution map of the defect region to obtain a modified probability map. The boundary point set of the modified probability map is extracted, the multi-moment features of the boundary point set are calculated, a level set function is constructed on the modified probability map, and the evolution speed of the level set function is adaptively adjusted based on the multi-moment features. The defect region is obtained by segmenting the zero level plane of the level set function.

6. The method according to claim 1, characterized in that, Extract the topological features of the defect region, establish a defect feature description, classify defects based on the defect feature description, and output the defect type and confidence score, including: An adaptive threshold boundary tracking algorithm is used to extract the closed contour of the defect region. The closed contour is then denoised and smoothed to obtain a continuous boundary. A contour point sequence is established, and a distance transformation field is constructed based on the contour point sequence. The center line is extracted from the distance transformation field using the gradient descent method, and a center point sequence is constructed. The contour point sequence and the center point sequence are mapped to the same coordinate space to construct the topology of the defect region. The branch nodes and intersections in the topology are extracted, the connection relationship between adjacent nodes is calculated, a topology tree is constructed based on the connection relationship, and the structural features of the topology tree are extracted as the topology features of the defect region. Normalize each feature component of the topological structure feature, calculate the correlation coefficient between the normalized feature components, construct a feature correlation matrix, calculate the discrimination ability of each feature component based on the feature correlation matrix, determine the feature weight according to the discrimination ability, and use the feature weight to weight and combine the feature components to establish a defect feature description. The defect feature descriptions are matched with preset multi-type defect samples, the distance and similarity between the feature descriptions are calculated, a feature matching matrix is ​​constructed, the matching degree between the test sample and each type of defect sample in the feature matching matrix is ​​analyzed, the type with the highest matching degree is determined as the defect type, and its corresponding matching degree is used as the confidence score.

7. The method according to claim 1, characterized in that, A dynamic decision matrix is ​​constructed based on region enhancement weights and confidence scores. A comprehensive score is calculated for each defect region within this matrix. Defects are then classified according to their comprehensive scores, generating a quantitative index of defect severity. The defect detection results are then output, including: The uniformity of the distribution of the region enhancement weights and the fluctuation range of the confidence scores are calculated. The uniformity of the distribution and the fluctuation range are used as scoring weight coefficients to perform weighted fusion of the region enhancement weights and the confidence scores to generate a scoring matrix. The variation pattern of the region enhancement weight with location is extracted to obtain the region variation feature, and the stability of the confidence score is extracted to obtain the reliability feature. The dimension weights of the scoring matrix are calculated based on the region variation feature and the reliability feature, and a dynamic decision matrix is ​​constructed based on the dimension weights. The relative importance between dimensions is calculated in the dynamic decision matrix to generate an importance weight vector. The importance weight vector is then multiplied by the scoring matrix to obtain a comprehensive score for each defect area. A graded evaluation standard is constructed based on the comprehensive score. The severity distribution is calculated based on the graded evaluation standard. The severity distribution is transformed into a severity quantification index using a membership function. The defect level is classified according to the severity quantification index and the detection results are output.

8. A machine vision defect real-time detection and classification system based on deep learning, used to implement the method of any one of claims 1-7, characterized in that, include: The first unit is used to acquire image data of the surface of industrial products, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the region enhancement weight based on the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the region enhancement weight to obtain the enhanced image data. The second unit is used to extract the structural features of the enhanced image data, establish a feature transfer sequence, calculate the correlation matrix between features based on the feature transfer sequence, establish a feature optimization path based on the correlation matrix, and perform progressive fusion of features along the feature optimization path to obtain fused feature data. The third unit is used to generate a probability distribution map of defect areas based on the fused feature data, correct the probability distribution map of defect areas by combining the region enhancement weight, determine the defect areas, extract the topological structure features of the defect areas, establish defect feature descriptions, classify defects based on the defect feature descriptions, and output defect types and confidence scores. The fourth unit is used to construct a dynamic decision matrix based on regional enhancement weights and confidence scores. It calculates the comprehensive score for each defect region in the dynamic decision matrix, classifies the defects according to the comprehensive score, generates a quantitative index of defect severity, and outputs the defect detection results.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images