Machine vision defect real-time detection and classification method and system based on deep learning

Abstract

The invention provides a machine vision defect real-time detection and classification method and system based on deep learning, and relates to the field of machine vision detection.The method comprises the steps that regional enhancement weights are determined by calculating local entropy and gradient direction consistency, and regional self-adaptive enhancement is carried out; establishing a feature transfer sequence and progressively fusing features; generating and correcting a defect area probability distribution diagram; and constructing a dynamic decision matrix to calculate a comprehensive score for defect grading. According to the method, the defect detection accuracy under a complex background can be improved, false detection and missing detection are reduced, and real-time defect positioning and accurate classification are realized.

Description

Technical Field

[0001] The present invention relates to machine vision detection technology, and in particular to a method and system for real-time detection and classification of machine vision defects based on deep learning. Background Art

[0002] Surface defect detection for industrial products is a critical step in quality control. Traditional manual inspection methods suffer from low efficiency and poor consistency. With the development of deep learning technology, automated inspection methods based on machine vision have become a research hotspot. However, detection accuracy and real-time performance remain challenging in complex backgrounds and lighting conditions.

[0003] The existing deep learning detection methods mainly have the following problems: insufficient image feature extraction leads to low detection accuracy; inaccurate positioning of defect areas affects classification results; single defect assessment standards make it difficult to meet the quality control needs of different products; the detection process is computationally complex and difficult to achieve real-time requirements.

[0004] Currently, there is an urgent need for a detection method that can adaptively enhance image features, optimize feature fusion paths, accurately locate defect areas, and implement graded evaluation, so as to improve detection accuracy and real-time performance and realize intelligent detection and classification of surface defects of industrial products. Summary of the Invention

[0005] The embodiments of the present invention provide a method and system for real-time detection and classification of machine vision defects based on deep learning, which can solve the problems in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a method for real-time detection and classification of machine vision defects based on deep learning, comprising: Acquire image data of the surface of industrial products, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the regional enhancement weight based on the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the regional enhancement weight to obtain enhanced image data; Extract the structural features of the enhanced image data, establish a feature transfer sequence, calculate the correlation matrix between features based on the feature transfer sequence, establish a feature optimization path based on the correlation matrix, and perform progressive fusion of features along the feature optimization path to obtain fused feature data; Generate a probability distribution map of the defect area based on the fused feature data, modify the probability distribution map of the defect area based on the regional enhancement weight, determine the defect area, extract the topological structure features of the defect area, establish a defect feature description, classify the defects based on the defect feature description, and output the defect type and confidence score; A dynamic decision matrix is ​​constructed based on the regional enhancement weights and confidence scores. The comprehensive score of each defect area is calculated in the dynamic decision matrix. The defects are graded according to the comprehensive score, a quantitative indicator of the defect severity is generated, and the defect detection results are output.

[0007] In an optional embodiment, Acquire image data of the surface of industrial products, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the regional enhancement weight based on the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the regional enhancement weight. The enhanced image data includes: Acquire image data of the surface of an industrial product, and perform regional segmentation on the image data along a preset direction sequence to obtain multiple regional sub-images; Within each regional sub-image, a correlation strength matrix is ​​constructed between the pixel and its neighborhood. The local entropy value of each pixel is calculated based on the correlation strength matrix. The gradient vector of each pixel is extracted and projected in different directions. The gradient direction consistency of each pixel is calculated based on the projected component. The local entropy value and gradient direction consistency are combined to generate regional characteristic data. Extract the extreme points of local entropy and gradient direction consistency of each pixel in the regional characteristic data, cluster the extreme points that meet the preset extreme value range to obtain regional seed points, establish the topological connection relationship between the regional seed points, calculate the characteristic transfer coefficient, and generate the regional growth path; The local entropy value and gradient direction consistency of the regional seed point are transferred outward along the regional growth path. The attenuation factor is calculated according to the distance from the pixel point to the regional seed point. The product of the local entropy value, gradient direction consistency and attenuation factor is used as the regional enhancement weight. The regional characteristic data is weighted based on the regional enhancement weight to obtain the enhancement coefficient; A nonlinear mapping function is constructed according to the enhancement coefficient to enhance each regional sub-image; a characteristic transfer coefficient is introduced at the regional boundary to adjust the enhancement strength, and the enhanced regional sub-images are recombined to obtain the enhanced image data.

[0008] In an optional embodiment, Extracting the structural features of the enhanced image data and establishing a feature transfer sequence include: Constructing a pyramid hierarchical structure for the enhanced image data, dividing overlapping sub-regions in each layer of the structure, extracting local gradient distributions of the overlapping sub-regions, and adjusting filter direction parameters based on the local gradient distributions; The adjusted filter is used to extract the structural features of the overlapping sub-regions, a corresponding mapping is established for the structural features in the overlapping regions of adjacent layers, a feature fusion order is determined according to the corresponding mapping, and the structural features of each layer are combined according to the feature fusion order to obtain a multi-level structural feature; A hierarchical transfer tree is constructed based on the feature fusion order, and a feature transfer link is established in the hierarchical transfer tree. The feature transfer link includes transfer nodes between adjacent features. The spatial structure distribution and directional consistency between adjacent transfer nodes are analyzed, and the node transfer priority is determined according to the spatial structure distribution and directional consistency. A feature transfer sequence is generated based on the node transfer priority.

[0009] In an optional embodiment, According to the feature transfer sequence, the correlation matrix between features is calculated, and a feature optimization path is established based on the correlation matrix. The features are progressively fused along the feature optimization path to obtain the fused feature data including: Calculating feature correlation strengths between adjacent transfer nodes in a feature transfer sequence, transferring the feature correlation strengths step by step in the feature transfer sequence, and constructing a correlation matrix; Determining an initial transfer node in the association matrix, expanding the next transfer node step by step from the initial transfer node based on the feature association strength, establishing transfer connections between nodes, and connecting the transfer connections in series in descending order of feature association strength to form a feature optimization path; Along the feature optimization path, starting from the initial transfer node, the features of the current transfer node are weightedly fused with the features of the next transfer node. The weight of the weighted fusion is determined by the feature correlation strength of the corresponding transfer connection. The fused features are used as the new current node features and continue to be fused with the features of the next transfer node. The fusion of all transfer node features is completed step by step to obtain the final fused feature data.

[0010] In an optional embodiment, Generate a probability distribution map of defect areas based on the fused feature data, and modify the probability distribution map of defect areas by combining the regional enhancement weights to determine the defect areas, including: Calculating local statistics for the fused feature data, constructing an adaptive kernel function based on the local statistics, decomposing the adaptive kernel function using an orthogonal transformation to obtain a basis function group, and convolving the basis function group with the fused feature data to obtain a feature response map; Performing gradient diffusion on the characteristic response map to obtain a density flow field, extracting gradient trajectories in the density flow field, calculating regional clustering features based on convergence points and convergence directions of the gradient trajectories, and generating a defect region probability distribution map based on the regional clustering features; The regional enhancement weight is adaptively weighted fused with the defect area probability distribution map to obtain a corrected probability map, the boundary point set of the corrected probability map is extracted, the multi-order moment features of the boundary point set are calculated, and a level set function is constructed for the corrected probability map. The evolution speed of the level set function is adaptively adjusted based on the multi-order moment features, and the defect area is obtained by segmentation using the zero level plane of the level set function.

[0011] In an optional embodiment, Extract the topological structure features of the defect area, establish the defect feature description, classify the defects according to the defect feature description, and output the defect type and confidence score including: A closed contour of the defect area is extracted using a boundary tracking algorithm with an adaptive threshold, and the closed contour is subjected to noise reduction and smoothing to obtain a continuous boundary. A contour point sequence is established, and a distance transform field is constructed based on the contour point sequence. A center line is extracted from the distance transform field using a gradient descent method to construct a center point sequence. Mapping the contour point sequence and the center point sequence to the same coordinate space to construct a topological structure of the defect area, extracting branch nodes and intersections in the topological structure, calculating the connection relationship between adjacent nodes, constructing a topological tree based on the connection relationship, and extracting structural features of the topological tree as topological structure features of the defect area; Normalizing each feature component of the topological structure feature, calculating the correlation coefficient between the normalized feature components, constructing a feature correlation matrix, calculating the distinguishing ability of each feature component based on the feature correlation matrix, determining a feature weight based on the distinguishing ability, and using the feature weight to perform a weighted combination of the feature components to establish a defect feature description; The defect feature description is matched with preset multi-type defect samples, the distance and similarity between the feature descriptions are calculated, a feature matching matrix is ​​constructed, the matching degree between the sample to be tested and each type of defect sample in the feature matching matrix is ​​analyzed, the type with the highest matching degree is determined as the defect type, and its corresponding matching degree is used as the confidence score.

[0012] In an optional embodiment, A dynamic decision matrix is ​​constructed based on the regional enhancement weights and confidence scores. The comprehensive score of each defect region is calculated in the dynamic decision matrix. The defects are graded according to the comprehensive score, and a quantitative indicator of the defect severity is generated. The defect detection results are output, including: Calculating the distribution uniformity of the regional enhancement weight and the fluctuation range of the confidence score, using the distribution uniformity and fluctuation range as scoring weight coefficients, performing weighted fusion on the regional enhancement weight and the confidence score to generate a scoring matrix; Extracting the variation pattern of the regional enhancement weight with position to obtain regional variation characteristics, extracting the stability of the confidence score to obtain reliability characteristics, calculating the dimension weights of the scoring matrix based on the regional variation characteristics and the reliability characteristics, and constructing a dynamic decision matrix based on the dimension weights; Calculating the relative importance of the dimensions in the dynamic decision matrix to generate an importance weight vector, and multiplying the importance weight vector by the scoring matrix to obtain a comprehensive score for each defect area; A hierarchical evaluation standard is constructed for the comprehensive score, a severity distribution is calculated according to the hierarchical evaluation standard, a membership function is used to convert the severity distribution into a severity quantification index, defect levels are divided according to the severity quantification index, and a detection result is output.

[0013] A second aspect of an embodiment of the present invention provides a real-time machine vision defect detection and classification system based on deep learning, comprising: The first unit is used to obtain image data of the surface of the industrial product, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the regional enhancement weight according to the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the regional enhancement weight to obtain enhanced image data; The second unit is used to extract the structural features of the enhanced image data, establish a feature transfer sequence, calculate the correlation matrix between features based on the feature transfer sequence, establish a feature optimization path based on the correlation matrix, and progressively fuse the features along the feature optimization path to obtain fused feature data; The third unit is used to generate a probability distribution map of the defect area based on the fused feature data, modify the probability distribution map of the defect area based on the regional enhancement weight, determine the defect area, extract the topological structure features of the defect area, establish a defect feature description, classify the defects based on the defect feature description, and output the defect type and confidence score; The fourth unit is used to construct a dynamic decision matrix based on the regional enhancement weights and confidence scores, calculate the comprehensive score of each defect area in the dynamic decision matrix, grade the defects according to the comprehensive score, generate a quantitative indicator of the defect severity, and output the defect detection results.

[0014] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0015] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0016] In this embodiment, regional enhancement weights are calculated by using local entropy and gradient direction consistency to achieve regional adaptive enhancement, improve the contrast and clarity of defect areas, solve the problem that traditional methods are difficult to identify low-contrast and blurred defects, and significantly improve the accuracy and sensitivity of defect detection. A feature transfer sequence and a correlation matrix are established to achieve progressive fusion of features, effectively retain the multi-scale structural information of defects, and through topological structure feature extraction and defect feature description, accurate classification of different types of defects is achieved, improving the system's robustness to defect recognition in complex backgrounds. A dynamic decision matrix is ​​constructed and a comprehensive score of the defect area is calculated to achieve quantitative grading of defect severity, providing an accurate reference for product quality control. At the same time, the feature extraction and fusion process is optimized, the computational complexity is reduced, and the real-time performance of the system is guaranteed, making it suitable for online detection needs of high-speed industrial production lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of the process of real-time detection and classification of machine vision defects based on deep learning according to an embodiment of the present invention; Figure 2 Schematic diagram showing performance comparison of different defect detection methods according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the intelligent defect detection and comprehensive rating process of this embodiment. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0019] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0020] Figure 1 FIG is a flow chart of a method for real-time detection and classification of machine vision defects based on deep learning according to an embodiment of the present invention. Figure 1 As shown, the method includes: Acquire image data of the surface of industrial products, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the regional enhancement weight based on the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the regional enhancement weight to obtain enhanced image data; Extract the structural features of the enhanced image data, establish a feature transfer sequence, calculate the correlation matrix between features based on the feature transfer sequence, establish a feature optimization path based on the correlation matrix, and perform progressive fusion of features along the feature optimization path to obtain fused feature data; Generate a probability distribution map of the defect area based on the fused feature data, modify the probability distribution map of the defect area based on the regional enhancement weight, determine the defect area, extract the topological structure features of the defect area, establish a defect feature description, classify the defects based on the defect feature description, and output the defect type and confidence score; A dynamic decision matrix is ​​constructed based on the regional enhancement weights and confidence scores. The comprehensive score of each defect area is calculated in the dynamic decision matrix. The defects are graded according to the comprehensive score, a quantitative indicator of the defect severity is generated, and the defect detection results are output.

[0021] In an optional embodiment, image data of the surface of an industrial product is obtained, the local entropy value and gradient direction consistency of each pixel in the image data are calculated, a regional enhancement weight is determined based on the local entropy value and gradient direction consistency, and the image data is adaptively enhanced by region based on the regional enhancement weight. The enhanced image data includes: Acquire image data of the surface of an industrial product, and perform regional segmentation on the image data along a preset direction sequence to obtain multiple regional sub-images; Within each regional sub-image, a correlation strength matrix is ​​constructed between the pixel and its neighborhood. The local entropy value of each pixel is calculated based on the correlation strength matrix. The gradient vector of each pixel is extracted and projected in different directions. The gradient direction consistency of each pixel is calculated based on the projected component. The local entropy value and gradient direction consistency are combined to generate regional characteristic data. Extract the extreme points of local entropy and gradient direction consistency of each pixel in the regional characteristic data, cluster the extreme points that meet the preset extreme value range to obtain regional seed points, establish the topological connection relationship between the regional seed points, calculate the characteristic transfer coefficient, and generate the regional growth path; The local entropy value and gradient direction consistency of the regional seed point are transferred outward along the regional growth path. The attenuation factor is calculated according to the distance from the pixel point to the regional seed point. The product of the local entropy value, gradient direction consistency and attenuation factor is used as the regional enhancement weight. The regional characteristic data is weighted based on the regional enhancement weight to obtain the enhancement coefficient; A nonlinear mapping function is constructed according to the enhancement coefficient to enhance each regional sub-image; a characteristic transfer coefficient is introduced at the regional boundary to adjust the enhancement strength, and the enhanced regional sub-images are recombined to obtain the enhanced image data.

[0022] In this implementation, image data of industrial product surfaces is acquired using a high-resolution industrial camera. During acquisition, the camera is positioned perpendicular to the product surface to maintain uniform lighting and avoid strong light reflections and shadows. The image capture resolution is set to 2048 × 1536 pixels, and the image is saved in 8-bit grayscale format. Image preprocessing includes denoising and contrast correction to ensure pristine image quality.

[0023] The acquired image data is segmented along a sequence of preset horizontal, vertical, and diagonal directions. A multi-scale approach is employed, with sliding windows of varying sizes set in each direction: 16×16, 32×32, and 64×64 pixels. The step size of the sliding windows is set to 1 / 4 of the window size to ensure sufficient overlap between adjacent regions. The sliding windows scan the entire image, generating multiple sub-images.

[0024] When constructing the correlation strength matrix between a pixel and its neighborhood within each sub-image, a 5×5 pixel neighborhood is defined, centered around the current pixel. The grayscale difference between the central pixel and each pixel in the neighborhood is calculated; smaller differences indicate greater correlation strength. The correlation strength is determined using the inverse function of the grayscale difference. When the grayscale difference is 0, the correlation strength reaches its maximum value of 1; when the grayscale difference exceeds a threshold of 20, the correlation strength drops below 0.1.

[0025] When calculating the local entropy value for each pixel based on the correlation strength matrix, the correlation strength matrix is ​​normalized so that the sum of all its elements is 1, forming a probability distribution. For the pixel with coordinates (i, j) in the image, the distribution of grayscale values ​​within its 5×5 neighborhood is statistically analyzed, and the entropy of this distribution is calculated as the local entropy value of the pixel. Higher local entropy values ​​indicate greater texture complexity in the area, while lower local entropy values ​​indicate greater smoothness.

[0026] To extract the gradient vector for each pixel, the Sobel operator is used to calculate the horizontal and vertical gradient components. For a pixel at coordinates (i, j) in the image, the horizontal gradient is calculated by the grayscale difference between the pixels at (i, j+1) and (i, j-1), and the vertical gradient is calculated by the grayscale difference between the pixels at (i+1, j) and (i-1, j). The horizontal and vertical gradient components are combined to determine the magnitude and direction of the gradient vector.

[0027] The gradient vector is projected in eight different directions, with angles spaced 45 degrees apart. For each pixel, the distribution of the gradient directions of all pixels within its 11×11 neighborhood is calculated. If the gradient component in a particular direction accounts for more than 60%, the region is considered to have high directional consistency; if the gradient components in all directions are evenly distributed, the directional consistency is low. The gradient directional consistency range is [0, 1], with values ​​closer to 1 indicating more consistent gradient directions within the region.

[0028] When combining local entropy and gradient directional consistency to generate regional feature data, a weighted average of the two is performed with a weight ratio of 6:4. For regions rich in detail, the local entropy weight is increased to 0.7; for edge and texture regions, the gradient directional consistency weight is increased to 0.6. The combined feature data reflects the complexity and structural characteristics of the region. When extracting the extreme points of the local entropy and gradient directional consistency for each pixel in the regional feature data, the local entropy threshold is set to 0.75, and the gradient directional consistency threshold is set to 0.8. A search is performed within a 7×7 window for local maxima or minima, and points that meet the conditions are marked as candidate extreme points. The preset extreme value range is points with local entropy values ​​between [0.7, 0.9] or gradient directional consistency between [0.75, 0.95].

[0029] When clustering extreme points that meet the preset extreme value range, a density clustering algorithm is used, with a cluster radius of 15 pixels. When the Euclidean distance between two extreme points is less than the cluster radius and the characteristic difference is less than 0.2, they are classified into the same cluster. Each cluster center point is determined as a regional seed point. For a 2048×1536 image, 30-50 regional seed points are typically obtained.

[0030] When establishing topological connections between regional seed points, connections are established between seed points within a distance of 50 pixels. The connection strength is proportional to the feature similarity between the seed points and inversely proportional to the distance. The feature transfer coefficient is calculated based on the connection strength, ranging from 0.3 to 0.9. The transfer coefficient is close to 0.9 between seed points with high similarity. When generating a region growing path, the region starts from the seed point with the highest feature value and gradually expands along the direction of maximum connection strength. During the growth process, the rate of change of features from the current point to the next point does not exceed 0.15 to ensure the smoothness of the growth path. The growth path is recorded as a sequence of coordinate points for subsequent feature transfer. When transferring the local entropy and gradient direction consistency of regional seed points along the region growing path, the maximum transfer radius is set to 100 pixels. The transfer strength gradually decreases with increasing distance. An exponential decay function is used as the attenuation factor: 0.9 at a distance of 10 pixels, 0.5 at a distance of 50 pixels, and 0.2 at a distance of 100 pixels.

[0031] Regional enhancement weights are calculated by multiplying the local entropy, gradient directional consistency, and an attenuation factor. For high-texture regions, the local entropy contribution weight is 0.7; for regions with strong directional characteristics, the gradient directional consistency contribution weight is 0.6. The enhancement weight range is [0.1, 1.0], with larger weights indicating greater enhancement effects. When constructing a nonlinear mapping function based on the enhancement coefficient to enhance the regional sub-image, an S-shaped curve mapping is used. For regions with high enhancement coefficients, the curve slope is steeper, enhancing contrast; for regions with low enhancement coefficients, the curve slope is smaller, preserving the original characteristics. The parameters of the mapping function are dynamically adjusted based on regional characteristics to ensure that the enhancement effect is adaptive to different regions. When a feature transfer coefficient is introduced at region boundaries to adjust the enhancement strength, the boundary width is set to 10 pixels. The enhancement strength of the boundary region is the weighted average of the enhancement strengths of adjacent regions, with the weight determined by the feature transfer coefficient. When the feature transfer coefficient is 0.7, the enhancement strength of the boundary region inherits 70% of the characteristics of the more enhanced region and 30% of the characteristics of the less enhanced region.

[0032] When recombining the enhanced sub-images, overlapping regions are fused using a weighted average method. The weight is inversely proportional to the distance between the pixel and the region center, ensuring a smooth transition between the recombined images. The final output enhanced image has the same resolution as the input image, but with sharper details and more appropriate contrast, adapting to the specific requirements of different regions.

[0033] The above technical solution introduces the local entropy value and gradient direction consistency joint modeling to accurately depict the fine-grained texture and edge direction features in the image, combines extreme point clustering and topological connection to generate regional seed points, and realizes the diffusion enhancement of information to the regional boundary through feature transfer and attenuation control, effectively improving the image contrast and structural clarity in the defect area. By constructing a nonlinear mapping function to achieve adaptive enhancement, it avoids the traditional image enhancement of the entire Figure 1 The problem of knife cutting is solved, which improves the recognition sensitivity of low-contrast, small-area or blurred-boundary defects, lays a high-quality data foundation for subsequent feature extraction and defect positioning, and improves detection accuracy and robustness.

[0034] In an optional embodiment, extracting structural features of the enhanced image data and establishing a feature transfer sequence includes: Constructing a pyramid hierarchical structure for the enhanced image data, dividing overlapping sub-regions in each layer of the structure, extracting local gradient distributions of the overlapping sub-regions, and adjusting filter direction parameters based on the local gradient distributions; The adjusted filter is used to extract the structural features of the overlapping sub-regions, a corresponding mapping is established for the structural features in the overlapping regions of adjacent layers, a feature fusion order is determined according to the corresponding mapping, and the structural features of each layer are combined according to the feature fusion order to obtain a multi-level structural feature; A hierarchical transfer tree is constructed based on the feature fusion order, and a feature transfer link is established in the hierarchical transfer tree. The feature transfer link includes transfer nodes between adjacent features. The spatial structure distribution and directional consistency between adjacent transfer nodes are analyzed, and the node transfer priority is determined according to the spatial structure distribution and directional consistency. A feature transfer sequence is generated based on the node transfer priority.

[0035] For example, when constructing a pyramid hierarchical structure for the enhanced image data, the original image is downsampled according to the size ratio, usually using a scaling ratio of 1 / 2, to form a pyramid structure that gradually increases from the top to the bottom. For example, for a 1024×1024 pixel image, a 5-layer pyramid structure can be constructed, where the 0th layer is the original size of 1024×1024, the 1st layer is 512×512, the 2nd layer is 256×256, the 3rd layer is 128×128, and the 4th layer is 64×64. When dividing the overlapping sub-regions in each layer of the structure, an overlap ratio of 30% to 50% is usually used. For example, for a 512×512 image in the first layer, it can be divided into multiple 256×256 sub-regions, with an overlap of 128 pixels between adjacent sub-regions to ensure feature continuity and smooth transitions.

[0036] When extracting the local gradient distribution of overlapping sub-regions, the gradient amplitude and direction of the pixel are calculated for each sub-region. In specific implementation, the gradient of each pixel can be calculated using the horizontal and vertical differential operators. For example, for the pixel at position (x, y), its horizontal gradient Gx and vertical gradient Gy are calculated, and then the gradient amplitude G and direction θ are synthesized. The gradient amplitude G is equal to the square root of the sum of the squares of the horizontal gradient and the vertical gradient, and the gradient direction θ is equal to the inverse tangent of the vertical gradient and the horizontal gradient. The distribution of the gradient direction in the sub-region is statistically analyzed, usually dividing 360 degrees into 8 or 16 direction intervals, calculating the cumulative intensity of the gradient in each direction interval, and obtaining the gradient direction histogram of the sub-region.

[0037] When adjusting the filter directional parameters based on the local gradient distribution, the directional sensitivity of the filter is adjusted according to the main directions in the gradient direction histogram. For example, if the gradient of a sub-region is mainly distributed in the 45-degree and 225-degree directions, the filter parameters are adjusted to be more sensitive to these directions. In specific implementation, the center direction of the directional filter can be adjusted to θmax according to the directional angle θmax corresponding to the highest peak in the gradient direction histogram, and the directional bandwidth of the filter can be adjusted according to the discreteness of the histogram. When the gradient direction distribution is more concentrated, set a narrower directional bandwidth (such as 15 degrees); when the directional distribution is more dispersed, set a wider directional bandwidth (such as 45 degrees).

[0038] When using the adjusted filter to extract structural features in overlapping subregions, the adjusted directional filter is applied to the corresponding subregion to extract structural information such as edges and textures. For example, for a subregion with a dominant orientation of 45 degrees, a convolution operation is performed using a directional filter with a central orientation of 45 degrees and a bandwidth of 30 degrees to extract structural features in that orientation. For each subregion, structural features in multiple directions can be extracted to form a feature vector. In practical applications, features can be extracted in 8 or 16 directions, with each direction corresponding to a feature channel.

[0039] When establishing corresponding mappings between structural features in overlapping areas of adjacent layers, it is necessary to consider the scale changes between different layers. For example, a 256×256 sub-region in the first layer corresponds to a 512×512 region in the 0th layer, and the mapping is established through position correspondence. In specific implementation, the bilinear interpolation method can be used to upsample smaller-scale features to a larger scale, or downsample larger-scale features to a smaller scale, and then calculate the similarity matrix between the two feature maps. The similarity calculation can use cosine similarity or Euclidean distance. For feature point pairs whose similarity exceeds a threshold (such as 0.7), a corresponding mapping relationship is established.

[0040] When determining the feature fusion order based on corresponding mappings, analyze the strength of the correlation between features at different levels. Specifically, the average similarity of feature mapping points at adjacent levels can be calculated. The higher the similarity, the higher the fusion priority. For example, if the average mapping similarity between layer 1 and layer 0 is 0.85, and the average mapping similarity between layer 2 and layer 1 is 0.76, the feature fusion order is to fuse layer 1 with layer 0 first, and then with layer 2.

[0041] When combining the structural features of each layer according to the feature fusion order to obtain multi-level structural features, a weighted fusion method is used. For example, if the determined fusion order is to fuse the first layer and the 0th layer first, with weights of 0.4 and 0.6 respectively, then the fused feature F01 is equal to 0.4 multiplied by the first layer feature F1 plus 0.6 multiplied by the 0th layer feature F0. F01 is then fused with the second layer feature F2 with weights of 0.7 and 0.3 to obtain F012. This is repeated in this way to complete the feature fusion of all layers.

[0042] When constructing a hierarchical transfer tree based on the feature fusion order, the feature nodes of each layer are connected in the fusion order to form a tree structure. For example, if the fusion order is to fuse layers 0 and 1 first, and then layer 2, the root node of the tree is the fused F012, the next lower nodes are F01 and F2, and the next lower nodes are F0 and F1. When establishing feature transfer links in the hierarchical transfer tree, feature nodes of adjacent layers are connected through edges to form a transfer path.

[0043] When analyzing the spatial structure distribution and directional consistency between adjacent transfer nodes, the spatial distribution similarity and directional consistency scores of the node features are calculated. Spatial distribution similarity can be measured by the absolute value of the difference in the spatial distribution entropy of the feature graphs. The closer the entropy values, the more similar the distributions. Directional consistency is measured by calculating the angle between the main directions of two nodes. The smaller the angle, the higher the consistency. Node transfer priority is determined based on spatial distribution similarity and directional consistency. A weighted sum approach can be used. For example, the priority score is equal to 0.6 times the spatial distribution similarity plus 0.4 times the directional consistency score.

[0044] When generating a feature transfer sequence based on node transfer priority, all transfer nodes are sorted from high to low priority to form a transfer sequence. For example, if the priority score of the node pair (F0, F1) is 0.92 and the priority score of the node pair (F01, F2) is 0.85, the transfer sequence is to first transfer F0 to F1, and then transfer F01 to F2. The resulting feature transfer sequence can guide the effective fusion and transfer of multi-level features, improving the accuracy and robustness of image structural feature extraction.

[0045] In this embodiment, by constructing a pyramid hierarchical structure and introducing overlapping sub-region division, the filter directional parameters are dynamically adjusted in combination with the local gradient distribution, so that the structural feature extraction is made more directional adaptable and multi-scale perceptual. By mapping and fusing the structural features of adjacent levels, multi-level structural features are constructed, which enhances the expression ability of complex textures and subtle deformations. The spatial structure distribution and directional consistency analysis between the hierarchical transfer tree and the nodes are introduced to ensure that the feature transfer process has structural continuity and semantic relevance, thereby generating a feature transfer sequence with hierarchical and priority control, providing stable and high-quality structural information support for subsequent feature fusion and defect area identification.

[0046] In an optional embodiment, a correlation matrix between features is calculated according to a feature transfer sequence, a feature optimization path is established based on the correlation matrix, and features are progressively fused along the feature optimization path to obtain fused feature data including: Calculating feature correlation strengths between adjacent transfer nodes in a feature transfer sequence, transferring the feature correlation strengths step by step in the feature transfer sequence, and constructing a correlation matrix; Determining an initial transfer node in the association matrix, expanding the next transfer node step by step from the initial transfer node based on the feature association strength, establishing transfer connections between nodes, and connecting the transfer connections in series in descending order of feature association strength to form a feature optimization path; Along the feature optimization path, starting from the initial transfer node, the features of the current transfer node are weightedly fused with the features of the next transfer node. The weight of the weighted fusion is determined by the feature correlation strength of the corresponding transfer connection. The fused features are used as the new current node features and continue to be fused with the features of the next transfer node. The fusion of all transfer node features is completed step by step to obtain the final fused feature data.

[0047] Exemplarily, when calculating the feature association strength between adjacent transfer nodes in a feature transfer sequence, for any two adjacent nodes, the feature association strength is determined by calculating the similarity of their feature vectors. The similarity calculation uses the cosine distance metric, taking the dot product of the two feature vectors divided by the product of their respective module lengths. When the two extracted node features are 128-dimensional vectors, if the angle between the two vectors is 30 degrees, the calculated association strength is approximately 0.866; if the angle is 60 degrees, the association strength is approximately 0.5. The association strength threshold is usually set to 0.75, and node pairs below this threshold are considered weakly associated.

[0048] The progressive transmission of feature association strength in the feature transfer sequence utilizes a cumulative decay mechanism. Starting from the starting node, the association strength decays by a certain percentage with each node passed through the transfer sequence. The decay coefficient is dynamically adjusted based on the distance between nodes. Typically, the decay coefficient for adjacent nodes with a distance of 1 is 0.9, for nodes with a distance of 2 it is 0.8, and for nodes with a distance of 3 it is 0.7. Specifically, if the association strength between nodes A and B is 0.85, and the association strength between B and C is 0.8, then the association strength transmitted from A to C via B is 0.85 × 0.8 × 0.9 = 0.612.

[0049] When constructing the correlation matrix, the direct and indirect correlation strengths are calculated for each pair of nodes in the feature transfer sequence, and the maximum of the two is taken as the final correlation strength. Each element in the matrix represents the correlation strength between different nodes, forming a symmetric matrix. A sequence containing five transfer nodes might generate the following correlation matrix: the first and second nodes are 0.89, the second and third nodes are 0.76, the third and fourth nodes are 0.82, the fourth and fifth nodes are 0.65, the first and third nodes are 0.68, the second and fourth nodes are 0.62, the third and fifth nodes are 0.53, the first and fourth nodes are 0.56, the second and fifth nodes are 0.48, and the first and fifth nodes are 0.36.

[0050] When determining the initial transfer node in the affinity matrix, the node with the highest average affinity strength is selected. This is calculated by summing the elements in each row of the matrix and selecting the node corresponding to the row with the largest sum. If the sum of the affinity strengths of the second node and all other nodes is 2.75, which is higher than the sum of the affinity strengths of all other nodes, the second node is selected as the initial transfer node.

[0051] When expanding from the initial transfer node to the next transfer node based on feature association strength, a greedy strategy is used to select the unvisited node with the highest association strength with the current node. If the initial node is the second node, and its association strengths with the first, third, fourth, and fifth nodes are 0.89, 0.76, 0.62, and 0.48, respectively, the first node is selected as the next transfer node. Each time a new node is selected, the set of visited nodes is updated, and the unvisited node with the highest association strength continues to be selected until all nodes have been visited.

[0052] When establishing transitive connections between nodes, the selected node sequence is converted into directed connections. For example, if the selected node sequence is [second, first, third, fourth, fifth], the transitive connections established are from second to first, first to third, third to fourth, and fourth to fifth, with each connection labeled with the corresponding association strength. Transitive connections are connected in descending order of feature association strength to form a feature optimization path.

[0053] When performing weighted fusion along the feature optimization path, starting from the initial transitive node, the fusion proceeds to the next node. The fusion weight is determined by the strength of the feature correlation between the corresponding transitive connections. For example, if the correlation strength between the second node and the first node is 0.89, the fusion weight is set to 0.89 to 0.11, meaning that the second node's features account for 89% and the first node's features account for 11%. The fusion operation uses a weighted average; for each feature dimension, the fused value is equal to the weighted average of the feature values ​​of the two nodes.

[0054] In specific implementation, for two 128-dimensional feature vectors with a correlation strength of 0.89, the fusion result is calculated as: fused feature = 0.89 × second node feature + 0.11 × first node feature. The fused feature is used as the new current node feature and then fused with the feature of the third node, the next node. If the correlation strength between the fused feature and the third node is 0.76, the fusion weights are 0.76 to 0.24, and the fusion result = 0.76 × fused feature + 0.24 × third node feature.

[0055] The progressive fusion process specifically includes the following steps: first, the feature of the second node of the initial transfer node is taken as the current feature; the correlation strength between the current feature and the first node feature of the next transfer node is calculated to be 0.89; the fusion weight is set based on the correlation strength, and the current feature and the feature of the next node are weighted averaged to obtain the new current feature; the correlation strength between the new current feature and the feature of the third node of the next transfer node is calculated to be 0.76; the fusion weight is continued to be set based on the correlation strength, and the weighted average is performed to update the current feature; and so on, until all transfer nodes are involved in the fusion, and the final fused feature data is obtained. Through this progressive fusion method, the features of all nodes are gradually fused along the feature optimization path, and finally the fused feature data containing complete information is obtained. The final fused feature can effectively express the characteristics of various types of defects on the product surface. The fused feature has the same dimension as the original feature, but contains comprehensive information at multiple scales and directions, significantly improving the accuracy of defect detection and classification.

[0056] In this embodiment, by quantifying the feature correlation strength between adjacent nodes in the feature transfer sequence, a correlation matrix that accurately reflects the similarity and dependency relationship between features is constructed, thereby avoiding the problems of disorder or equal-weight superposition in traditional feature fusion; based on the correlation matrix, an ordered and prioritized feature optimization path is constructed, so that the feature fusion process follows the principle of strong correlation priority, and a progressive fusion strategy from local to global and from shallow to deep is realized; through a dynamic weighted method, the features of each transfer node are integrated step by step, key structural information is retained and redundant interference is suppressed, thereby improving the discriminability and stability of the fused features, and providing higher-quality feature expression for defect identification and classification.

[0057] In an optional embodiment, a defect area probability distribution map is generated based on the fused feature data, and the defect area probability distribution map is modified in combination with the regional enhancement weight. Determining the defect area includes: Calculating local statistics for the fused feature data, constructing an adaptive kernel function based on the local statistics, decomposing the adaptive kernel function using an orthogonal transformation to obtain a basis function group, and convolving the basis function group with the fused feature data to obtain a feature response map; Performing gradient diffusion on the characteristic response map to obtain a density flow field, extracting gradient trajectories in the density flow field, calculating regional clustering features based on convergence points and convergence directions of the gradient trajectories, and generating a defect region probability distribution map based on the regional clustering features; The regional enhancement weight is adaptively weighted fused with the defect area probability distribution map to obtain a corrected probability map, the boundary point set of the corrected probability map is extracted, the multi-order moment features of the boundary point set are calculated, and a level set function is constructed for the corrected probability map. The evolution speed of the level set function is adaptively adjusted based on the multi-order moment features, and the defect area is obtained by segmentation using the zero level plane of the level set function.

[0058] In implementing the present invention, local statistics of the fused feature data are first calculated. This involves calculating the mean, variance, skewness, and kurtosis of each local region in the feature space using a sliding window approach. In this embodiment, a 7×7 pixel local window with a sliding step size of 1 is used to calculate these statistics for the feature values ​​within each window. For example, for a particular region, local statistical features with a mean of 0.65, a variance of 0.23, a skewness of 0.08, and a kurtosis of 3.12 may be obtained.

[0059] An adaptive kernel function is constructed based on the calculated local statistics, and the kernel function can adaptively adjust the shape and scale according to the local feature distribution. In this embodiment, the bandwidth matrix of the kernel function is determined by the covariance matrix of the local area. When the feature variance is large, the scalability of the kernel function in this direction is stronger. For example, for areas with complex textures, the kernel function exhibits anisotropy, the major axis direction is consistent with the texture direction, and the ratio of the minor axis to the major axis is 1:3. The adaptive kernel function is decomposed by orthogonal transformation to obtain a set of basis functions. In this embodiment, the singular value decomposition technique is used to decompose the kernel function into 5 orthogonal basis functions, each of which captures feature information in different directions and scales. The weights of these basis functions are 0.45, 0.25, 0.15, 0.10 and 0.05, respectively, representing their respective importance in feature expression.

[0060] The basis function group is convolved with the fused feature data to generate a feature response map. In this embodiment, the convolution operation is accelerated using a fast Fourier transform (FFT). The response map corresponding to each basis function is calculated and then weighted and combined according to the basis function weights to form the final feature response map. For example, for a 512×512 pixel image, a feature response map of the corresponding size is generated, where the response value of the defect area is significantly higher than that of the background area. The response value range of a typical defect area is [0.75, 0.95], while the response value range of the background area is [0.05, 0.30].

[0061] Gradient diffusion is performed on the characteristic response map to obtain a density flow field. In this embodiment, an anisotropic diffusion method is used, the diffusion time is 10 iterative steps, and the diffusion coefficient is adaptively adjusted according to the gradient amplitude. The diffusion in the area with a large gradient is weak, and the diffusion in the area with a small gradient is strong. For example, the gradient threshold is set to 0.2. When the gradient amplitude is less than the threshold, the diffusion coefficient is 0.8, otherwise it is 0.2×(1-gradient amplitude). The gradient trajectory is extracted from the density flow field, and the trajectory is formed by randomly selecting a starting point in the flow field and tracking along the gradient direction. In this embodiment, 1000 random starting points are selected, the maximum number of tracking steps for each trajectory is 100, the step size is 0.5 pixels, and tracking is stopped when the distance between trajectory points is less than 0.1 pixels or exceeds the image boundary.

[0062] The regional clustering features are calculated based on the convergence points and convergence directions of the gradient trajectory. In this embodiment, a density clustering algorithm is used to classify convergence points with a distance of less than 5 pixels into one category, and the center position, size, shape and direction of each cluster are calculated. For example, for a typical defect, 3-5 convergence regions may be formed, each region containing 50-200 convergence points. A probability distribution map of the defect area is generated based on the regional clustering features. For each pixel point, the distance and direction similarity to the nearest convergence area are calculated and converted into a probability value. For example, the distance function adopts a Gaussian attenuation model with a standard deviation of 15 pixels. When the distance from the point to the nearest convergence area is 0, the probability is 1, and when the distance is 30 pixels, the probability drops to 0.1.

[0063] Construct a regional enhancement weight map that takes into account the original image's edge information, texture complexity, and local contrast. In this example, edge weights are extracted using the Canny operator with thresholds set to 50 and 150; texture complexity is calculated using local entropy with a window size of 9×9; and contrast is calculated by dividing the local standard deviation by the local mean. The final weight is the weighted sum of these three factors, with weights of 0.4, 0.35, and 0.25, respectively.

[0064] The regional enhancement weights are adaptively weighted and fused with the defect region probability distribution map to produce a revised probability map. In this embodiment, the fusion formula is: revised probability = original probability × (1 + α × enhancement weight), where α is an adaptive coefficient determined based on global probability distribution statistics and typically has a value of 0.4. A boundary point set is extracted from the revised probability map and binarized using a probability threshold (e.g., 0.65). Multi-order moment features of the boundary point set are calculated, including centroid, principal axis orientation, and eccentricity. For example, for a typical crack defect, its second-order central moment may indicate a principal axis orientation of 32 degrees and a major-minor axis ratio of 5:1. A level set function is constructed for the revised probability map. The initial level set function is the revised probability map minus a threshold (e.g., 0.5). Based on the multi-order moment features, the evolution rate of the level set function is adaptively adjusted, with faster evolution along the principal axis and slower evolution perpendicular to the principal axis. In this embodiment, the velocity coefficient along the principal axis is 1.5 and 0.8 perpendicular to the principal axis. The final defect region is segmented using the zero-level plane of the level set function. Level set evolution uses an iterative approach with a maximum of 100 iterations. It stops when the boundary change is less than 0.5 pixels after five consecutive iterations. For example, for a defect with an area of ​​approximately 500 square pixels, convergence occurs after approximately 35 iterations, resulting in an accurate defect area outline.

[0065] The above technical solution calculates local statistics on the fused feature data and constructs an adaptive kernel function to improve the sensitivity of the feature response map to defects of different scales and shapes; introduces gradient diffusion to generate a density flow field and extracts gradient trajectories, which can effectively identify the spatial aggregation trend of defect features and improve the accuracy of defect area positioning; combines regional enhancement weights for weighted correction to achieve adaptive enhancement of defects with blurred boundaries or low contrast; guides the evolution of the level set function through multi-order moment features to make the defect segmentation boundary more refined and smoother, ultimately achieving high-precision extraction of defect areas and enhancing the robustness of detection and segmentation accuracy.

[0066] Figure 2 FIG. 1 is a schematic diagram showing the performance comparison of different defect detection methods according to an embodiment of the present invention. Figure 2 The figure shows the detection accuracy of three different defect detection algorithms (adaptive kernel method, traditional convolutional neural network, and level set segmentation method) in four different defect types (crack detection, surface depression, texture anomaly, and complex background) scenarios. The data clearly shows that the adaptive kernel method performed best in all test scenarios, with an accuracy ranging from 85.2% to 92.7%. The traditional convolutional neural network performed second, and the level set segmentation method performed relatively poorly overall. In particular, under complex background conditions, the adaptive kernel method's accuracy (85.2%) far exceeded that of the traditional methods (71.8% and 68.5%), demonstrating the algorithm's superiority in handling complex scenes.

[0067] In an optional embodiment, extracting the topological structure features of the defect area, establishing a defect feature description, classifying the defect based on the defect feature description, and outputting the defect type and confidence score include: A closed contour of the defect area is extracted using a boundary tracking algorithm with an adaptive threshold, and the closed contour is subjected to noise reduction and smoothing to obtain a continuous boundary. A contour point sequence is established, and a distance transform field is constructed based on the contour point sequence. A center line is extracted from the distance transform field using a gradient descent method to construct a center point sequence. Mapping the contour point sequence and the center point sequence to the same coordinate space to construct a topological structure of the defect area, extracting branch nodes and intersections in the topological structure, calculating the connection relationship between adjacent nodes, constructing a topological tree based on the connection relationship, and extracting structural features of the topological tree as topological structure features of the defect area; Normalizing each feature component of the topological structure feature, calculating the correlation coefficient between the normalized feature components, constructing a feature correlation matrix, calculating the distinguishing ability of each feature component based on the feature correlation matrix, determining a feature weight based on the distinguishing ability, and using the feature weight to perform a weighted combination of the feature components to establish a defect feature description; The defect feature description is matched with preset multi-type defect samples, the distance and similarity between the feature descriptions are calculated, a feature matching matrix is ​​constructed, the matching degree between the sample to be tested and each type of defect sample in the feature matching matrix is ​​analyzed, the type with the highest matching degree is determined as the defect type, and its corresponding matching degree is used as the confidence score.

[0068] Exemplarily, when extracting the closed contour of the defect area, an adaptive threshold boundary tracking algorithm is used. The algorithm first performs grayscale processing on the image, and determines the threshold by calculating the grayscale mean and standard deviation of the local area. The threshold is set to the mean minus 0.5 times the standard deviation. For example, for an 8-bit grayscale image, if the grayscale mean of a region is 120 and the standard deviation is 40, the threshold of the region is 120-0.5×40=100. Based on this threshold, the image is binarized to obtain the initial boundary of the defect area. The boundary tracking process uses the 8-connectivity criterion, starting from the first boundary point in the upper left corner, exploring adjacent points in a clockwise direction, recording the coordinates of all boundary points, and returning to the starting point to complete the extraction of the closed contour.

[0069] When performing noise reduction and smoothing on the extracted closed contours, a sliding window averaging method is used, with a window size of 7 pixels. For each contour point, the coordinates of the three points before and after it are taken, and the average of these seven points is calculated as the new position of the current point. For example, if the original contour point coordinates are [(10, 15), (12, 15), (14, 16), (16, 17), (18, 18), (20, 20), (22, 21)], after smoothing, the new coordinates of the middle point (16, 17) are (16, 17.4). After processing all points, a smooth and continuous sequence of boundary and contour points is obtained.

[0070] When constructing the distance transform field, the shortest distance from each point in the image to the boundary is calculated. The implementation uses a two-dimensional array to store distance values, initializing the distances to boundary points to 0 and non-boundary points to infinity. The distance value for each point is calculated using two scans (forward and reverse). The forward scan runs from the top left corner to the bottom right corner, and the reverse scan runs from the bottom right corner to the top left corner. Each scan updates the distance value of the point to the minimum of the current value and the distance to the adjacent point plus 1. For example, if the current distance value of a point is 5 and the distance to its upper neighbor is 3, the updated value for that point is min(5, 3+1) = 4.

[0071] When extracting the centerline from the distance transform field using the gradient descent method, we start from the local maximum point of the distance field and iterate along the gradient direction. For each point, we calculate the gradient of the distance values ​​of its eight neighboring points and proceed in the direction with the maximum gradient until we reach the local extreme point or the visited centerline point. For example, if the eight-neighborhood distance values ​​of a point are [5, 6, 5, 4, 3, 4, 5, 6], the gradient direction is toward the second value (6). During the iteration process, the coordinates of all visited points are recorded to form a sequence of center points. If there are multiple starting points, they are merged into a complete centerline by analyzing the connectivity between the sequences.

[0072] When mapping the contour point sequence and the center point sequence to the same coordinate space, the original image coordinate system is kept unchanged, and the point sets of the two sequences are merged. After merging, a topological structure representation of the defect area is formed. When extracting branch nodes and intersection points in the topological structure, the number of centerline points in the neighborhood of each point on the centerline is calculated. The neighborhood radius is set to 3 pixels. If there are 3 or more centerline segments in different directions in the neighborhood, the point is an intersection point; if there are more than 2 centerline points but only 2 different directions, it is a branch node. For example, if there are 5 centerline points in the neighborhood of a point, each coming from a line segment in 3 different directions, the point is marked as an intersection point.

[0073] To calculate the connections between adjacent nodes, we start from each node and trace along the centerline until we reach the next node, recording the distance, angle, and centerline width change between the two nodes. For example, the distance between nodes A and B is 30 pixels, the average width is 5 pixels, and the standard deviation of the width is 0.8 pixels. Based on these connections, we construct a topological tree, where each node represents a branch node or intersection, and edges represent the connections between nodes.

[0074] When extracting the structural features of a topological tree, we calculate metrics such as the number of nodes, the number of leaf nodes, the tree depth, and the branch density. For example, a defect topological tree has 15 nodes, 8 of which are leaf nodes, a tree depth of 4, and a branch density of 0.6. These metrics form the topological structural feature vector of the defect region.

[0075] The maximum and minimum normalization method is used to normalize the topological structure features. For example, the minimum value of the number of nodes in the sample is 5, the maximum value is 25, and the number of nodes in a certain sample is 15, then the normalized value is (15-5) / (25-5)=0.5. Similar processing is performed on all feature components to obtain normalized feature vectors. When calculating the correlation coefficient between feature components, the Pearson correlation coefficient between each feature is used. For example, the correlation coefficient between the number of nodes and the number of leaf nodes is 0.85, indicating a high correlation. All correlation coefficients form a feature association matrix. The information gain method is used to calculate the discriminative ability of each feature component based on the feature association matrix. For each feature, its information gain value in the classification task is calculated. The higher the value, the stronger the discriminative ability. For example, the information gain of the branch density feature is 0.75, indicating that the feature has a strong discriminative ability.

[0076] The information gain value of each feature is normalized and used as the weight of the corresponding feature. For example, in a defect classification task, the topological structure features include the number of nodes, the number of leaf nodes, the tree depth, and the branch density, with weights of 0.2, 0.15, 0.25, and 0.4, respectively. These weights are used to weight the feature components and combine them to form a defect feature description. The defect feature description is matched with pre-set defect samples of multiple types, and the Euclidean distance and cosine similarity between the test sample and the feature description of each defect type are calculated. For example, the Euclidean distance and cosine similarity between the test sample and Class A defects are 0.25 and 0.92, respectively; the Euclidean distance and cosine similarity between the test sample and Class B defects are 0.47 and 0.81, respectively. These distance and similarity values ​​form the feature matching matrix.

[0077] When analyzing the degree of match between the sample under test and each defect type in the feature matching matrix, the system takes into account both distance and similarity to calculate a comprehensive matching score. For example, the comprehensive score for Class A defects is 0.85, while that for Class B defects is 0.72. The class with the highest score is selected as the defect type, and its corresponding matching score is used as the confidence score. In this example, the system outputs the recognition result as "Defect Type: Class A; Confidence: 0.85."

[0078] Based on the above technical solution, it is possible to achieve structured modeling and accurate classification of complex defect areas. By extracting closed contours and centerlines to construct a topological structure, the geometric features and spatial connection relationships of the defect area are retained, improving the comprehensiveness and separability of defect representation. Based on the topological tree structure, key node relationships are extracted and their characteristics are quantified, which helps to distinguish defects with similar morphology but obvious structural differences. Correlation analysis and weighted combination between feature components are introduced to construct defect feature descriptions with high discriminative power, improving the accuracy and robustness of classification in feature matching. The final output defect type and confidence score support intelligent and explainable defect identification and subsequent quality control.

[0079] In an optional embodiment, a dynamic decision matrix is ​​constructed based on the regional enhancement weights and confidence scores. A comprehensive score for each defect region is calculated in the dynamic decision matrix. Defects are graded based on the comprehensive score to generate a quantitative indicator of defect severity. The defect detection results are output including: Calculating the distribution uniformity of the regional enhancement weight and the fluctuation range of the confidence score, using the distribution uniformity and fluctuation range as scoring weight coefficients, performing weighted fusion on the regional enhancement weight and the confidence score to generate a scoring matrix; Extracting the variation pattern of the regional enhancement weight with position to obtain regional variation characteristics, extracting the stability of the confidence score to obtain reliability characteristics, calculating the dimension weights of the scoring matrix based on the regional variation characteristics and the reliability characteristics, and constructing a dynamic decision matrix based on the dimension weights; Calculating the relative importance of the dimensions in the dynamic decision matrix to generate an importance weight vector, and multiplying the importance weight vector by the scoring matrix to obtain a comprehensive score for each defect area; A hierarchical evaluation standard is constructed for the comprehensive score, a severity distribution is calculated according to the hierarchical evaluation standard, a membership function is used to convert the severity distribution into a severity quantification index, defect levels are divided according to the severity quantification index, and a detection result is output.

[0080] like Figure 3 As shown, a schematic diagram of the intelligent defect detection and comprehensive rating process of this embodiment is shown.

[0081] In this embodiment, when calculating the distribution uniformity of the regional enhancement weight, it is necessary to analyze the distribution of the enhancement weight within the defective area. The distribution uniformity is expressed by calculating the coefficient of variation of the enhancement weight within the region, where the coefficient of variation is the standard deviation divided by the mean. For a detected defective area of ​​a solder joint of an electronic component, if the mean value of the enhancement weight is 0.75 and the standard deviation is 0.15, the coefficient of variation is 0.2, indicating a relatively uniform distribution; if the standard deviation is 0.3 and the coefficient of variation is 0.4, it indicates an uneven distribution. The higher the distribution uniformity, the larger the corresponding scoring weight coefficient, which usually ranges from 0.6 to 0.9.

[0082] The confidence score's fluctuation range is determined by analyzing the difference between the maximum and minimum confidence scores for all pixels within the defect area. For a detected microcrack defect, if the maximum confidence score for the region is 0.92 and the minimum is 0.78, the fluctuation range is 0.14; if the maximum is 0.95 and the minimum is 0.55, the fluctuation range is 0.4. A smaller fluctuation range indicates a more stable confidence score and a corresponding larger scoring weight, typically ranging from 0.65 to 0.95.

[0083] Using distribution uniformity and fluctuation range as scoring weight coefficients, the regional enhancement weight and confidence score are weighted and fused to generate a scoring matrix. Specifically, for each defect area, the product of the distribution uniformity weight and the regional enhancement weight is calculated, and then the product of the fluctuation range weight and the confidence score is added to form the element value of the scoring matrix. For a product surface scratch defect, if the distribution uniformity weight is 0.85, the regional enhancement weight is 0.78, the fluctuation range weight is 0.75, and the confidence score is 0.83, then the corresponding element value of the scoring matrix is ​​0.85 × 0.78 + 0.75 × 0.83 = 1.29.

[0084] To extract the regional variation characteristics from the positional variation of the regional enhancement weight, a gradient analysis method is used to calculate the rate of change of the enhancement weight in the horizontal and vertical directions. For a metal surface concave defect, if the horizontal enhancement weight changes from left to right by 0.65, 0.72, 0.85, 0.79, and 0.68, the difference between adjacent positions can be calculated to be 0.07, 0.13, -0.06, and -0.11, indicating an initial increase followed by a decrease, forming a peak characteristic. If a similar vertical variation trend is observed, the regional variation characteristic is considered "centrally prominent" and a higher dimension weight of 0.8 is assigned. If the variation is monotonically from one side to the other, the regional variation characteristic is considered "gradually varying" and a medium dimension weight of 0.6 is assigned.

[0085] When extracting the stability of confidence scores to obtain reliability features, the local and cross-regional consistency of the confidence scores are analyzed. Local consistency refers to the difference in confidence scores between adjacent pixels and is typically measured using the local variance. Cross-regional consistency refers to the difference in confidence scores between different sub-regions and is typically measured using the inter-region variance. If the local variance is 0.02 and the inter-region variance is 0.05, the reliability feature value is 0.93, indicating high confidence stability. If the local variance is 0.08 and the inter-region variance is 0.12, the reliability feature value is 0.8, indicating moderate confidence stability.

[0086] The dimension weights of the scoring matrix are calculated based on the regional variation and reliability characteristics, and a dynamic decision matrix is ​​constructed based on the dimension weights. Dimension weights are calculated using a weighted average method, with the ratio of regional variation weight to reliability weight typically being 7:3. For example, if the regional variation weight is 0.75 and the reliability weight is 0.88, the dimension weights are 0.75 × 0.7 + 0.88 × 0.3 = 0.789. The dynamic decision matrix is ​​constructed by multiplying each element of the scoring matrix by the corresponding dimension weight.

[0087] The relative importance of dimensions in the dynamic decision matrix is ​​calculated to generate an importance weight vector. The relative importance of dimensions is determined based on a defect type signature library, with different importance weights assigned to different defect types. Regional enhancement weights are typically more important than confidence scores, and the importance weight vector might be [0.65, 0.35]. For solder joint defects on electronic components, the confidence score is typically more important than the regional enhancement weights, and the importance weight vector might be [0.4, 0.6]. The importance weight vector is multiplied by the scoring matrix to obtain a comprehensive score for each defect region. For a fine crack defect on a glass surface, if the corresponding elements of the scoring matrix are [1.25, 1.35] and the importance weight vector is [0.55, 0.45], the comprehensive score is 1.25 × 0.55 + 1.35 × 0.45 = 1.29.

[0088] A tiered assessment standard is constructed based on the comprehensive score, and the severity distribution is calculated based on this tiered assessment standard. This tiered assessment standard is typically determined based on historical data and expert experience. For example, for surface defects on automotive parts, a comprehensive score of less than 0.8 is considered a minor defect, 0.8 to 1.2 is considered a general defect, 1.2 to 1.6 is considered a severe defect, and greater than 1.6 is considered a fatal defect. The severity distribution is calculated by calculating the proportion of the defect area within each level range.

[0089] The membership function is used to convert the severity distribution into a severity quantification index. The membership function is a piecewise linear function that maps the comprehensive score to a severity quantification index range of 0 to 10. For example, for a pitting defect on the surface of a precision bearing, if the comprehensive score is 1.29, which falls within the severe defect range, the severity quantification index calculated by the membership function is 7.2 points. The defect level is divided according to the severity quantification index and the detection results are output. The defect level classification standard is, for example: 0 to 3 points for first-level defects, 3 to 6 points for second-level defects, 6 to 8 points for third-level defects, and 8 to 10 points for fourth-level defects. For defects with a severity quantification index of 7.2 points, it is determined to be a third-level defect, marked as a red warning in the detection results, and detailed information such as the coordinates, area, and shape characteristics of the defect area is provided to provide a decision-making basis for subsequent processing.

[0090] Based on the above technical solution, it is possible to achieve quantitative evaluation and intelligent grading of defect detection results. By fusing regional enhancement weights and confidence scores, a scoring matrix and a dynamic decision matrix are constructed to effectively characterize the importance of defect areas and the stability of detection. Dimension weights are further introduced by combining regional change characteristics with reliability characteristics to improve the accuracy and interpretability of scoring results. Relative importance analysis and weighted fusion calculation are used to calculate comprehensive scores, enhancing the sensitivity of distinguishing different types of defects. Finally, a continuous quantitative expression of defect levels is achieved through severity distribution and membership function, providing precise support for defect management and quality assessment.

[0091] A second aspect of an embodiment of the present invention provides a real-time machine vision defect detection and classification system based on deep learning, the system comprising: The first unit is used to obtain image data of the surface of the industrial product, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the regional enhancement weight according to the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the regional enhancement weight to obtain enhanced image data; The second unit is used to extract the structural features of the enhanced image data, establish a feature transfer sequence, calculate the correlation matrix between features based on the feature transfer sequence, establish a feature optimization path based on the correlation matrix, and progressively fuse the features along the feature optimization path to obtain fused feature data; The third unit is used to generate a probability distribution map of the defect area based on the fused feature data, modify the probability distribution map of the defect area based on the regional enhancement weight, determine the defect area, extract the topological structure features of the defect area, establish a defect feature description, classify the defects based on the defect feature description, and output the defect type and confidence score; The fourth unit is used to construct a dynamic decision matrix based on the regional enhancement weights and confidence scores, calculate the comprehensive score of each defect area in the dynamic decision matrix, grade the defects according to the comprehensive score, generate a quantitative indicator of the defect severity, and output the defect detection results.

[0092] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0093] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0094] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time detection and classification method for machine vision defects based on deep learning, characterized by: include: Acquire image data of the surface of industrial products, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the regional enhancement weight based on the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the regional enhancement weight to obtain enhanced image data; Extract the structural features of the enhanced image data, establish a feature transfer sequence, calculate the correlation matrix between features based on the feature transfer sequence, establish a feature optimization path based on the correlation matrix, and perform progressive fusion of features along the feature optimization path to obtain fused feature data; Generate a probability distribution map of the defect area based on the fused feature data, modify the probability distribution map of the defect area based on the regional enhancement weight, determine the defect area, extract the topological structure features of the defect area, establish a defect feature description, classify the defects based on the defect feature description, and output the defect type and confidence score; A dynamic decision matrix is ​​constructed based on the regional enhancement weights and confidence scores. The comprehensive score of each defect area is calculated in the dynamic decision matrix. The defects are graded according to the comprehensive score, a quantitative indicator of the defect severity is generated, and the defect detection results are output.

2. The method according to claim 1, characterized in that Acquire image data of the surface of industrial products, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the regional enhancement weight based on the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the regional enhancement weight. The enhanced image data includes: Acquire image data of the surface of an industrial product, and perform regional segmentation on the image data along a preset direction sequence to obtain multiple regional sub-images; Within each regional sub-image, a correlation strength matrix is ​​constructed between the pixel and its neighborhood. The local entropy value of each pixel is calculated based on the correlation strength matrix. The gradient vector of each pixel is extracted and projected in different directions. The gradient direction consistency of each pixel is calculated based on the projected component. The local entropy value and gradient direction consistency are combined to generate regional characteristic data. Extract the extreme points of local entropy and gradient direction consistency of each pixel in the regional characteristic data, cluster the extreme points that meet the preset extreme value range to obtain regional seed points, establish the topological connection relationship between the regional seed points, calculate the characteristic transfer coefficient, and generate the regional growth path; The local entropy value and gradient direction consistency of the regional seed point are transferred outward along the regional growth path. The attenuation factor is calculated according to the distance from the pixel point to the regional seed point. The product of the local entropy value, gradient direction consistency and attenuation factor is used as the regional enhancement weight. The regional characteristic data is weighted based on the regional enhancement weight to obtain the enhancement coefficient; A nonlinear mapping function is constructed according to the enhancement coefficient to enhance each regional sub-image; a characteristic transfer coefficient is introduced at the regional boundary to adjust the enhancement strength, and the enhanced regional sub-images are recombined to obtain the enhanced image data.

3. The method according to claim 1, characterized in that Extracting the structural features of the enhanced image data and establishing a feature transfer sequence include: Constructing a pyramid hierarchical structure for the enhanced image data, dividing overlapping sub-regions in each layer of the structure, extracting local gradient distributions of the overlapping sub-regions, and adjusting filter direction parameters based on the local gradient distributions; The adjusted filter is used to extract the structural features of the overlapping sub-regions, a corresponding mapping is established for the structural features in the overlapping regions of adjacent layers, a feature fusion order is determined according to the corresponding mapping, and the structural features of each layer are combined according to the feature fusion order to obtain a multi-level structural feature; A hierarchical transfer tree is constructed based on the feature fusion order, and a feature transfer link is established in the hierarchical transfer tree. The feature transfer link includes transfer nodes between adjacent features. The spatial structure distribution and directional consistency between adjacent transfer nodes are analyzed, and the node transfer priority is determined according to the spatial structure distribution and directional consistency. A feature transfer sequence is generated based on the node transfer priority.

4. The method according to claim 1, wherein According to the feature transfer sequence, the correlation matrix between features is calculated, and a feature optimization path is established based on the correlation matrix. The features are progressively fused along the feature optimization path to obtain the fused feature data including: Calculating feature correlation strengths between adjacent transfer nodes in a feature transfer sequence, transferring the feature correlation strengths step by step in the feature transfer sequence, and constructing a correlation matrix; Determining an initial transfer node in the association matrix, expanding the next transfer node step by step from the initial transfer node based on the feature association strength, establishing transfer connections between nodes, and connecting the transfer connections in series in descending order of feature association strength to form a feature optimization path; Along the feature optimization path, starting from the initial transfer node, the features of the current transfer node are weightedly fused with the features of the next transfer node. The weight of the weighted fusion is determined by the feature correlation strength of the corresponding transfer connection. The fused features are used as the new current node features and continue to be fused with the features of the next transfer node. The fusion of all transfer node features is completed step by step to obtain the final fused feature data.

5. The method according to claim 1, wherein Generate a probability distribution map of defect areas based on the fused feature data, and modify the probability distribution map of defect areas by combining the regional enhancement weights to determine the defect areas, including: Calculating local statistics for the fused feature data, constructing an adaptive kernel function based on the local statistics, decomposing the adaptive kernel function using an orthogonal transformation to obtain a basis function group, and convolving the basis function group with the fused feature data to obtain a feature response map; Performing gradient diffusion on the characteristic response map to obtain a density flow field, extracting gradient trajectories in the density flow field, calculating regional clustering features based on convergence points and convergence directions of the gradient trajectories, and generating a defect region probability distribution map based on the regional clustering features; The regional enhancement weight is adaptively weighted fused with the defect area probability distribution map to obtain a corrected probability map, the boundary point set of the corrected probability map is extracted, the multi-order moment features of the boundary point set are calculated, and a level set function is constructed for the corrected probability map. The evolution speed of the level set function is adaptively adjusted based on the multi-order moment features, and the defect area is obtained by segmentation using the zero level plane of the level set function.

6. The method according to claim 1, characterized in that Extract the topological structure features of the defect area, establish the defect feature description, classify the defects according to the defect feature description, and output the defect type and confidence score including: A closed contour of the defect area is extracted using a boundary tracking algorithm with an adaptive threshold, and the closed contour is subjected to noise reduction and smoothing to obtain a continuous boundary. A contour point sequence is established, and a distance transform field is constructed based on the contour point sequence. A center line is extracted from the distance transform field using a gradient descent method to construct a center point sequence. Mapping the contour point sequence and the center point sequence to the same coordinate space to construct a topological structure of the defect area, extracting branch nodes and intersections in the topological structure, calculating the connection relationship between adjacent nodes, constructing a topological tree based on the connection relationship, and extracting structural features of the topological tree as topological structure features of the defect area; Normalizing each feature component of the topological structure feature, calculating the correlation coefficient between the normalized feature components, constructing a feature correlation matrix, calculating the distinguishing ability of each feature component based on the feature correlation matrix, determining a feature weight based on the distinguishing ability, and using the feature weight to perform a weighted combination of the feature components to establish a defect feature description; The defect feature description is matched with preset multi-type defect samples, the distance and similarity between the feature descriptions are calculated, a feature matching matrix is ​​constructed, the matching degree between the sample to be tested and each type of defect sample in the feature matching matrix is ​​analyzed, the type with the highest matching degree is determined as the defect type, and its corresponding matching degree is used as the confidence score.

7. The method according to claim 1, characterized in that A dynamic decision matrix is ​​constructed based on the regional enhancement weights and confidence scores. The comprehensive score of each defect region is calculated in the dynamic decision matrix. The defects are graded according to the comprehensive score, and a quantitative indicator of the defect severity is generated. The defect detection results are output, including: Calculating the distribution uniformity of the regional enhancement weight and the fluctuation range of the confidence score, using the distribution uniformity and fluctuation range as scoring weight coefficients, performing weighted fusion on the regional enhancement weight and the confidence score to generate a scoring matrix; Extracting the variation pattern of the regional enhancement weight with position to obtain regional variation characteristics, extracting the stability of the confidence score to obtain reliability characteristics, calculating the dimension weights of the scoring matrix based on the regional variation characteristics and the reliability characteristics, and constructing a dynamic decision matrix based on the dimension weights; Calculating the relative importance of the dimensions in the dynamic decision matrix to generate an importance weight vector, and multiplying the importance weight vector by the scoring matrix to obtain a comprehensive score for each defect area; A hierarchical evaluation standard is constructed for the comprehensive score, a severity distribution is calculated according to the hierarchical evaluation standard, a membership function is used to convert the severity distribution into a severity quantification index, defect levels are divided according to the severity quantification index, and a detection result is output.

8. A deep learning-based machine vision defect real-time detection and classification system, used to implement the method of any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain image data of the surface of the industrial product, calculate the local entropy value and gradient direction consistency of each pixel in the image data, determine the regional enhancement weight according to the local entropy value and gradient direction consistency, and perform regional adaptive enhancement on the image data based on the regional enhancement weight to obtain enhanced image data; The second unit is used to extract the structural features of the enhanced image data, establish a feature transfer sequence, calculate the correlation matrix between features based on the feature transfer sequence, establish a feature optimization path based on the correlation matrix, and progressively fuse the features along the feature optimization path to obtain fused feature data; The third unit is used to generate a probability distribution map of the defect area based on the fused feature data, modify the probability distribution map of the defect area based on the regional enhancement weight, determine the defect area, extract the topological structure features of the defect area, establish a defect feature description, classify the defects based on the defect feature description, and output the defect type and confidence score; The fourth unit is used to construct a dynamic decision matrix based on the regional enhancement weights and confidence scores, calculate the comprehensive score of each defect area in the dynamic decision matrix, grade the defects according to the comprehensive score, generate a quantitative indicator of the defect severity, and output the defect detection results.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images