Method and system for detecting inclusion defects on the surface of drawn steel plate based on machine vision

Abstract

The present invention discloses a method and system for detecting defects of inclusions on the surface of a brushed steel plate based on machine vision, including: S1: obtaining a multi-light source image sequence of the surface of a brushed steel plate and performing brightness standardization; S2: decomposing the image sequence after brightness standardization through a Gaussian-Laplacian pyramid and performing Laplace pyramid fusion and image reconstruction to obtain a fused image; S3: performing directional analysis on the brushed texture in the fused image, extracting the main direction features of the brushed texture, and obtaining a brushed direction consistency feature map; S4: performing feature extraction and feature fusion on the fused image and the brushed direction consistency feature map based on a dual-stream feature extraction network, and performing a global average pooling operation; S5: constructing a defect classification network and performing training to predict the defect category on the surface of a brushed steel plate. The present invention can overcome the problems existing in the existing brushed steel plate surface defect detection technology and accurately detect defects of inclusions on the surface of a brushed steel plate.

Description

Technical Field

[0001] The present invention relates to the technical field of wire drawing steel plate defect detection, and in particular to a method and system for detecting surface inclusion defects of a wire drawing steel plate based on machine vision. Background Art

[0002] Due to its unique surface texture and good mechanical properties, brushed steel plates are widely used in architectural decoration, electrical appliance housings, kitchen equipment and other fields. In the production process of brushed steel plates, due to the influence of factors such as raw materials and processing technology, surface defects such as inclusions, scratches, and indentations are prone to occur. These defects not only affect the appearance quality of the product, but may also lead to a decrease in product performance. Therefore, accurate detection of surface defects of brushed steel plates is of great significance.

[0003] At present, manual visual inspection is widely used in industrial production to detect surface defects of brushed steel plates. This method relies on the subjective experience and judgment of the inspectors. Not only is the inspection efficiency low, but it is also easily affected by factors such as personnel fatigue, making it difficult to ensure the consistency and reliability of the inspection results.

[0004] With the development of machine vision technology, researchers have proposed a variety of image processing-based detection methods for structural surface defect detection, such as edge detection, morphological processing and other technologies, as well as grayscale co-occurrence matrix analysis based on texture features, local binary pattern and other methods. In the prior art, the patent document (CN119090862A) discloses a machine vision-based screw module surface defect online detection method and device, which includes: image acquisition and preprocessing of the screw module surface; decomposition of the preprocessed image into a structure layer and a texture layer; generation of a contour image based on the initial structure layer; texture optimization of the initial texture layer; feature extraction and fusion through an edge-aware multi-level interactive network; and finally defect detection and positioning based on a multi-scale feature map. However, this method only considers the structure and texture features of the image, and uses an edge-aware multi-level interactive network for feature extraction. If this method is directly applied to the detection of inclusion defects on the surface of brushed steel plates, it will be difficult to effectively detect directional defects such as inconsistent drawing directions because it is not specifically designed for the extraction of directional texture features unique to the surface of brushed steel plates. At the same time, this method mainly relies on the visual features of the image when detecting defects, and lacks quantitative analysis of the directional characteristics of the surface of brushed steel plates, which affects the accuracy and reliability of detection. Summary of the invention

[0005] In view of this, the present invention provides a method and system for detecting surface inclusion defects of brushed steel plates based on machine vision, introduces an improved directional filtering algorithm, quantitatively analyzes the brushed texture through anisotropic Gaussian filter kernel group, effectively extracts the brushing direction consistency features, and designs a dual-stream feature extraction network to simultaneously process image features and directional features, and realizes adaptive fusion of features through a channel attention mechanism, aiming to overcome the problems existing in the existing brushed steel plate surface defect detection technology.

[0006] To achieve the above object, the present invention provides a method for detecting surface inclusion defects of a drawn steel plate based on machine vision, comprising the following steps:

[0007] S1: Acquire a multi-light source image sequence of the brushed steel plate surface and perform brightness standardization to obtain a brightness-standardized multi-light source image sequence;

[0008] S2: Decompose the brightness-normalized multi-light source image sequence through Gaussian-Laplacian pyramid, perform Laplacian pyramid fusion and image reconstruction to obtain a fused image;

[0009] S3: Based on the improved directional filtering algorithm, the brushed texture in the fused image is subjected to directional analysis, the main directional features of the brushed texture are extracted, and the brushed directional consistency feature map is obtained;

[0010] S4: Based on the dual-stream feature extraction network, feature extraction and feature fusion are performed on the fused image and the drawing direction consistency feature map, and a global average pooling operation is performed to obtain a global average pooling result, including:

[0011] ResNet18 is used as the backbone network to construct a dual-stream feature extraction network. After removing the fully connected layer of ResNet18, an image feature branch and a directional feature branch are constructed respectively. The image feature branch inputs the fused image and extracts image features through the ResNet18 network with the fully connected layer removed. The directional feature branch inputs the wire drawing direction consistency feature map and extracts directional features through the ResNet18 network with the fully connected layer removed.

[0012] The image features and directional features are spliced ​​in the feature channel dimension; the spliced ​​features are used to implement the channel attention mechanism through the compression and excitation modules to obtain the fused features; the fused features are subjected to global average pooling to obtain the global average pooling results;

[0013] S5: Based on the global average pooling results, a defect classification network is constructed and trained; the trained defect classification network is used to predict the defect category on the surface of the brushed steel plate to be inspected.

[0014] Optionally, the step S1 includes the following steps:

[0015] S11: Six light sources at different angles are evenly distributed above the surface of the brushed steel plate. The angle of the light source is , is the light source number, ;

[0016] Acquire multi-light source image sequences of brushed steel plate surfaces ,in For the Surface images of brushed steel plates collected under different light sources;

[0017] S12: Adaptively normalize the brightness of each image in the multi-light source image sequence of the brushed steel plate surface to obtain a multi-light source image sequence after brightness normalization , specifically:

[0018] ;

[0019] in, and are the horizontal and vertical coordinates of the surface image of the brushed steel plate, respectively; For the The image after brightness standardization under each light source; For the The image after brightness standardization under the light source In pixel coordinates The pixel value at ; For the The image of the brushed steel plate surface collected under the light source is at the pixel coordinates The pixel value at ; and are the target brightness mean and standard deviation, respectively; and Respectively The image of the brushed steel plate surface collected under a light source is expressed in pixel coordinates. As the center, Mean and standard deviation of pixel values ​​within the range; is a natural constant; To adjust the parameters.

[0020] Optionally, the step S2 includes the following steps:

[0021] S21: Use the brightness-normalized multi-light source image sequence to construct a Gaussian pyramid, specifically:

[0022] ;

[0023] in, For the The image after brightness standardization under the light source Gaussian pyramid image of the layer; for In pixel coordinates The pixel value at ; For the pyramid level, , is the total number of layers, when hour, ; for Gaussian kernel weight matrix, , Transpose the matrix; is the Gaussian kernel weight matrix exist the value at the index, and They are Gaussian kernel weight matrices Horizontal and vertical indexes, For the The image after brightness standardization under the light source Gaussian pyramid image of layers In pixel coordinates The pixel value at ;

[0024] S22: Construct a Laplace pyramid, specifically:

[0025] ;

[0026] in, For the The image after brightness standardization under the light source Laplacian pyramid image of the layer; for In pixel coordinates The pixel value at hour, ; and Respectively The image after brightness standardization under the light source Layer and The Gaussian pyramid image of the layer in pixel coordinates The pixel value at ; For extended operations, specifically:

[0027] ;

[0028] in, For the The image after brightness standardization under the light source The Gaussian pyramid image of the layer in pixel coordinates The pixel value at ; This is a round down operation;

[0029] S23: Perform Laplacian pyramid fusion and image reconstruction to obtain a fused image.

[0030] Optionally, the step S23 includes the following steps:

[0031] S231: Average fusion of Laplace pyramid images under different light sources to obtain the fused Laplacian pyramid image of the layer , In pixel coordinates The pixel value at ;

[0032] S232: Obtain the fused image by image reconstruction, wherein the image reconstruction is:

[0033] ;

[0034] in, and Reconstructed Layer and The fused image of the layer; and They are and In pixel coordinates The pixel value at hour, ; The fused image for , For The fused image at that time.

[0035] Optionally, the step S3 includes the following steps:

[0036] S31: Constructing anisotropic Gaussian directional filter kernels , specifically:

[0037] ;

[0038] in, is the filter kernel angle, , the sampling interval is ; and are the horizontal and vertical coordinates of the filter kernel, respectively, and their value range is Integer in ; The filter kernel angle is The filter kernel exist The value at and is the horizontal and vertical standard deviation;

[0039] S32: The fused image Perform directional filtering to obtain the response intensity map in each direction , specifically:

[0040] ;

[0041] in, The filter kernel angle is Response intensity diagram In pixel coordinates The response intensity value at ; The fused image In pixel coordinates The pixel value at ; It is a two-dimensional convolution operation;

[0042] S33: Construct the strongest response map and local main direction map for each pixel coordinate , at all filter kernel angles Corresponding In the graph, the maximum response intensity value is selected as the strongest response graph. In pixel coordinates The response intensity value , and the corresponding filter kernel angle Recorded as local main direction map In pixel coordinates The local principal direction value ;

[0043] S34: Adaptively smooth the local main directional pattern based on the response intensity to obtain a smoothed local main directional pattern , specifically:

[0044] ;

[0045] in, is the smoothed local main direction map In pixel coordinates The local principal direction value at ; , , and The value range is Integer in ;

[0046] S35: Extract the wire drawing direction consistency feature:

[0047] Local main direction map after statistical smoothing The angle with the highest frequency is taken as the overall main direction value of the brushed texture. ; Calculate each pixel The difference between the local main direction value and the overall main direction value at , and the wire drawing direction consistency feature map is obtained In pixel coordinates Pixel value .

[0048] Optionally, the step S4 includes the following steps:

[0049] S41: ResNet18 is used as the backbone network to construct a dual-stream feature extraction network, and the fully connected layer of ResNet18 is removed to construct an image feature branch and a directional feature branch respectively; the dual-stream feature extraction network uses the image feature branch and the directional feature branch to obtain image features respectively and directional characteristics , specifically:

[0050] ;

[0051] in, ResNet18 network with fully connected layers removed;

[0052] S42: Design a feature fusion module and use the channel attention mechanism to perform feature fusion to obtain the fused features , specifically:

[0053] ;

[0054] in, It is the concatenation operation of the feature channel dimension; It is a compression and excitation module used to implement the channel attention mechanism;

[0055] S43: Perform global average pooling operation on the fused features, specifically:

[0056] ;

[0057] in, It is the global average pooling operation; is the global average pooling result.

[0058] Optionally, the step S5 includes the following steps:

[0059] S51: Construct the output layer and loss function of the defect classification network. The output layer of the defect classification network is specifically:

[0060] ;

[0061] in, is a fully connected layer with an output dimension of , is the number of defect categories; is the normalized exponential function, which converts the output of the fully connected layer into a probability distribution; is the probability vector output by the output layer, is the probability vector No. Dimension, for The predicted probability of class defect, ,when When it is 1, it means there is no defect;

[0062] Loss function for defect classification network for:

[0063] ;

[0064] in, It is a label for the real defects on the surface of the brushed steel plate. Class defects, No. dimension is 1, otherwise is 0;

[0065] S52: training a defect classification network using training samples and a random gradient descent method to obtain a trained defect classification network, wherein the training samples are samples marked with real defects existing on the surface of the brushed steel plate;

[0066] S53: Extract the fused image and the drawing direction consistency feature map of the image of the brushed steel plate to be detected and input them into the trained defect classification network to obtain the prediction probability vector of the brushed steel plate to be detected ; According to the predicted probability vector of the brushed steel plate to be tested To judge the defects of the surface inclusions of the wire drawing steel plate, When , the defect category corresponding to the maximum predicted probability is output. When , it is judged as a defect-free category, where is the predicted probability vector The maximum value in .

[0067] The present invention also discloses a machine vision-based surface inclusion defect detection system for a wire drawing steel plate, comprising:

[0068] Standardization module: obtains a multi-light source image sequence of the brushed steel plate surface and performs brightness standardization to obtain a brightness-standardized multi-light source image sequence;

[0069] Fusion module: Decompose the brightness-normalized multi-light source image sequence through Gaussian-Laplacian pyramid, perform Laplacian pyramid fusion and image reconstruction to obtain the fused image;

[0070] Direction analysis module: Based on the improved directional filtering algorithm, the brushed texture in the fused image is subjected to directional analysis, the main direction features of the brushed texture are extracted, and the brushed direction consistency feature map is obtained;

[0071] Feature extraction module: Based on the dual-stream feature extraction network, feature extraction and feature fusion are performed on the fused image and the drawing direction consistency feature map, and a global average pooling operation is performed to obtain the global average pooling result;

[0072] Model training and application module: construct a defect classification network and train it; use the trained defect classification network to predict the defect category on the surface of the brushed steel plate to be tested;

[0073] Beneficial effects:

[0074] The detection method based on multi-light source image sequence proposed in the present invention effectively solves the problem that defect features are not obvious under a single light source by arranging 6 light sources at different angles and adopting an adaptive brightness normalization algorithm. At the same time, the Gaussian-Laplacian pyramid decomposition and fusion strategy is adopted to make full use of the complementary information of multi-light source images, significantly improve the image contrast and detail performance, make the defect features more prominent, and lay a good foundation for subsequent defect detection. This method effectively overcomes the problems of uneven illumination and shadow interference in traditional detection methods, and improves the reliability and accuracy of detection.

[0075] The present invention designs an improved directional filtering algorithm, which realizes the accurate extraction of the main direction of the brushed texture by constructing anisotropic Gaussian directional filtering kernel groups and adaptive smoothing based on response intensity. The algorithm can not only effectively distinguish the defect area from the background texture, but also highlight the abnormal features of the defect area through the directional consistency feature map. This method based on texture direction analysis significantly improves the detection algorithm's ability to identify different types of defects, especially for defects that deviate from the brushed texture direction.

[0076] The present invention adopts a dual-stream feature extraction network structure, inputs the fused image features and directional consistency features into two feature branches for parallel processing, and realizes effective feature fusion through the channel attention mechanism. This dual-stream network structure not only makes full use of the appearance features and directional feature information of the image, but also realizes the complementary advantages of the two types of features through the feature fusion module. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1The present invention is a flowchart of a method for detecting surface inclusion defects of a brushed steel plate based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0078] The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

[0079] Embodiment 1: A method for detecting inclusion defects on the surface of a brushed steel plate based on machine vision, such as Figure 1 As shown, the following steps are included:

[0080] S1: Obtain a multi-light source image sequence of the brushed steel plate surface and perform brightness standardization to obtain a brightness-standardized multi-light source image sequence:

[0081] S11: Six light sources at different angles are evenly distributed above the surface of the brushed steel plate. The specific angles of the light sources are:

[0082] ;

[0083] in, is the light source number, the value range is , is an integer; For the The angle of the light source;

[0084] Acquire multi-light source image sequences of brushed steel plate surfaces ,in For the Surface images of brushed steel plates collected under different light sources;

[0085] S12: Adaptively normalize the brightness of each image in the multi-light source image sequence of the brushed steel plate surface to obtain a multi-light source image sequence after brightness normalization , specifically:

[0086] ;

[0087] in, and are the horizontal and vertical coordinates of the surface image of the brushed steel plate, respectively; For the The image after brightness standardization under each light source; For the The image after brightness standardization under the light source In pixel coordinates The pixel value at ; For the The image of the brushed steel plate surface collected under the light source is at the pixel coordinates The pixel value at ; and are the target brightness mean and standard deviation, which are 128 and 45 in this embodiment; and Respectively The image of the brushed steel plate surface collected under a light source is expressed in pixel coordinates. As the center, Mean and standard deviation of pixel values ​​within the range; is a natural constant; is an adjustment parameter, which is 22.5 in this embodiment.

[0088] In this step, a light source is arranged every sixty degrees to form a full range of lighting coverage on the surface of the brushed steel plate, ensuring that surface defects can be fully displayed under different lighting angles. This multi-angle lighting design effectively solves the problem of blind spots in traditional single light sources, especially for some subtle defects that are difficult to observe at certain angles, such as small scratches or shallow indentations, clear imaging effects can be obtained under lighting at other angles. This step achieves precise adjustment of image brightness by simultaneously considering the brightness mean and standard deviation of the local area. The algorithm uses an exponential function as a weight factor, which can adaptively adjust the normalization intensity according to the contrast characteristics of the local area of ​​the image, ensuring the uniformity of the overall brightness of the image and well preserving the local detail information of the defective area. This adaptive normalization method effectively overcomes the defect of traditional fixed threshold normalization that easily loses detail information.

[0089] S2: Decompose the brightness-normalized multi-light source image sequence through the Gaussian-Laplacian pyramid, perform Laplacian pyramid fusion and image reconstruction to obtain the fused image:

[0090] S21: Use the brightness-normalized multi-light source image sequence to construct a Gaussian pyramid, specifically:

[0091] ;

[0092] in, For the The image after brightness standardization under the light source Gaussian pyramid image of the layer; for In pixel coordinates The pixel value at ; For the pyramid level, , is the total number of layers, in this embodiment ,when hour, ; for Gaussian kernel weight matrix, , Transpose the matrix; is the Gaussian kernel weight matrix exist the value at the index, and They are Gaussian kernel weight matrices Horizontal and vertical indexes, For the The image after brightness standardization under the light source Gaussian pyramid image of layers In pixel coordinates The pixel value at ;

[0093] S22: Construct a Laplace pyramid, specifically:

[0094] ;

[0095] in, For the The image after brightness standardization under the light source Laplacian pyramid image of the layer; for In pixel coordinates The pixel value at hour, ; and Respectively The image after brightness standardization under the light source Layer and The Gaussian pyramid image of the layer in pixel coordinates The pixel value at ; For extended operations, specifically:

[0096] ;

[0097] in, For the The image after brightness standardization under the light source The Gaussian pyramid image of the layer in pixel coordinates The pixel value at ; This is a round down operation;

[0098] S23: Perform Laplacian pyramid fusion and image reconstruction to obtain the fused image:

[0099] S231: Average fusion of Laplacian pyramid images under different light sources, specifically:

[0100] ;

[0101] in, For the fusion of Laplacian pyramid image of the layer; for In pixel coordinates The pixel value at ;

[0102] S232: Obtain the fused image by image reconstruction, wherein the image reconstruction is:

[0103] ;

[0104] in, and Reconstructed Layer and The fused image of the layer; and They are and In pixel coordinates The pixel value at hour, ; The fused image for , For The fused image at that time.

[0105] This step uses a multi-scale analysis method of Gaussian-Laplacian pyramid decomposition, which can effectively capture the feature information of the image at different scales. Through the layer-by-layer downsampling of the Gaussian pyramid, a multi-scale representation of the image from fine to coarse is achieved, while the Laplacian pyramid retains the detail difference information between adjacent scales. This multi-scale decomposition method is particularly suitable for processing the defect characteristics of the surface of brushed steel plates, because defects of different types and sizes often show significant characteristics at different scales. In this step, by fusing the images of each layer of the Laplacian pyramid under six light sources, not only the complementary information of defects under different lighting conditions is retained, but also the influence of image noise is effectively suppressed through multi-scale decomposition. Especially when processing the surface of brushed steel plates with complex texture backgrounds, this fusion method can better highlight the defect characteristics while maintaining the continuity of the background texture.

[0106] S3: Based on the improved directional filtering algorithm, the brushed texture in the fused image is directional analyzed, the main directional features of the brushed texture are extracted, and the brushed directional consistency feature map is obtained:

[0107] S31: Constructing anisotropic Gaussian directional filter kernels , specifically:

[0108] ;

[0109] in, is the filter kernel angle, , the sampling interval is ; and are the horizontal and vertical coordinates of the filter kernel, respectively, and their value range is Integer in ; The filter kernel angle is The filter kernel exist The value at and are the horizontal and vertical standard deviations, which are 2.5 and 4 respectively in this embodiment;

[0110] S32: The fused image Perform directional filtering to obtain the response intensity map in each direction , specifically:

[0111] ;

[0112] in, The filter kernel angle is Response intensity diagram In pixel coordinates The response intensity value at ; The fused image In pixel coordinates The pixel value at ; It is a two-dimensional convolution operation;

[0113] S33: Construct the strongest response map and local main direction map for each pixel coordinate , at all filter kernel angles Corresponding In the graph, the maximum response intensity value is selected as the strongest response graph. In pixel coordinates The response intensity value , and the corresponding filter kernel angle Recorded as local main direction map In pixel coordinates The local principal direction value ;

[0114] S34: Adaptively smooth the local main directional pattern based on the response intensity to obtain a smoothed local main directional pattern , specifically:

[0115] ;

[0116] in, is the smoothed local main direction map In pixel coordinates The local principal direction value at ; , , and The value range is Integer in ;

[0117] S35: Extract the wire drawing direction consistency feature:

[0118] Local main direction map after statistical smoothing The angle with the highest frequency is taken as the overall main direction value of the brushed texture. ; Calculate each pixel The difference between the local main direction value and the overall main direction value at , and the wire drawing direction consistency feature map is obtained In pixel coordinates Pixel value .

[0119] The anisotropic Gaussian directional filter kernel group designed in this step has unique directional selectivity. By setting different lateral and longitudinal standard deviations, the filter kernel has a stronger response characteristic in a specific direction and a weaker response in the vertical direction. This asymmetric filter kernel design is particularly suitable for texture feature extraction on the surface of brushed steel plates, and can accurately capture texture information in different directions. When constructing the local main direction map, this step does not simply perform directional statistics, but introduces response intensity as a weight factor for weighted smoothing. This processing method fully considers the spatial distribution characteristics of texture intensity, so that areas with strong texture features have greater influence in directional decisions, and effectively suppresses the interference of noise and weak texture areas on directional judgment. In particular, the use of a 5×5 local window for smoothing not only ensures the local consistency of directional information, but also retains sufficient directional details.

[0120] S4: Based on the dual-stream feature extraction network, feature extraction and feature fusion are performed on the fused image and the drawing direction consistency feature map, and a global average pooling operation is performed to obtain the global average pooling result:

[0121] ResNet18 is used as the backbone network to construct a dual-stream feature extraction network. After removing the fully connected layer of ResNet18, an image feature branch and a directional feature branch are constructed respectively. The image feature branch inputs the fused image and extracts image features through the ResNet18 network with the fully connected layer removed. The directional feature branch inputs the wire drawing direction consistency feature map and extracts directional features through the ResNet18 network with the fully connected layer removed.

[0122] The image features and directional features are spliced ​​in the feature channel dimension; the spliced ​​features are used to implement the channel attention mechanism through the compression and excitation modules to obtain the fused features; the fused features are subjected to global average pooling to obtain the global average pooling results;

[0123] Specifically, step S4 includes the following steps:

[0124] S41: ResNet18 is used as the backbone network to construct a dual-stream feature extraction network, and the fully connected layer of ResNet18 is removed to construct an image feature branch and a directional feature branch respectively; the dual-stream feature extraction network uses the image feature branch and the directional feature branch to obtain image features respectively and directional characteristics , specifically:

[0125] ;

[0126] in, ResNet18 network with fully connected layers removed;

[0127] S42: Design a feature fusion module and use the channel attention mechanism to perform feature fusion to obtain the fused features , specifically:

[0128] ;

[0129] in, It is the concatenation operation of the feature channel dimension; It is a compression and excitation module used to implement the channel attention mechanism;

[0130] S43: Perform global average pooling operation on the fused features, specifically:

[0131] ;

[0132] in, It is the global average pooling operation; is the global average pooling result.

[0133] This step can process the fused image and the drawing direction consistency feature map simultaneously by constructing the image feature branch and the direction feature branch respectively, thus realizing the parallel extraction of multimodal features. This step can adaptively weight the spliced ​​features by introducing the channel attention mechanism, and can intelligently adjust the importance of different feature channels. This design enables the network to automatically learn and filter the most valuable feature combinations for defect detection, which not only retains the detailed features of the image, but also makes full use of the direction consistency information. Especially when dealing with different types of defects, the network can adaptively adjust the weight ratio of the two features according to the characteristics of the defects.

[0134] S5: Based on the global average pooling results, a defect classification network is constructed and trained; the defect category of the surface of the brushed steel plate to be inspected is predicted using the trained defect classification network:

[0135] S51: Construct the output layer and loss function of the defect classification network. The output layer of the defect classification network is specifically:

[0136] ;

[0137] in, is a fully connected layer with an output dimension of , is the number of defect categories; is the normalized exponential function, which converts the output of the fully connected layer into a probability distribution; is the probability vector output by the output layer, is the probability vector No. Dimension, for The predicted probability of class defect, ,when When it is 1, it means there is no defect;

[0138] Loss function for defect classification network for:

[0139] ;

[0140] in, It is a label for the real defects on the surface of the brushed steel plate. Class defects, No. dimension is 1, otherwise is 0;

[0141] S52: training a defect classification network using training samples and a random gradient descent method to obtain a trained defect classification network, wherein the training samples are samples marked with real defects existing on the surface of the brushed steel plate;

[0142] S53: Extract the fused image and the drawing direction consistency feature map of the image of the brushed steel plate to be detected and input them into the trained defect classification network to obtain the prediction probability vector of the brushed steel plate to be detected ; According to the predicted probability vector of the brushed steel plate to be tested To judge the defects of the surface inclusions of the wire drawing steel plate, When , the defect category corresponding to the maximum predicted probability is output. When , it is judged as a defect-free category, where is the predicted probability vector The maximum value in .

[0143] Embodiment 2: The present invention also discloses a machine vision-based surface inclusion defect detection system for a brushed steel plate, comprising the following five modules:

[0144] Standardization module: obtains a multi-light source image sequence of the brushed steel plate surface and performs brightness standardization to obtain a brightness-standardized multi-light source image sequence;

[0145] Fusion module: Decompose the brightness-normalized multi-light source image sequence through Gaussian-Laplacian pyramid, perform Laplacian pyramid fusion and image reconstruction to obtain the fused image;

[0146] Direction analysis module: Based on the improved directional filtering algorithm, the brushed texture in the fused image is subjected to directional analysis, the main direction features of the brushed texture are extracted, and the brushed direction consistency feature map is obtained;

[0147] Feature extraction module: Based on the dual-stream feature extraction network, feature extraction and feature fusion are performed on the fused image and the drawing direction consistency feature map, and a global average pooling operation is performed to obtain the global average pooling result;

[0148] Model training and application module: construct a defect classification network and train it; use the trained defect classification network to predict the defect category on the surface of the brushed steel plate to be inspected.

[0149] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0150] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0151] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting surface inclusion defects of a brushed steel plate based on machine vision, characterized in that: The following steps are involved: S1: Acquire a multi-light source image sequence of the brushed steel plate surface and perform brightness standardization to obtain a brightness-standardized multi-light source image sequence; S2: Decompose the brightness-normalized multi-light source image sequence through Gaussian-Laplacian pyramid, perform Laplacian pyramid fusion and image reconstruction to obtain a fused image; S3: Based on the improved directional filtering algorithm, the brushed texture in the fused image is subjected to directional analysis, the main directional features of the brushed texture are extracted, and the brushed directional consistency feature map is obtained; S4: Based on the dual-stream feature extraction network, feature extraction and feature fusion are performed on the fused image and the drawing direction consistency feature map, and a global average pooling operation is performed to obtain a global average pooling result, including: ResNet18 is used as the backbone network to construct a dual-stream feature extraction network. After removing the fully connected layer of ResNet18, an image feature branch and a directional feature branch are constructed respectively. The image feature branch inputs the fused image and extracts image features through the ResNet18 network with the fully connected layer removed. The directional feature branch inputs the wire drawing direction consistency feature map and extracts directional features through the ResNet18 network with the fully connected layer removed. The image features and directional features are spliced ​​in the feature channel dimension; the spliced ​​features are used to implement the channel attention mechanism through the compression and excitation modules to obtain the fused features; the fused features are subjected to global average pooling to obtain the global average pooling results; S5: Based on the global average pooling results, a defect classification network is constructed and trained; the trained defect classification network is used to predict the defect category on the surface of the brushed steel plate to be inspected.

2. The method for detecting surface inclusion defects of a drawn steel plate based on machine vision according to claim 1, characterized in that: The step S1 includes the following steps: S11: Six light sources at different angles are evenly distributed above the surface of the brushed steel plate. The angle of the light source is , is the light source number, ; Acquire multi-light source image sequences of brushed steel plate surfaces ,in For the Surface images of brushed steel plates collected under different light sources; S12: Adaptively normalize the brightness of each image in the multi-light source image sequence of the brushed steel plate surface to obtain a multi-light source image sequence after brightness normalization , specifically: ； in, and are the horizontal and vertical coordinates of the surface image of the brushed steel plate, respectively; For the The image after brightness standardization under each light source; For the The image after brightness standardization under the light source In pixel coordinates The pixel value at ; For the The image of the brushed steel plate surface collected under a light source is at pixel coordinates The pixel value at ; and are the target brightness mean and standard deviation, respectively; and Respectively The image of the brushed steel plate surface collected under a light source is expressed in pixel coordinates. As the center, Mean and standard deviation of pixel values ​​within the range; is a natural constant; To adjust the parameters.

3. The method for detecting surface inclusion defects of a drawn steel plate based on machine vision according to claim 2, characterized in that: The step S2 includes the following steps: S21: Use the brightness-normalized multi-light source image sequence to construct a Gaussian pyramid, specifically: ； in, For the The image after brightness standardization under the light source Gaussian pyramid image of the layer; for In pixel coordinates The pixel value at ; For the pyramid level, , is the total number of layers, when hour, ; for Gaussian kernel weight matrix, , Transpose the matrix; is the Gaussian kernel weight matrix exist the value at the index, and They are Gaussian kernel weight matrices Horizontal and vertical indexes, For the The image after brightness standardization under the light source Gaussian pyramid image of layers In pixel coordinates The pixel value at ; S22: Construct a Laplace pyramid, specifically: ； in, For the The image after brightness standardization under the light source Laplacian pyramid image of the layer; for In pixel coordinates The pixel value at hour, ; and Respectively The image after brightness standardization under the light source Layer and The Gaussian pyramid image of the layer in pixel coordinates The pixel value at ; For extended operations, specifically: ； in, For the The image after brightness standardization under the light source The Gaussian pyramid image of the layer in pixel coordinates The pixel value at ; This is a round down operation; S23: Perform Laplacian pyramid fusion and image reconstruction to obtain a fused image.

4. The method for detecting surface inclusion defects of a drawn steel plate based on machine vision according to claim 3 is characterized in that: The step S23 includes the following steps: S231: Average fusion of Laplace pyramid images under different light sources to obtain the fused Laplacian pyramid image of the layer , In pixel coordinates The pixel value at ; S232: Obtain the fused image by image reconstruction, wherein the image reconstruction is: ； in, and Reconstructed Layer and The fused image of the layer; and They are and In pixel coordinates The pixel value at hour, ; The fused image for , For The fused image at that time.

5. The method for detecting surface inclusion defects of a drawn steel plate based on machine vision according to claim 3, characterized in that: The step S3 includes the following steps: S31: Constructing anisotropic Gaussian directional filter kernels , specifically: ； in, is the filter kernel angle, , the sampling interval is ; and are the horizontal and vertical coordinates of the filter kernel, respectively, and their value range is Integer in ; The filter kernel angle is The filter kernel exist The value at and is the horizontal and vertical standard deviation; S32: The fused image Perform directional filtering to obtain the response intensity map in each direction , specifically: ； in, The filter kernel angle is Response intensity diagram In pixel coordinates The response intensity value at ; The fused image In pixel coordinates The pixel value at ; It is a two-dimensional convolution operation; S33: Construct the strongest response map and local main direction map for each pixel coordinate , at all filter kernel angles Corresponding In the graph, the maximum response intensity value is selected as the strongest response graph. In pixel coordinates The response intensity value , and the corresponding filter kernel angle Recorded as local main direction map In pixel coordinates The local principal direction value ; S34: Adaptively smooth the local main directional pattern based on the response intensity to obtain a smoothed local main directional pattern , specifically: ； in, is the smoothed local main direction map In pixel coordinates The local principal direction value at ; , , and The value range is Integer in ; S35: Extract the wire drawing direction consistency feature: Local main direction map after statistical smoothing The angle with the highest frequency is taken as the overall main direction value of the brushed texture. ; Calculate each pixel The difference between the local main direction value and the overall main direction value at , and the wire drawing direction consistency feature map is obtained In pixel coordinates Pixel value .

6. The method for detecting surface inclusion defects of a drawn steel plate based on machine vision according to claim 5, characterized in that: The step S4 includes the following steps: S41: ResNet18 is used as the backbone network to construct a dual-stream feature extraction network, and the fully connected layer of ResNet18 is removed to construct an image feature branch and a directional feature branch respectively; the dual-stream feature extraction network uses the image feature branch and the directional feature branch to obtain image features respectively and directional characteristics , specifically: ； in, ResNet18 network with fully connected layers removed; S42: Design a feature fusion module and use the channel attention mechanism to perform feature fusion to obtain the fused features , specifically: ； in, It is the concatenation operation of the feature channel dimension; It is a compression and excitation module used to implement the channel attention mechanism; S43: Perform global average pooling operation on the fused features, specifically: ； in, It is the global average pooling operation; is the global average pooling result.

7. The method for detecting surface inclusion defects of a drawn steel plate based on machine vision according to claim 6, characterized in that: The step S5 includes the following steps: S51: Construct the output layer and loss function of the defect classification network. The output layer of the defect classification network is specifically: ； in, is a fully connected layer with an output dimension of , is the number of defect categories; is a normalized exponential function that converts the output of the fully connected layer into a probability distribution; is the probability vector output by the output layer, is the probability vector No. Dimension, for The predicted probability of class defect, ,when When it is 1, it means there is no defect; Loss function for defect classification network for: ； in, It is a label for the real defects on the surface of the brushed steel plate. Class defects, No. dimension is 1, otherwise is 0; S52: training a defect classification network using training samples and a random gradient descent method to obtain a trained defect classification network, wherein the training samples are samples marked with real defects existing on the surface of the brushed steel plate; S53: Extract the fused image and the drawing direction consistency feature map of the image of the brushed steel plate to be detected and input them into the trained defect classification network to obtain the prediction probability vector of the brushed steel plate to be detected ; According to the predicted probability vector of the brushed steel plate to be tested To judge the defects of the surface inclusions of the wire drawing steel plate, When , the defect category corresponding to the maximum predicted probability is output. When , it is judged as a defect-free category, where is the predicted probability vector The maximum value in .

8. The machine vision-based surface inclusion defect detection system for brushed steel plates is characterized by: include: Standardization module: obtains a multi-light source image sequence of the brushed steel plate surface and performs brightness standardization to obtain a brightness-standardized multi-light source image sequence; Fusion module: Decompose the brightness-normalized multi-light source image sequence through Gaussian-Laplacian pyramid, perform Laplacian pyramid fusion and image reconstruction to obtain the fused image; Direction analysis module: Based on the improved directional filtering algorithm, the brushed texture in the fused image is subjected to directional analysis, the main direction features of the brushed texture are extracted, and the brushed direction consistency feature map is obtained; Feature extraction module: Based on the dual-stream feature extraction network, feature extraction and feature fusion are performed on the fused image and the drawing direction consistency feature map, and a global average pooling operation is performed to obtain the global average pooling result; Model training and application module: construct a defect classification network and train it; use the trained defect classification network to predict the defect category on the surface of the brushed steel plate to be tested; To realize the method for detecting surface inclusion defects of a drawn steel plate based on machine vision as described in any one of claims 1 to 7.

Images