Strip defect identification and grading control method based on template intelligent detection platform

Abstract

The present application relates to a strip steel defect identification and grading control method based on a template intelligent detection platform, and belongs to the technical field of cold-rolled strip steel template detection. The method comprises the following steps: obtaining two-dimensional images, three-dimensional topography and thickness distribution data of the template and registering them to the same coordinate system; extracting a defect feature map, inputting the weighted fusion data into a data fusion model, and outputting a defect type, geometric parameters and recognition confidence; when the confidence is lower than a threshold, performing review identification, and storing inconsistent samples in a database for labeling; outputting a defect prediction probability of adjacent areas by a spatial correlation prediction model and dynamically adjusting detection parameters; outputting a comprehensive quality grade based on multi-dimensional weighted scoring, generating a grading early warning combined with the prediction probability; when the threshold is reached, generating a control instruction according to a defect-process correlation rule base and sending it to a production line control system; periodically or when the database threshold is reached, incrementally fine-tuning the model and updating the rule base. The method realizes a leap from point recognition to area prediction, forms a closed-loop control, and improves detection accuracy and efficiency.

Description

Technical Field

[0001] This invention belongs to the field of cold-rolled strip steel sample inspection technology, specifically involving a method for strip steel defect identification and graded control based on a sample intelligent inspection platform. Background Technology

[0002] The shape, width, thickness, and surface quality of cold-rolled strip steel products are crucial quality indicators. Frequent sampling and testing are necessary during production to monitor product quality and guide process adjustments. Currently, the cold-rolling industry largely relies on manual methods for sample inspection. Operators place the sampled strips on a testing platform and use traditional measuring tools such as calipers and micrometers to measure width and thickness. Surface defects are then observed visually or after polishing with an oilstone. This inspection method suffers from problems such as high workload, low efficiency, strong subjectivity, and susceptibility to data recording errors, making it difficult to meet the requirements of smart manufacturing for real-time performance, accuracy, and traceability.

[0003] In recent years, some companies have attempted to replace manual inspection with automated testing equipment. Existing technologies include several sample inspection platforms or systems, such as those using contact sensors to detect plate shape, laser sensors to measure width and thickness, and cameras to capture surface images. However, these existing technologies still have the following shortcomings: First, the detection function is limited. Most solutions can only detect one or a few of the following: shape, width, thickness, or surface defects. They cannot perform comprehensive detection of geometric dimensions, shape, original surface defects, and hidden defects after grinding on the same platform, resulting in a fragmented and inefficient detection process.

[0004] Second, the detection results only output alarm signals and cannot automatically identify defect types, quantify defect levels, or correlate detection results with production process parameters to form a closed-loop control. Operators still need to manually determine process adjustment strategies based on alarm information, resulting in a delayed response and making it difficult to achieve adaptive optimization of the production line.

[0005] Third, the detection parameters are fixed. Existing systems typically use uniform scanning resolution, exposure time, and transport speed to uniformly inspect the entire surface of the sample, without considering the spatial distribution and correlation of defects. This leads to over-inspection of low-risk areas, wasting resources, while high-risk areas may be missed due to insufficient detection parameters.

[0006] Fourth, it lacks self-evaluation and self-correction capabilities. When using deep learning models for defect identification, the model may output erroneous results with high confidence when encountering out-of-distribution samples or unknown defect types. The system cannot detect this error, nor can it actively review or correct it, thus reducing the reliability of the detection system.

[0007] Fifth, it lacks continuous learning capabilities. The large amount of data generated during the detection process is not effectively accumulated for model optimization, the system's detection performance cannot improve with running time, and it has poor adaptability to newly emerging defect types.

[0008] To address the aforementioned issues, there is an urgent need to provide a sample inspection method capable of intelligent defect identification, multi-dimensional hierarchical evaluation, spatial propagation prediction, and process closed-loop control, in order to overcome the shortcomings of existing technologies such as incomplete detection, lack of self-learning ability, and lack of process linkage. Summary of the Invention

[0009] To address the aforementioned problems in the existing technology, this invention provides a method for strip steel defect identification and graded control based on a template intelligent inspection platform. The objective of this invention can be achieved through the following technical solutions: A method for strip steel defect identification and graded control based on a template intelligent inspection platform includes: Acquire the two-dimensional image, three-dimensional morphology, and thickness distribution data of the template and register them to the same coordinate system; extract the defect feature maps of the three types of data, and input them into the data fusion model after weighted fusion to output the defect type, geometric parameters, and recognition confidence. When the confidence level is lower than the threshold, a review identification is performed on the defective area, and samples whose review results are inconsistent with the original results are stored in the unlabeled database. The system uses identified defects as nodes and constructs a topology based on the spatial relationships between defects. The defect spatial correlation prediction model outputs the predicted probability of defects in adjacent areas, and the subsequent detection parameters are dynamically adjusted accordingly. At the same time, the system performs weighted scoring based on multiple evaluation dimensions of defects, outputs a comprehensive quality level, and generates graded early warnings by combining the predicted probabilities. When the overall quality level or graded early warning reaches the automatic intervention threshold, a control command is generated based on the defect-process association rule base and sent to the production line control system; otherwise, the detection results and process adjustment suggestions are pushed to the operation terminal. According to a preset cycle, or when the number of samples in the database to be labeled reaches a preset sample threshold, the data fusion model and the defect space association prediction model are incrementally fine-tuned, and the rule base is updated.

[0010] As a preferred embodiment of the present invention, the weighted fusion is implemented using an attention mechanism, which is a channel attention module, a spatial attention module, or a combination of both. The channel attention module extracts a global description of the feature map through global average pooling and global max pooling, maps it through a shared multilayer perceptron, adds it element-wise, and then activates it with a sigmoid function to generate channel weights. The spatial attention module performs channel-dimensional average pooling and max pooling on the feature map, concatenates it, and then activates it with a convolutional layer and a sigmoid function to generate spatial weights. During weighted fusion, the attention weights are multiplied element-wise with the original feature map.

[0011] Specifically, the data fusion model is a multi-branch convolutional neural network, including: a first branch is a two-dimensional convolutional neural network that receives two-dimensional image data; a second branch is a one-dimensional or three-dimensional convolutional neural network that receives three-dimensional shape data; and a third branch is a fully connected network or a one-dimensional convolutional network that receives thickness distribution data. The feature vectors output by the three branches are weighted and fused, then concatenated, and then output through a fully connected layer and a Softmax layer to output the probability distribution of defect types. At the same time, the defect bounding box parameters and recognition confidence are output through a regression branch.

[0012] Specifically, the verification and identification includes at least one of the following operations: The control line scan camera and 3D laser camera perform a second high-resolution scan of the defect area, and the new data is re-input into the data fusion model for secondary identification. The backup classifier, which is pre-trained based on historical defect data, is invoked to identify the features of the defect area. The backup classifier includes support vector machine, random forest or shallow neural network. The multimodal data and initial identification results of the defective area are pushed to the manual review terminal; when multiple review operations coexist, a voting or priority mechanism is used to determine the final review result.

[0013] Specifically, the topology is a graph structure, with each defect as a node, its spatial coordinates and defect features are extracted; the Euclidean distance between any two nodes is calculated, and when the distance is less than a preset neighborhood radius, an edge connection is established to form an undirected graph; or the K nearest neighbor method is used to connect each node to the K nearest other nodes, and the K value is dynamically determined according to the defect density; for isolated nodes, a self-loop edge is used to connect them to themselves.

[0014] Specifically, the defect spatial association prediction model is a graph convolutional network or a graph attention network. The graph convolutional network normalizes the adjacency matrix and multiplies it with the node feature matrix, then updates the node representation through linear transformation and nonlinear activation. The graph attention network learns attention coefficients for each neighboring node and sums them by weighted coefficients. The model input is the node feature matrix and the adjacency matrix, and the output is the defect prediction probability of adjacent regions. The row vector of the node feature matrix is ​​concatenated by the defect type code, geometric size, confidence level and location coordinates and mapped to a preset dimension through a fully connected layer.

[0015] Specifically, the dynamic adjustment of subsequent detection parameters includes: setting a first threshold and a second threshold, wherein the first threshold is less than the second threshold; when the predicted probability exceeds the first threshold but is lower than the second threshold, increasing the scanning frame rate of the line scan camera or reducing the sample delivery speed; when the predicted probability exceeds the second threshold, triggering the 3D laser camera to perform encrypted scanning of the area and supplementing the encrypted scanning data to the input of the data fusion model in real time; when the predicted probability is lower than the first threshold, maintaining the normal detection parameters.

[0016] Specifically, the multiple evaluation dimensions include at least five dimensions: defect type, size, density, location, and process impact. A pre-defined percentage scoring standard is used for each dimension. Defect type is assigned a value based on a severity table; defect size is linearly mapped by area or length; defect density is scored in segments based on quantity; defect location is assigned different weights based on whether it is the edge, center, working side, or transmission side; and process impact is assigned a value based on its influence on subsequent processes. The comprehensive defect index is obtained by summing the scores of each dimension after multiplying them by weighting coefficients. The weighting coefficients are determined using either the analytic hierarchy process (AHP) or the entropy weighting method, depending on the data characteristics.

[0017] Specifically, the registration to the same coordinate system includes: establishing a world coordinate system with the fixed reference point of the detection platform as the origin, obtaining the external parameter matrix of each sensor through calibration, and transforming the data from pixel or sensor coordinates to the world coordinate system; for thickness distribution data, spatial position interpolation is performed according to the sample conveying speed and timestamp to keep the sampling interval consistent with the two-dimensional and three-dimensional data.

[0018] Specifically, the defect-process association rule base is constructed using decision tree or association rule mining algorithms. Each rule includes the defect type, defect feature range, and corresponding process parameter adjustment suggestions. The decision tree uses the defect type as the prediction target and extracts the process parameter threshold through information gain or Gini coefficient split nodes. The association rule mining uses Apriori or FP-Growth algorithms to mine frequent itemsets between the defect type and the process parameter value range. When automatic intervention is triggered, the control command generated according to the defect type and comprehensive quality level matching rules includes at least one of the following: roll force adjustment value, rolling speed correction amount, roll coolant flow rate adjustment value, or tension setting value.

[0019] Specifically, the incremental fine-tuning employs either empirical replay or elastic weight consolidation algorithms. The empirical replay method maintains a fixed-size replay buffer and trains the model by randomly selecting historical samples from the database to be labeled and mixing them with new samples. The elastic weight consolidation method calculates the importance weights of parameters after incremental learning and applies a secondary penalty term to the updates of important parameters. The triggering condition for incremental fine-tuning is that the number of samples in the database to be labeled reaches a preset threshold or that the time elapsed since the last fine-tuning exceeds a preset duration. After triggering, the database to be labeled is cleared or marked.

[0020] Specifically, the identification confidence level is the maximum value of the Softmax probability distribution output by the data fusion model; when the maximum value is lower than the preset confidence level threshold, a review identification is triggered; the confidence level threshold ranges from 0.7 to 0.9, and is adaptively adjusted according to the steel type, thickness, or surface roughness of the sample.

[0021] The beneficial effects of this invention are as follows: This invention achieves multimodal data fusion to improve defect identification accuracy: It simultaneously acquires two-dimensional images, three-dimensional morphology, and thickness distribution data of a sample, and deeply fuses these three types of heterogeneous data at the feature level through a weighted fusion mechanism. Two-dimensional images can reflect surface texture and color difference defects, three-dimensional morphology can detect depth information such as waviness and indentation, and thickness distribution can identify uneven thickness and edge drop issues. These three types of data complement each other, overcoming the limitations of single-sensor detection and significantly improving the accuracy and detection rate of defect identification, especially for defects with insignificant surface color differences but abnormal three-dimensional morphology.

[0022] This invention enhances system reliability by introducing a confidence assessment and intelligent verification mechanism: The deep learning model outputs a confidence score along with the recognition result. When the confidence score falls below a preset threshold, a verification mechanism is automatically triggered, including a second high-resolution scan, voting by a backup classifier, or manual review. This design effectively solves the problem that deep learning models may output high-confidence erroneous results when encountering out-of-distribution samples or unknown defects, enabling the system to have self-assessment and self-correction capabilities, greatly reducing the risk of missed and false detections.

[0023] This invention achieves defect spatial propagation prediction and dynamic adjustment of detection parameters, moving from point detection to area prediction: It constructs a topology structure using identified defects as nodes, outputs the predicted probability of defects in adjacent areas through a defect spatial association prediction model, and dynamically adjusts subsequent detection parameters accordingly. Unlike existing technologies that only perform post-processing identification of collected areas, this invention can proactively predict the defect risk in undetected areas, enabling an adaptive detection strategy of intensive scanning of high-risk areas and routine scanning of low-risk areas. While maintaining the total detection time, it can significantly improve the defect detection rate in key areas while avoiding resource waste.

[0024] This invention constructs a multi-dimensional hierarchical evaluation and closed-loop process control system: Based on five dimensions—defect type, size, density, location, and process impact—this invention weights and scores defects, outputting a comprehensive quality level, and generates tiered early warnings by incorporating spatial prediction probabilities. When the quality level or early warning reaches an intervention threshold, control commands are automatically generated based on a defect-process association rule base and sent to the production line control system, achieving an automatic closed loop from detection results to process adjustments. Compared to existing technologies that only output alarm signals and require manual judgment and adjustment, this invention significantly shortens response time and provides technical support for adaptive optimization of the production line.

[0025] An incremental learning and knowledge base iteration mechanism is established to achieve continuous system evolution: This invention stores samples that are inconsistent with the original results after verification into a database to be labeled. Periodically, or when the database reaches a threshold, an incremental learning algorithm is used to fine-tune the model, and the defect-process association rule base is updated synchronously. With accumulated running time, the system's recognition accuracy and process control capabilities continuously improve, exhibiting good adaptability to newly emerging defect types, overcoming the traditional model's "performance degradation after deployment" defect. Attached Figure Description

[0026] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0027] Figure 1 This is a flowchart illustrating the overall method of the present invention; Figure 2 This is a flowchart of the multimodal data acquisition, registration, and fusion recognition process of the present invention; Figure 3 This is a flowchart of the confidence assessment and intelligent review process of the present invention; Figure 4 This is a flowchart of the defect space topology construction and propagation prediction process of the present invention; Figure 5 This is a flowchart of the multi-dimensional hierarchical evaluation and closed-loop control process of the present invention. Detailed Implementation

[0028] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0029] To facilitate understanding of this invention, some terms are explained below: The “sample intelligent inspection platform” refers to a complete system used to achieve sample inspection, including but not limited to sampling robotic arms, line scan cameras, 3D laser cameras, inspection platforms and data processing units.

[0030] "Testing platform" refers to a component of the intelligent testing platform for samples, specifically the table surface used to place and transport samples.

[0031] Please see Figures 1-5 A method for strip steel defect identification and graded control based on a template intelligent inspection platform includes: Acquire the two-dimensional image, three-dimensional morphology, and thickness distribution data of the template and register them to the same coordinate system; extract the defect feature maps of the three types of data, and input them into the data fusion model after weighted fusion to output the defect type, geometric parameters, and recognition confidence. When the confidence level is lower than the threshold, a review identification is performed on the defective area, and samples whose review results are inconsistent with the original results are stored in the unlabeled database. The system uses identified defects as nodes and constructs a topology based on the spatial relationships between defects. The defect spatial correlation prediction model outputs the predicted probability of defects in adjacent areas, and the subsequent detection parameters are dynamically adjusted accordingly. At the same time, the system performs weighted scoring based on multiple evaluation dimensions of defects, outputs a comprehensive quality level, and generates graded early warnings by combining the predicted probabilities. When the overall quality level or graded early warning reaches the automatic intervention threshold, a control command is generated based on the defect-process association rule base and sent to the production line control system; otherwise, the detection results and process adjustment suggestions are pushed to the operation terminal. According to a preset cycle, or when the number of samples in the database to be labeled reaches a preset sample threshold, the data fusion model and the defect space association prediction model are incrementally fine-tuned, and the rule base is updated.

[0032] Specifically, the weighted fusion is implemented using an attention mechanism, which is a channel attention module, a spatial attention module, or a combination of both. The channel attention module extracts a global description of the feature map through global average pooling and global max pooling, maps it through a shared multilayer perceptron, adds it element-wise, and then activates it with a sigmoid function to generate channel weights. The spatial attention module performs channel-dimensional average pooling and max pooling on the feature map, concatenates them, and then activates them with a convolutional layer and a sigmoid function to generate spatial weights. During weighted fusion, the attention weights are multiplied element-wise with the original feature map.

[0033] Specifically, the data fusion model is a multi-branch convolutional neural network, including: a first branch is a two-dimensional convolutional neural network that receives two-dimensional image data; a second branch is a one-dimensional or three-dimensional convolutional neural network that receives three-dimensional shape data; and a third branch is a fully connected network or a one-dimensional convolutional network that receives thickness distribution data. The feature vectors output by the three branches are weighted and fused, then concatenated, and then output through a fully connected layer and a Softmax layer to output the probability distribution of defect types. At the same time, the defect bounding box parameters and recognition confidence are output through a regression branch.

[0034] Specifically, the verification and identification includes at least one of the following operations: The control line scan camera and 3D laser camera perform a second high-resolution scan of the defect area, and the new data is re-input into the data fusion model for secondary identification. The backup classifier, which is pre-trained based on historical defect data, is invoked to identify the features of the defect area. The backup classifier includes support vector machine, random forest or shallow neural network. The multimodal data and initial identification results of the defective area are pushed to the manual review terminal; when multiple review operations coexist, a voting or priority mechanism is used to determine the final review result.

[0035] Specifically, the topology is a graph structure, with each defect as a node, its spatial coordinates and defect features are extracted; the Euclidean distance between any two nodes is calculated, and when the distance is less than a preset neighborhood radius, an edge connection is established to form an undirected graph; or the K nearest neighbor method is used to connect each node to the K nearest other nodes, and the K value is dynamically determined according to the defect density; for isolated nodes, a self-loop edge is used to connect them to themselves.

[0036] Specifically, the defect spatial association prediction model is a graph convolutional network or a graph attention network. The graph convolutional network normalizes the adjacency matrix and multiplies it with the node feature matrix, then updates the node representation through linear transformation and nonlinear activation. The graph attention network learns attention coefficients for each neighboring node and sums them by weighted coefficients. The model input is the node feature matrix and the adjacency matrix, and the output is the defect prediction probability of adjacent regions. The row vector of the node feature matrix is ​​concatenated by the defect type code, geometric size, confidence level and location coordinates and mapped to a preset dimension through a fully connected layer.

[0037] Specifically, the dynamic adjustment of subsequent detection parameters includes: setting a first threshold and a second threshold, wherein the first threshold is less than the second threshold; when the predicted probability exceeds the first threshold but is lower than the second threshold, increasing the scanning frame rate of the line scan camera or reducing the sample delivery speed; when the predicted probability exceeds the second threshold, triggering the 3D laser camera to perform encrypted scanning of the area and supplementing the encrypted scanning data to the input of the data fusion model in real time; when the predicted probability is lower than the first threshold, maintaining the normal detection parameters.

[0038] Specifically, the multiple evaluation dimensions include at least five dimensions: defect type, size, density, location, and process impact. A pre-defined percentage scoring standard is used for each dimension. Defect type is assigned a value based on a severity table; defect size is linearly mapped by area or length; defect density is scored in segments based on quantity; defect location is assigned different weights based on whether it is the edge, center, working side, or transmission side; and process impact is assigned a value based on its influence on subsequent processes. The comprehensive defect index is obtained by summing the scores of each dimension after multiplying them by weighting coefficients. The weighting coefficients are determined using either the analytic hierarchy process (AHP) or the entropy weighting method, depending on the data characteristics.

[0039] Specifically, the registration to the same coordinate system includes: establishing a world coordinate system with the fixed reference point of the detection platform as the origin, obtaining the external parameter matrix of each sensor through calibration, and transforming the data from pixel or sensor coordinates to the world coordinate system; for thickness distribution data, spatial position interpolation is performed according to the sample conveying speed and timestamp to keep the sampling interval consistent with the two-dimensional and three-dimensional data.

[0040] Specifically, the defect-process association rule base is constructed using decision tree or association rule mining algorithms. Each rule includes the defect type, defect feature range, and corresponding process parameter adjustment suggestions. The decision tree uses the defect type as the prediction target and extracts the process parameter threshold through information gain or Gini coefficient split nodes. The association rule mining uses Apriori or FP-Growth algorithms to mine frequent itemsets between the defect type and the process parameter value range. When automatic intervention is triggered, the control command generated according to the defect type and comprehensive quality level matching rules includes at least one of the following: roll force adjustment value, rolling speed correction amount, roll coolant flow rate adjustment value, or tension setting value.

[0041] Specifically, the incremental fine-tuning employs either empirical replay or elastic weight consolidation algorithms. The empirical replay method maintains a fixed-size replay buffer and trains the model by randomly selecting historical samples from the database to be labeled and mixing them with new samples. The elastic weight consolidation method calculates the importance weights of parameters after incremental learning and applies a secondary penalty term to the updates of important parameters. The triggering condition for incremental fine-tuning is that the number of samples in the database to be labeled reaches a preset threshold or that the time elapsed since the last fine-tuning exceeds a preset duration. After triggering, the database to be labeled is cleared or marked.

[0042] Specifically, the identification confidence level is the maximum value of the Softmax probability distribution output by the data fusion model; when the maximum value is lower than the preset confidence level threshold, a review identification is triggered; the confidence level threshold ranges from 0.7 to 0.9, and is adaptively adjusted according to the steel type, thickness, or surface roughness of the sample.

[0043] Example This embodiment takes a strip steel sample of a certain specification produced by a cold rolling mill as an example. The sample dimensions are 1200mm in length, 800mm in width, and 0.8mm in thickness.

[0044] I. Detection Preparation and Multimodal Data Acquisition After the testing system is started, the PLC receives the steel coil number, nominal width, and nominal thickness sent by the host computer. The sampling robotic arm grabs a sample from the production line shearing station and places it at the predetermined starting position on the testing platform. A conveyor belt is installed on the surface of the testing platform to move the sample at a uniform speed, which is set to 100 mm / s.

[0045] A line scan camera, a 3D laser camera, and a coaxial cable thickness gauge are sequentially arranged along the template's movement path. The line scan camera system consists of four coaxially mounted CCD cameras, each covering a 200mm width, stitched together to cover the entire template surface. The sampling resolution is set to 50μm / pixel, used to acquire two-dimensional grayscale images of the template. The 3D laser camera system is divided into two groups, located on the working side and the transmission side above the template, respectively. It uses a red line laser to project onto the template surface, acquiring three-dimensional point cloud data of the template surface through triangulation. The lateral sampling interval is 0.5mm, and the longitudinal sampling interval is 1.0mm, used to detect the template shape (wave height, steepness) and width. The coaxial cable thickness gauge consists of two symmetrically arranged sets of laser displacement sensors, coaxially aligned, measuring the template thickness with a measurement accuracy of ±1μm and a sampling frequency of 200Hz.

[0046] A conveyor belt moves the template, and three types of sensors simultaneously trigger data acquisition. The line scan camera acquires one line of image per trigger, the 3D laser camera acquires one contour line per trigger, and the thickness gauge records a thickness value every 5ms. Simultaneously, the system records the template displacement via an encoder, providing spatial location information for subsequent data registration.

[0047] II. Data Registration and Coordinate System Establishment A world coordinate system is established with the fixed reference point at the upper left corner of the testing platform as the origin, the X-axis along the direction of template movement, the Y-axis perpendicular to the direction of movement, and the Z-axis perpendicular to the plate surface upwards.

[0048] The extrinsic parameter matrices (rotation matrix R and translation vector T) of the line scan camera and 3D laser camera are pre-obtained using a checkerboard calibration method, transforming the data collected by each sensor from pixel coordinates or sensor coordinates to the world coordinate system. For thickness measurement data, since the thickness gauge uses a point measurement method, spatial position interpolation needs to be performed on each thickness measurement point based on the sample transport speed and acquisition timestamp to ensure consistency with the spatial sampling interval of the 2D image and 3D topography data. This embodiment uses linear interpolation, with the interpolated thickness data spaced 1.0 mm in the X direction and the Y direction interval determined according to the transverse scanning range of the probe (the probe is mounted on the transverse module and scans stepwise along the Y direction).

[0049] After registration, each spatial location (x, y) on the template surface can obtain three types of data: gray value I(x, y), height value H(x, y), and thickness value T(x, y), forming a multimodal feature vector.

[0050] III. Multimodal Feature Weighted Fusion and Defect Recognition Defect feature maps are extracted from three types of data. For two-dimensional image data, a pre-trained convolutional neural network (such as the first four layers of ResNet18, containing 2-5 convolutional layers and 2 pooling layers) is used to extract multi-scale texture features, outputting feature map F.img The dimensions are 64×64×128. For 3D topographic data, a one-dimensional convolutional neural network (containing 2 to 5 convolutional layers) is used to extract the fluctuation features of the height curve along the X direction, and the output feature map F is generated. 3d The dimensions are 64×64×64. For thickness distribution data, a fully connected network is used to extract statistical features of the thickness values ​​(mean, variance, edge descent slope), and the output feature vector F is calculated. thk The dimension is 128.

[0051] This embodiment uses a channel attention module (SE module, preferably a channel attention module) to perform weighted fusion of the three types of features. Specifically, F img F 3d and F thk The initial fused feature map F is obtained by concatenating the features along the channel dimension. concat The SE module uses global average pooling to optimize F. concat The global description vector is compressed to 1×1×C, passed through two fully connected layers (the first fully connected layer has C / 16 neurons, and the second fully connected layer restores it to C), and activated by a sigmoid function to generate attention weights for each channel. Finally, the attention weights are compared with F. concat Multiplying each channel sequentially yields the weighted fused feature map F. fusion .

[0052] F fusion Input data fusion model (this embodiment uses a multi-branch convolutional neural network). The model output includes: (1) the probability distribution of defect types, output through the Softmax layer, including the probability values ​​of 5 types of defects such as scratches, roller marks, vibration marks, color difference, and inclusions; (2) the parameters of the defect bounding box (center coordinates, width, height); (3) the recognition confidence, which is the maximum probability value output by Softmax.

[0053] For example, in a certain inspection, the model identified a defect on the left side of the sample (x=150mm, y=50mm), outputting the defect type as "roller mark", with a probability distribution of [scratches: 0.05, roller mark: 0.92, vibration marks: 0.02, color difference: 0.01, inclusions: 0.00], and a confidence level of 0.92, which is higher than the preset threshold of 0.8. The system accepts this identification result.

[0054] IV. Confidence Assessment and Intelligent Review If the confidence level is lower than a preset threshold (e.g., 0.8), the system automatically triggers a review mechanism. This embodiment provides three review operations, and one or more can be selected depending on the situation: Operation 1 (Secondary Scan): The control line scan camera and 3D laser camera perform a secondary high-resolution scan of the defect area. The newly acquired multimodal data is then re-inputted into the data fusion model for secondary identification. For example, if the initial identification confidence of a defect is only 0.65, after the secondary scan, the resolution is increased to 20μm / pixel, and the model re-identifies the defect with a confidence level of 0.89. The secondary identification result is then accepted.

[0055] Operation 2 (Backup Classifier Voting): A backup classifier (in this example, a random forest containing 100 decision trees) pre-trained based on historical defect data is invoked to discriminate the features of the defect region. The backup classifier outputs the voting result. If the backup classifier result is consistent with the original model result, it is retained; otherwise, the voting mechanism is initiated.

[0056] Operation 3 (Manual Review): The multimodal data and initial identification results of the defective area are pushed to the manual review terminal, where quality inspectors confirm or correct the defect labels on the terminal, and the system receives the correct labels manually marked.

[0057] When multiple review operations coexist, a priority mechanism is adopted: the results of manual review have the highest priority, followed by the results of secondary scanning, and finally the voting results of the backup classifier.

[0058] Samples whose verification results are inconsistent with the original identification results (including samples that were incorrectly identified by the model but corrected manually, samples that were missed by the model but were discovered in a second scan, etc.) are stored in the unlabeled database for subsequent incremental training of the model.

[0059] V. Defect Space Topology Construction and Propagation Prediction Construct a graph structure using the identified defects as nodes. Assume a total of M defects have been identified in the current template, and record the spatial coordinates (x, y, z) of each defect i. i y i The defects are identified by their characteristics (type encoding, size, confidence level). Edge connections are established using the K-nearest neighbor method: the Euclidean distance between any two defects is calculated, and each node is connected to the K nearest other nodes. In this embodiment, the value of K is dynamically determined based on the defect density; the higher the density, the larger the value of K. The value of K ranges from 3 to 10. For isolated nodes, self-loop edges are used to connect them to themselves.

[0060] After constructing the graph structure, the node feature matrix X (size M×F, where F is the feature dimension) and the adjacency matrix A (size M×M) are input into the defect space association prediction model. This embodiment uses a two-layer graph convolutional network (GCN). The row vectors of the node feature matrix are concatenated by defect type encoding (one-hot encoding, 5-dimensional), defect geometric dimensions (length, width, 2-dimensional), defect confidence (1-dimensional), and defect location coordinates (x, y, 2-dimensional), and then mapped to a 64-dimensional feature space through a fully connected layer.

[0061] GCN outputs the predicted probability p of defects occurring in the adjacent regions of each node. i (A real number between 0 and 1). For example, after identifying a scratch defect in the middle of a template, the model predicts that the probability of a scratch appearing in the adjacent area within 10 mm to its right is 0.85, indicating that there is a high risk in that area.

[0062] VI. Dynamically adjust subsequent testing parameters The scanning parameters for subsequent detection areas are dynamically adjusted based on the predicted probability. The preset first threshold θ1 = 0.5 and the second threshold θ2 = 0.8 are: When the predicted probability p < θ1, maintain the normal detection parameters (scanning resolution 50 μm / pixel, transport speed 100 mm / s).

[0063] When θ1 ≤ p < θ2, increase the scanning frame rate of the line scan camera from 10kHz to 20kHz, or decrease the sample transport speed from 100mm / s to 60mm / s to obtain denser sampling in this area.

[0064] When p ≥ θ2, the 3D laser camera is triggered to perform a high-precision scan of the area (the horizontal sampling interval is increased from 0.5mm to 0.1mm, and the vertical sampling interval is increased from 1.0mm to 0.2mm). The three-dimensional topographic data obtained by the high-precision scan is then added to the input of the data fusion model in real time, replacing the original low-resolution data and achieving high-precision detection of key areas.

[0065] VII. Multi-dimensional hierarchical evaluation and early warning Defects are scored using a weighted average based on five dimensions: defect type, size, density, location, and impact on the production process. Each dimension has a pre-defined scoring range of 0-100 points. Defect types: scratches 85 points, roller prints 75 points, vibration marks 80 points, color difference 45 points, inclusions 90 points (the higher the score, the more serious the defect).

[0066] Defect size: linearly mapped by area, area 0-10mm 2 Mapping 0-50 points, 10-50mm 2 Mapping 50-80 points, >50mm 2 Mapping score: 80-100.

[0067] Defect density: The number of defects per unit area (100mm×100mm). 0 defects: 0 points; 1-3 defects: 30 points; 4-6 defects: 60 points; 7 or more defects: 100 points.

[0068] Defect location: edge (<50mm from the edge) weight 1.2, middle weight 0.8, working side weight 1.0, transmission side weight 1.0.

[0069] Process impact: The degree of impact of the defect on subsequent annealing, leveling and coating processes is assigned a score, with no impact 0 points, slight impact 40 points, moderate impact 70 points, and severe impact 100 points.

[0070] The comprehensive defect index is obtained by summing the scores of each dimension after multiplying them by their corresponding weighting coefficients. In this embodiment, the Analytic Hierarchy Process (AHP) is used to determine the weighting coefficients: defect type 0.30, size 0.25, density 0.15, location 0.10, and process influence 0.20. After calculation, the comprehensive defect index is mapped to five quality levels: A, B, C, D, and E, according to a preset range.

[0071] Simultaneously, a tiered early warning system is generated based on spatial prediction probabilities: a yellow warning is generated when the overall quality level is C or above and the prediction probability exceeds 0.5; a red warning is generated when the overall quality level is D or above or the prediction probability exceeds 0.8.

[0072] 8. Closed-loop feedback control A defect-process association rule base is established, and rules are mined from historical inspection data using a decision tree algorithm. The rule base in this embodiment includes the following example rules: If the defect type is "roll mark" and the defect size is >5mm, it is recommended to "inspect the surface condition of the roll and replace the roll if necessary", and the control instruction is "increase the roll coolant flow rate" (for example, if the defect is a roll mark and the size is >5mm, then increase the roll coolant flow rate).

[0073] If the defect type is "vibration marks" and the predicted probability is >0.7, it is recommended to "reduce the rolling speed by 5% and adjust the tension setting value by ±2kN". The control command is "reduce the rolling speed by 0.3m / s and increase the front tension by 1.5kN".

[0074] If the defect type is "edge waviness" and the overall quality level is ≥D, it is recommended to "adjust the bending roll force" and the control command is "increase the bending roll force of the work roll by 50kN".

[0075] When the overall quality level or graded early warning reaches the automatic intervention threshold (set to level D or red warning in this embodiment), the system automatically matches the corresponding rule in the rule base, generates a control command, and sends it to the production line control system (PLC) via the OPC UA protocol. After executing the command, the production line control system feeds back the execution result to this system, forming a closed loop.

[0076] If the automatic intervention threshold is not reached, the system will push the detection results and process adjustment suggestions to the operation terminal, which will then be manually adjusted by the operator after confirmation.

[0077] IX. Incremental Learning and Model Iteration The annotation database is used to store samples whose verification results are inconsistent with the original recognition results. Incremental learning is triggered when the number of samples in the database reaches a preset threshold (e.g., 1000) or when more than 7 days have passed since the last incremental fine-tuning.

[0078] This embodiment employs an empirical replay method for incremental fine-tuning: a replay buffer with a capacity of 2000 samples is maintained. Historical samples, roughly equivalent in number to the current new samples, are randomly selected from the database to be labeled and mixed with the new samples to form a training set. The data fusion model and the defect spatial association prediction model are then fine-tuned for 5 epochs. After fine-tuning, the database to be labeled is cleared.

[0079] Meanwhile, newly identified defect-process association rules (such as the newly discovered "a certain type of inclusion defect is strongly correlated with rolling speed") are automatically added to the association rule base during incremental learning. Through continuous iteration, the system's recognition accuracy and process control capabilities continuously improve over time.

[0080] 10. Output of Test Results After inspection, the system uploads the defect identification results (defect type, location, size, confidence level), grading evaluation results (overall quality level, scores for each dimension), grading early warning information, and closed-loop control command execution records to the Manufacturing Execution System (MES) in standardized JSON format, and stores them on the local server with the steel coil number as the filename. Simultaneously, the verified defect images and their labels are added to the training dataset for the next round of incremental fine-tuning.

[0081] XI. Supplementary Explanation Regarding the adaptive adjustment of the confidence threshold: In this embodiment, the confidence threshold is fixed at 0.8. In other application scenarios, the threshold can be adaptively adjusted according to the steel grade and thickness. For example, for high-strength steel with complex surfaces and easily confused defects, the threshold can be increased to 0.85; for low-carbon steel with simple surfaces and obvious defect characteristics, the threshold can be decreased to 0.75.

[0082] In actual production line operation, this embodiment achieves a complete inspection time of less than 3 minutes for a single sample, a shape inspection accuracy of 0.1mm, a thickness inspection accuracy of ±1μm, and a surface defect identification accuracy of over 98.5%. Compared with manual inspection, this improves efficiency by 5 times and reduces the defect miss rate by 80%, effectively supporting the intelligent operation of the production line.

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for strip steel defect identification and graded control based on a template intelligent inspection platform, characterized in that, include: Acquire the two-dimensional image, three-dimensional morphology, and thickness distribution data of the template and register them to the same coordinate system; extract the defect feature maps of the three types of data, and input them into the data fusion model after weighted fusion to output the defect type, geometric parameters, and recognition confidence. When the confidence level is lower than the threshold, a review identification is performed on the defective area, and samples whose review results are inconsistent with the original results are stored in the unlabeled database. The system uses identified defects as nodes and constructs a topology based on the spatial relationships between defects. The defect spatial correlation prediction model outputs the predicted probability of defects in adjacent areas, and the subsequent detection parameters are dynamically adjusted accordingly. At the same time, the system performs weighted scoring based on multiple evaluation dimensions of defects, outputs a comprehensive quality level, and generates graded early warnings by combining the predicted probabilities. When the overall quality level or graded early warning reaches the automatic intervention threshold, a control command is generated based on the defect-process association rule base and sent to the production line control system; otherwise, the detection results and process adjustment suggestions are pushed to the operation terminal. According to a preset cycle, or when the number of samples in the database to be labeled reaches a preset sample threshold, the data fusion model and the defect space association prediction model are incrementally fine-tuned, and the rule base is updated.

2. The method according to claim 1, characterized in that, The weighted fusion is implemented using an attention mechanism, which is a channel attention module, a spatial attention module, or a combination of both. The channel attention module extracts the global description of the feature map through global average pooling and global max pooling, maps it through a shared multilayer perceptron, adds it element by element, and then activates it with a sigmoid function to generate channel weights. The spatial attention module performs channel-dimensional average pooling and max pooling on the feature map, concatenates them, and then activates them with a convolutional layer and a sigmoid function to generate spatial weights. During weighted fusion, the attention weights are multiplied element-wise with the original feature map.

3. The method according to claim 1, characterized in that, The data fusion model is a multi-branch convolutional neural network, including: a first branch is a two-dimensional convolutional neural network that receives two-dimensional image data; a second branch is a one-dimensional or three-dimensional convolutional neural network that receives three-dimensional shape data; and a third branch is a fully connected network or a one-dimensional convolutional network that receives thickness distribution data. The feature vectors output by the three branches are weighted and fused, then concatenated, and then output through a fully connected layer and a Softmax layer to output the probability distribution of defect types. At the same time, the defect bounding box parameters and recognition confidence are output through a regression branch.

4. The method according to claim 1, characterized in that, The verification and identification includes at least one of the following operations: The control line scan camera and 3D laser camera perform a second high-resolution scan of the defect area, and the new data is re-input into the data fusion model for secondary identification. The backup classifier, which is pre-trained based on historical defect data, is invoked to identify the features of the defect area. The backup classifier includes support vector machine, random forest or shallow neural network. The multimodal data and initial identification results of the defective area are pushed to the manual review terminal; When multiple review operations coexist, a voting or priority mechanism is used to determine the final review result.

5. The method according to claim 1, characterized in that, The topology is a graph structure, with each defect as a node, from which its spatial coordinates and defect features are extracted. Calculate the Euclidean distance between any two nodes. When the distance is less than the preset neighborhood radius, establish an edge connection to form an undirected graph; or use the K nearest neighbor method to connect each node to the K nearest other nodes, with the K value dynamically determined according to the defect density; for isolated nodes, use self-loop edges to connect to themselves.

6. The method according to claim 1, characterized in that, The defect spatial association prediction model is a graph convolutional network or a graph attention network; the graph convolutional network normalizes the adjacency matrix and multiplies it with the node feature matrix, then updates the node representation through linear transformation and nonlinear activation; the graph attention network learns attention coefficients for each neighbor node and sums them up by weighted coefficients. The model takes node feature matrix and adjacency matrix as input and outputs the predicted probability of defects in adjacent regions. The row vectors of the node feature matrix are concatenated by defect type encoding, geometric size, confidence level and location coordinates and mapped to a preset dimension through a fully connected layer.

7. The method according to claim 1, characterized in that, The dynamic adjustment of subsequent detection parameters includes: setting a first threshold and a second threshold, with the first threshold being less than the second threshold; when the predicted probability exceeds the first threshold but is lower than the second threshold, increasing the scanning frame rate of the line scan camera or reducing the sample delivery speed; when the predicted probability exceeds the second threshold, triggering the 3D laser camera to perform encrypted scanning of the area and supplementing the encrypted scanning data to the input of the data fusion model in real time; when the predicted probability is lower than the first threshold, maintaining the normal detection parameters.

8. The method according to claim 1, characterized in that, The multiple evaluation dimensions include at least five dimensions: defect type, size, density, location, and process impact. A pre-defined percentage scoring standard is used for each dimension. Defect type is assigned a value based on a severity table; defect size is linearly mapped by area or length; defect density is scored in segments based on quantity; defect location is assigned different weights based on whether it is the edge, center, working side, or transmission side; and process impact is assigned a value based on its influence on subsequent processes. The comprehensive defect index is obtained by summing the scores of each dimension by their respective weighting coefficients. The weighting coefficients are determined using either the analytic hierarchy process (AHP) or the entropy weighting method, depending on the data characteristics.

9. The method according to claim 1, characterized in that, The registration to the same coordinate system includes: establishing a world coordinate system with the fixed reference point of the detection platform as the origin; obtaining the external parameter matrix of each sensor through calibration; and transforming the data from pixel or sensor coordinates to the world coordinate system. For thickness distribution data, spatial position interpolation is performed based on the sample conveying speed and timestamp to keep the sampling interval consistent with the two-dimensional and three-dimensional data.

10. The method according to claim 1, characterized in that, The defect-process association rule base is constructed using decision tree or association rule mining algorithms. Each rule includes the defect type, defect feature range, and corresponding process parameter adjustment suggestions. The decision tree uses the defect type as the prediction target and extracts the process parameter threshold through information gain or Gini coefficient split nodes. The association rule mining uses Apriori or FP-Growth algorithms to mine frequent itemsets between the defect type and the process parameter value range. When automatic intervention is triggered, the control command generated according to the defect type and comprehensive quality level matching rules includes at least one of the following: roll force adjustment value, rolling speed correction amount, roll coolant flow rate adjustment value, or tension setting value.

11. The method according to claim 1, characterized in that, The incremental fine-tuning employs either empirical replay or elastic weight consolidation algorithms. The empirical replay method maintains a fixed-size replay buffer and trains the model by randomly selecting historical samples from the database to be labeled and mixing them with new samples. The elastic weight consolidation method calculates the importance weights of parameters after incremental learning and applies a secondary penalty term to the update of important parameters. The triggering condition for incremental fine-tuning is that the number of samples in the database to be labeled reaches a preset threshold or that the time elapsed since the last fine-tuning exceeds a preset duration. After triggering, the database to be labeled is cleared or marked.

12. The method according to claim 1, characterized in that, The identification confidence level is the maximum value of the Softmax probability distribution output by the data fusion model; when the maximum value is lower than the preset confidence level threshold, a review identification is triggered; the confidence level threshold ranges from 0.7 to 0.9, and is adaptively adjusted according to the steel type, thickness, or surface roughness of the sample.

Images