Online intelligent detection and early warning system for metal processing quality based on AI vision

Abstract

This invention discloses an online intelligent detection and early warning system for metal processing quality based on AI vision, specifically relating to the field of metal processing quality inspection. The system employs a distributed industrial camera and intelligent light source in its image acquisition module, combined with FPGA synchronous control, to ensure clear and distortion-free images on high-speed production lines. The data preprocessing module enhances image quality through adaptive median filtering, distortion correction, and GPU acceleration. The AI ​​vision detection module accurately outputs defect locations, bounding boxes, and confidence scores based on an improved YOLOv3 model. The defect classification module uses a CNN-Transformer dual-branch network to determine five types of defects: cracks, scratches, dents, dimensional deviations, and surface deposits, and integrates dual-reset confidence verification. The early warning response module triggers graded audible and visual alarms and notifications based on defect levels, and coordinates with the PLC to execute shutdowns, marking and diverting processes, or continuous monitoring, achieving closed-loop management of "detection-classification-disposal."

Description

Technical Field

[0001] This invention relates to the field of metal processing quality inspection technology, and more specifically, to an online intelligent inspection and early warning system for metal processing quality based on AI vision. Background Technology

[0002] In the metal processing industry, product quality inspection is a crucial link in ensuring production stability and market competitiveness. Traditional metal processing quality inspection mainly relies on manual visual inspection or contact-based inspection equipment, which suffers from low inspection efficiency, high subjectivity, high missed detection rate, and inability to adapt to high-speed production lines. With the improvement of industrial automation, machine vision-based inspection technology is gradually being applied, but existing technologies still have significant shortcomings: Existing machine vision inspection systems mostly employ traditional image processing algorithms, which are insufficient in recognizing minute defects in complex backgrounds, especially limiting their detection accuracy for targets with inconspicuous features such as fine cracks and minor scratches on metal surfaces. Some inspection solutions employing AI vision technology, such as the improved YOLOv3 method proposed in patent publication number CN202111199291.8, have improved detection accuracy, but have not achieved a closed-loop process for detection and early warning, and lack real-time response and equipment linkage mechanisms for metal processing scenarios. At the same time, the AI ​​models of existing systems are mostly fixed architectures, unable to self-optimize according to newly emerging defect types on-site, resulting in insufficient generalization ability. Furthermore, the early warning mechanisms of traditional inspection systems are simplistic, making it difficult to implement graded responses based on defect severity, leading to an imbalance between production efficiency and quality control. Summary of the Invention

[0003] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an online intelligent detection and early warning system for metal processing quality based on AI vision, in order to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: Image acquisition module: Composed of a multi-view industrial camera, a light source adjustment unit and a synchronization control component, used to acquire surface and dimensional image data of metal parts in real time; Data preprocessing module: used to perform noise reduction, grayscale normalization, distortion correction and data enhancement on the acquired images. The data enhancement adopts a combination strategy of random flipping, brightness adjustment and Gaussian blur. AI visual inspection module: It adopts the improved YOLOv3 detection model to perform feature extraction and defect detection on standardized inspection images, and outputs the defect location coordinates, bounding box parameters and confidence scores; Defect classification module: used to determine the type of detected defects, including cracks, scratches, dents, dimensional deviations and surface deposits; Early warning response module: Used to generate graded early warning signals based on defect levels, and control the processing equipment to perform shutdown, marking and diversion operations.

[0005] Preferably, in the image acquisition module, the multi-view industrial cameras adopt a distributed layout scheme. 3-6 industrial cameras are configured according to the shape characteristics (sheet metal, profile, or component) of the metal processing parts. Among them, the surface inspection cameras are deployed at the top, bottom, left, and right key viewpoints of the processing parts, while the dimension inspection cameras are orthogonally arranged to acquire the contours of key dimensions such as length, width, and thickness of the processing parts. All industrial cameras meet the technical specifications of resolution not less than 1920×1080 pixels and frame rate ≥30fps, and support global shutter mode, which can effectively avoid image distortion caused by motion blur in high-speed production lines and ensure clear imaging of tiny defects with a minimum size ≥0.1mm. The light source adjustment unit adopts an intelligent supplementary lighting system. The core components include an LED ring light source, a strip light source, and a diffuse reflection light source. The ring light source is used for surface defect detection of the processed parts, the strip light source is adapted to the contour acquisition of long strip profiles, and the diffuse reflection light source is used to suppress glare for highly reflective metal materials (such as stainless steel and aluminum alloys). The light source adjustment unit has a built-in material recognition sensor that automatically identifies the material type (ferrous metal, non-ferrous metal) and surface roughness (Ra0.8-6.3μm) of the processed parts. It dynamically adjusts the light intensity (500-5000 lux) and the light angle (0°-45°) through a PID algorithm to ensure that the image contrast is ≥30dB and the gray level difference between the defect area and the background is ≥50 gray levels under different working conditions.

[0006] Preferably, in the data preprocessing module, during the denoising process, an adaptive median filtering algorithm is used to address common industrial dust interference, camera sensor noise, and light reflection noise in metal processing scenarios. This algorithm dynamically adjusts the filtering window size (ranging from 3×3 to 7×7) by analyzing the local pixel grayscale distribution characteristics of the image in real time: when a smooth area is detected, a 3×3 small window filter is used to preserve details; when defect edges or noise-dense areas are identified, it automatically switches to a 5×5 or 7×7 large window filter to achieve a balance between noise suppression and defect feature preservation. After processing, the peak signal-to-noise ratio (PSNR) of the image is ≥35dB. Grayscale normalization is performed by linearly transforming the pixel grayscale values ​​of the original image to the [0,1] interval. The distortion correction process is based on the camera intrinsic parameter matrix and distortion coefficients (including radial distortion coefficients k1, k2, k3 and tangential distortion coefficients p1, p2) and performs inverse transformation correction. The intrinsic parameter matrix and distortion coefficients are obtained in advance using the Zhang Zhengyou calibration method. The pixel coordinate deviation of the corrected image is ≤1 pixel, ensuring the accuracy of defect location and size measurement.

[0007] Preferably, the AI ​​vision inspection module receives standardized metal part image data, uses an optimized and improved YOLOv3 inspection model to complete the entire process of inference from image feature extraction, multi-scale defect prediction to bounding box regression and confidence calculation, and finally outputs accurate defect location coordinates, standardized bounding box parameters and defect confidence according to industrial inspection standards, providing highly reliable basic inspection data for subsequent defect classification and graded early warning; The improved YOLOv3 model uses Darknet-53 as the backbone feature extraction network and performs triple core optimizations for metal defect detection scenarios: Firstly, traditional convolutional layers in the backbone network are replaced with residual convolutional blocks (ResBlock). Residual mapping alleviates the gradient vanishing problem in deep networks and enhances the extraction and transfer of features from minute defects (such as 0.1mm-level microcracks). The specific method for calculating the feature mapping of a single residual unit is as follows: ; Among them, F out Represented as the output feature map, F in The input feature map is represented by the residual unit. W1 and W2 are represented by the 3×3 convolution kernel weight matrix, and b1 and b2 are represented by the bias terms. The slope of the LeakyReLU activation function is set to 0.1. The identity mapping ensures that the shallow features are passed into the deep layer without loss. Secondly, a bidirectional feature pyramid (FPN+PAN) of the fusion path aggregation network (PANet) is constructed to achieve bidirectional fusion of feature maps at three scales: 13×13, 26×26, and 52×52. Deep semantic features are transferred through top-down upsampling, and shallow detailed features are enhanced through bottom-up path aggregation. The specific calculation method for the multi-scale feature fusion formula is as follows: ; in, Represented as the fused feature map, F l+1 Let F be the feature map of the (l+1)th layer, and Upsample represent upsampling by a factor of 2 using bilinear interpolation. l The original feature map of layer l is represented as , and Conv represents a 1×1 convolutional dimensionality reduction operation. This is represented as element-wise addition, ensuring that the model adapts to all scales, including large-sized depressions and small-sized scratches; Third, abandoning the traditional IOU loss, we adopt the DIOU loss function to optimize bounding box regression, while considering the overlap between the predicted box and the ground truth box and the distance between the center points, which significantly improves the localization accuracy of the metal defect bounding box. The specific DIOU loss calculation method is as follows: ; Among them, Loss DIOUThis is represented by DIOU loss, IOU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, and q is the center coordinate of the predicted bounding box. gt Let be the center coordinates of the ground truth bounding box, and let c be the diagonal length of the smallest bounding rectangle enclosing both the predicted and ground truth bounding boxes. It is expressed as the square of the Euclidean distance; The model uses three detection heads of different scales to predict defects in the fused feature map. Each detection head outputs a set of feature tensors containing bounding box prediction parameters, class prediction scores, and target confidence scores. This is the core foundation for calculating defect location coordinates, bounding box parameters, and confidence scores. Each detection grid generates nine metal defect-specific prior boxes optimized by the K-means++ clustering algorithm. The prior box sizes are obtained by clustering the labeled data of five types of metal defects, including cracks and scratches, with an average intersection-over-union (AvgIOU) ≥ 0.85, effectively reducing the initial prediction error of the model.

[0008] Preferably, the defect classification module receives standardized defect detection data output by the AI ​​vision detection module, including the defect center location coordinates, the diagonal pixel coordinates of the bounding box, the width and height parameters, and the defect confidence score. Based on this data, the defect area is cropped and features are extracted. A customized dual-branch fusion classification network is used to determine the type of five types of defects in the metal processing parts: cracks, scratches, dents, dimensional deviations, and surface attachments. At the same time, the classification confidence score is output, which forms a dual reliability verification with the detection confidence score of the AI ​​vision detection module. The module receives structured output data in the format {x,y;x1,y1,x2,y2,w,h;Score}. det}, where x, y are the pixel coordinates of the defect center, x1, y1 and x2, y2 are the pixel coordinates of the top-left and bottom-right corners of the defect bounding box, w, h are the width and height of the bounding box, and Score det The defect detection confidence score is output by the AI ​​vision detection module. Based on the diagonal pixel coordinates (x1, y1) and (x2, y2) of the bounding box in the effective data, the defect region is accurately cropped from the original detection image after standardization by the data preprocessing module. During the cropping process, the edge outside the bounding box is expanded by 10 pixels to generate a defect region sub-image of uniform size (uniformly scaled to 224×224 pixels), which serves as the core input of the classification network. Secondly, for the cropped defect region sub-image, the module performs feature extraction using a customized CNN-Transformer dual-branch fusion classification network. This network is optimized for the feature differences of five types of metal defects. The CNN branch focuses on extracting local detail features of the defect, adapting to defects with obvious local features such as linear textures of cracks, elongated edges of scratches, and gray-level gradient changes in depressions. The Transformer branch captures the global semantic and spatial distribution features of the defect through a multi-head self-attention mechanism, adapting to defects with obvious global features such as overall contour anomalies due to size deviations and regional distribution features of surface attachments. Furthermore, the features extracted by both branches are based on the defect regions located by the AI ​​visual detection module, without background feature interference. The CNN branch uses a 4-layer convolution-pooling stacked structure. The specific feature mapping calculation method for the i-th convolution layer is as follows: ; in, Represented as the convolutional feature map of the i-th layer, Let b be the convolutional feature map of the (i-1)th layer. i Represented as the bias term of the i-th layer; The convolutional feature map is then subjected to max pooling and flattening to generate a local feature vector F with dimension 512. CNN ; The Transformer branch uses a 6-layer encoder structure, and the multi-head self-attention calculation method is as follows: ; Where Q represents the query matrix, K represents the key matrix, and V represents the value matrix; The query matrix, key matrix, and value matrix are encoded and flattened to generate a global feature vector F with a dimension of 512. Transformer Subsequently, the module adaptively weights and fuses the feature vectors extracted from the two branches. The fusion weights are dynamically learned through training on industrial site metal defect samples, and the feature differences for the five types of defects are optimized to ω. CNN =0.65、ω Transformer =0.35, the specific calculation method for the fused feature vector is as follows: ; Among them, F fusion This is represented as a fused feature vector; the final generated fused feature vector has a dimension of 512, which serves as the core basis for determining the defect type. The fused feature vector is input into a fully connected classification network with two hidden layers. After linear transformation and activation, the conditional probabilities of five defect types are output through the Softmax function for preliminary defect type determination. The probability calculation method is as follows: ; in, c is the original score of the nth type defect output by the fully connected network. n The defects are classified into five categories: cracks, scratches, dents, dimensional deviations, and surface deposits, respectively. k=5 represents the total number of defect categories. The category with the highest probability is taken as the preliminary classification result, and this highest probability is recorded as the classification confidence score. cls This results in a preliminary classification type and classification confidence level.

[0009] Preferably, the early warning response module receives structured data output by the defect classification module, wherein the "defect type" and "overall confidence score" are... com "Based on a pre-defined 'defect severity weighting table,' a multi-dimensional defect level determination model is established; defect levels are divided into three levels, each corresponding to a different handling strategy: Level 1 warning (fatal defect), Level 2 warning (serious defect), and Level 3 warning (minor defect). The level determination formula is: Level=f(T,Score)" com The score is calculated as follows: T is the defect type, and W is the defect severity weight (preset by the user based on the application scenario of the processed part, such as aerospace metal parts having a higher weight than ordinary mechanical parts); the specific judgment rule is: when the defect type is crack (W=0.9) or dimensional deviation (W=0.85) and Score is... com ≥0.85, or the defect type is dent (W=0.7) and Score com When the score is ≥0.9, it is judged as a Level 1 warning; when the defect type is scratch (W=0.6) or dent and 0.8≤Score com <0.9, or surface deposits (W=0.5) and Score com When the score is ≥0.9, it is judged as a level 2 warning; when the defect type is surface attachment and 0.8≤Score com <0.9, or 0.7≤Score for other defect types com A score <0.8 is considered a Level 3 warning; all scores com Suspected defects with a value <0.7 are directly marked as "pending re-inspection" without triggering tiered warnings; only a manual re-inspection notification is sent.

[0010] The technical effects and advantages of this invention are as follows: This invention utilizes multi-view cameras, intelligent light sources, and synchronous control to accurately acquire surface and dimensional data through image acquisition. Preprocessing, including noise reduction and normalization, improves data quality. AI detection employs an improved YOLOv3, optimizing feature extraction and bounding box regression to accurately output defect information. Defect classification uses a CNN-Transformer dual-branch network to determine the type, combined with double-reset confidence verification. Early warning response is based on defect type and comprehensive confidence level, classifying warnings into three levels. This triggers equipment shutdown, marking, and diversion operations, simultaneously pushing tiered alarms and notifications, achieving a closed-loop process from defect detection to handling, ensuring production quality and efficiency. This invention automates the entire process from image acquisition, defect detection, classification to early warning response, reducing manual intervention and labor costs. Furthermore, by linking with processing equipment, it enables real-time intervention in quality issues, reducing the defect rate. The model self-optimization module supports incremental training and can automatically adapt to new defect types without the need for frequent manual model updates. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the overall structure of the present invention.

[0012] Figure 2 This is a schematic diagram of the AI ​​visual inspection process of the present invention.

[0013] Figure 3 This is a schematic diagram of the defect classification process of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] Please see Figure 1 As shown, this invention provides an online intelligent detection and early warning system for metal processing quality based on AI vision, comprising: Image acquisition module: Composed of a multi-view industrial camera, a light source adjustment unit and a synchronization control component, used to acquire surface and dimensional image data of metal parts in real time; In the image acquisition module, the multi-view industrial cameras adopt a distributed layout scheme. 3-6 industrial cameras are configured according to the shape characteristics of the metal processing parts (sheet metal, profile metal, or component metal). Among them, the surface inspection cameras are deployed at the top, bottom, left, and right key viewpoints of the processing parts, while the dimension inspection cameras are orthogonally arranged to acquire the contours of key dimensions such as length, width, and thickness of the processing parts. All industrial cameras meet the technical specifications of resolution of not less than 1920×1080 pixels and frame rate ≥30fps, and support global shutter mode, which can effectively avoid image distortion caused by motion blur in high-speed production lines and ensure clear imaging of tiny defects with a minimum size ≥0.1mm. The light source adjustment unit adopts an intelligent supplementary lighting system. The core components include an LED ring light source, a strip light source, and a diffuse reflection light source. The ring light source is used for surface defect detection of the processed parts, the strip light source is adapted for contour acquisition of long strip profiles, and the diffuse reflection light source is used to suppress glare for highly reflective metal materials (such as stainless steel and aluminum alloys). The light source adjustment unit has a built-in material recognition sensor that automatically identifies the material type (ferrous metal, non-ferrous metal) and surface roughness (Ra0.8-6.3μm) of the processed parts. It also dynamically adjusts the light intensity (500-5000 lux) and the light angle (0°-45°) through a PID algorithm to ensure that the image contrast is ≥30dB and the gray level difference between the defect area and the background is ≥50 gray levels under different working conditions. The synchronization control component consists of a photoelectric sensor, a pulse encoder, and an FPGA synchronization controller. The photoelectric sensor is used to detect the entry and exit of the workpiece and trigger the camera to capture the signal. The pulse encoder is linked with the production line drive system to collect the production line speed data (0-60m / min) in real time and feed it back to the FPGA synchronization controller to achieve dynamic matching between the camera frame rate and the production line speed (the frame rate increases by 5fps for every 10m / min increase in speed). The FPGA synchronization controller achieves precise synchronization of the shooting sequence of multiple cameras through the EtherCAT bus, with the shooting delay deviation of each camera ≤10μs, ensuring the consistency of spatial coordinates of multi-view images.

[0016] Data preprocessing module: used to perform noise reduction, grayscale normalization, distortion correction and data enhancement on the acquired images. The data enhancement adopts a combination strategy of random flipping, brightness adjustment and Gaussian blur. In the data preprocessing module, during the denoising stage, an adaptive median filtering algorithm is used to address common industrial dust interference, camera sensor noise, and light reflection noise in metal processing scenarios. This algorithm dynamically adjusts the filtering window size (ranging from 3×3 to 7×7) by analyzing the local pixel grayscale distribution characteristics of the image in real time: when a smooth area is detected, a 3×3 small window filter is used to preserve details; when defect edges or noise-dense areas are identified, it automatically switches to a 5×5 or 7×7 large window filter to achieve a balance between noise suppression and defect feature preservation. After processing, the peak signal-to-noise ratio (PSNR) of the image is ≥35dB. Grayscale normalization is performed by linearly transforming the pixel grayscale values ​​of the original image to the [0,1] interval. The specific calculation formula is as follows: ; Where f(x,y) represents the original pixel value, and min(f(x,y)) and max(f(x,y)) are the minimum and maximum gray values ​​of a single image, respectively. This processing can eliminate the difference in image gray range under different lighting conditions and improve the stability of model training and inference. The distortion correction process is based on the camera intrinsic parameter matrix and distortion coefficients (including radial distortion coefficients k1, k2, k3 and tangential distortion coefficients p1, p2) and performs inverse transformation correction. The intrinsic parameter matrix and distortion coefficients are obtained in advance by Zhang Zhengyou calibration method. The pixel coordinate deviation of the corrected image is ≤1 pixel, ensuring the accuracy of defect location and size measurement. The data augmentation process employs a combination of strategies to further expand the diversity of training samples: random flipping includes horizontal and vertical flipping to adapt to the random distribution of defects in metal processing parts; brightness adjustment is achieved by randomly adjusting within ±15% of the original brightness to simulate the fluctuation of light intensity on the production line; Gaussian blur uses random parameters of σ=0.3~0.8 to simulate actual working conditions such as slight dust adhesion or slight camera defocus. This combination of data augmentation increases the sample diversity of the training dataset by more than 3 times, effectively reducing the risk of model overfitting and ensuring the system's generalization ability in complex industrial environments; the entire preprocessing process is accelerated by GPU parallel computing.

[0017] AI visual inspection module: It adopts the improved YOLOv3 detection model to perform feature extraction and defect detection on standardized inspection images, and outputs the defect location coordinates, bounding box parameters and confidence scores; The AI ​​vision inspection module receives standardized metal part image data and uses an optimized and improved YOLOv3 inspection model to complete the entire process of inference from image feature extraction, multi-scale defect prediction to bounding box regression and confidence calculation. Finally, it outputs accurate defect location coordinates, standardized bounding box parameters and defect confidence according to industrial inspection standards, providing highly reliable basic inspection data for subsequent defect classification and graded early warning. The improved YOLOv3 model uses Darknet-53 as the backbone feature extraction network and performs triple core optimizations for metal defect detection scenarios: Firstly, traditional convolutional layers in the backbone network are replaced with residual convolutional blocks (ResBlock). Residual mapping alleviates the gradient vanishing problem in deep networks and enhances the extraction and transfer of features from minute defects (such as 0.1mm-level microcracks). The specific method for calculating the feature mapping of a single residual unit is as follows: ; Among them, F out Represented as the output feature map, F in The input feature map is represented by the residual unit. W1 and W2 are represented by the 3×3 convolution kernel weight matrix, and b1 and b2 are represented by the bias terms. The slope of the LeakyReLU activation function is set to 0.1. The identity mapping ensures that the shallow features are passed into the deep layer without loss. Secondly, a bidirectional feature pyramid (FPN+PAN) of the fusion path aggregation network (PANet) is constructed to achieve bidirectional fusion of feature maps at three scales: 13×13, 26×26, and 52×52. Deep semantic features are transferred through top-down upsampling, and shallow detailed features are enhanced through bottom-up path aggregation. The specific calculation method for the multi-scale feature fusion formula is as follows: ; in, Represented as the fused feature map, F l+1 Let F be the feature map of the (l+1)th layer, and Upsample represent upsampling by a factor of 2 using bilinear interpolation. l The original feature map of layer l is represented as , and Conv represents a 1×1 convolutional dimensionality reduction operation. This is represented as element-wise addition, ensuring that the model adapts to all scales, including large-sized depressions and small-sized scratches; Third, abandoning the traditional IOU loss, we adopt the DIOU loss function to optimize bounding box regression, while considering the overlap between the predicted box and the ground truth box and the distance between the center points, which significantly improves the localization accuracy of the metal defect bounding box. The specific DIOU loss calculation method is as follows: ; Among them, Loss DIOU This is represented by DIOU loss, IOU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, and q is the center coordinate of the predicted bounding box. gt Let be the center coordinates of the ground truth bounding box, and let c be the diagonal length of the smallest bounding rectangle enclosing both the predicted and ground truth bounding boxes. It is expressed as the square of the Euclidean distance; The model uses three detection heads of different scales to predict defects in the fused feature map. Each detection head outputs a set of feature tensors containing bounding box prediction parameters, class prediction scores, and target confidence scores. This is the core foundation for calculating defect location coordinates, bounding box parameters, and confidence scores. Each detection grid generates nine metal defect-specific prior boxes optimized by the K-means++ clustering algorithm. The prior box sizes are obtained by clustering the labeled data of five types of metal defects, such as cracks and scratches, with an average intersection-over-union (AvgIOU) ≥ 0.85, effectively reducing the initial prediction error of the model. For the raw prediction parameters output by the model, the module calculates the defect location coordinates and bounding box parameters using a standardized decoding algorithm, and calculates the defect confidence score through two-layer score fusion. The specific output logic and calculation process are as follows: In the calculation output of the bounding box parameters and defect location coordinates, the original bounding box prediction value output by the improved YOLOv3 model is t. x t y t w t h This value is the offset relative to the prior box. It needs to be combined with the coordinates of the corresponding detection grid and the original width and height of the prior box, and converted into standardized bounding box parameters and defect center position coordinates in the image pixel coordinate system through a decoding formula. The pixel coordinate system has the origin (0,0) at the top left corner of the detection image, the x-axis is horizontal to the right, and the y-axis is vertical downward, with the unit being pixels. Let the pixel coordinates of the top left corner of a prediction grid in the model detection feature map be (c x ,c y The width of the prior box corresponding to this grid is p. w Height is p h The specific method for decoding and calculating the coordinates (x, y) of the defect center is as follows: ; ; in, The activation function maps the offset to the (0,1) interval to ensure that the prediction center is located within the current grid. strid is the downsampling step size of the feature map relative to the original image (13×13 scale step size 32, 26×26 scale step size 16, 52×52 scale step size 8). The defect center location coordinates are the precise center pixel position of the defect in the standardized inspection image. They are the core parameters for defect localization and can be directly mapped to the physical location of the metal workpiece.

[0018] Based on the coordinates of the defect center, the bounding box parameters are further decoded. First, the width w and height h of the predicted box in the pixel coordinate system are calculated. The specific calculation method is as follows: ; ; Then, by converting the center coordinates to the width and height, we can obtain the top-left corner pixel coordinates (x1, y1) and bottom-right corner pixel coordinates (x2, y2) of the bounding box. The specific calculation method is as follows: ; ; ; ; The module's final output bounding box parameters include two sets of parameters: bounding box width w, height h, and diagonal pixel coordinates (x1, y1) and (x2, y2). It also supports converting pixel coordinates to the physical coordinates of the metal part (physical coordinates = pixel coordinates × camera calibration pixels - physical conversion coefficient), meeting the detection requirements for dimensional deviation defects. After decoding all predicted boxes, the module filters redundant predicted boxes using the non-maximum suppression (NMS) algorithm, setting the IOU threshold to 0.45. That is, when the intersection-union ratio (IU) of two predicted boxes exceeds 0.45, the predicted box with higher confidence is retained, eliminating the problem of repeated detection of the same defect and ensuring that each defect outputs only a unique set of position coordinates and bounding box parameters. In the calculation output of the defect confidence score, the defect confidence score output by the module is a fusion score of the comprehensive target existence confidence score and the defect category confidence score. It is used to characterize the reliability of the model for the detection result of "metal processing defect". The confidence score range is [0,1]. The closer the value is to 1, the more reliable the detection result is. The module defaults to a confidence score threshold of 0.7. Prediction results below this threshold will be judged as invalid detections and filtered out. Only valid defect results with a confidence score ≥0.7 will be output. Ultimately, the defect confidence score... det There exists a confidence level C for the target. obj The method for calculating the target existence confidence score by multiplying it by the maximum class conditional probability is as follows: ; Where z0 represents the original target score output by the model. A higher value indicates a greater probability that the target within the prediction box is a defect; The specific method for calculating the defect confidence level is as follows: ; This score comprehensively reflects the dual reliability of the predicted bounding box containing a defect and the defect being of a specific type. When the module outputs the confidence score, it simultaneously associates the corresponding defect's location coordinates with the bounding box parameters, with the output format being {x,y;x1,y1,x2,y2,w,h;Score}. detIt also supports outputting confidence results as a percentage, which facilitates threshold determination and manual review in subsequent early warning systems.

[0019] Defect classification module: used to determine the type of detected defects, including cracks, scratches, dents, dimensional deviations and surface deposits; The defect classification module receives standardized defect detection data output by the AI ​​vision detection module, including the coordinates of the defect center position, the diagonal pixel coordinates of the bounding box, the width and height parameters, and the defect confidence score. Based on this data, the defect area is cropped and features are extracted. A customized dual-branch fusion classification network is used to determine the type of five types of defects in the metal processing parts: cracks, scratches, dents, dimensional deviations, and surface attachments. At the same time, the classification confidence score is output, which forms a dual reliability verification with the detection confidence score of the AI ​​vision detection module. The module receives structured output data in the format {x,y;x1,y1,x2,y2,w,h;Score}. det}, where x, y are the pixel coordinates of the defect center, x1, y1 and x2, y2 are the pixel coordinates of the top-left and bottom-right corners of the defect bounding box, w, h are the width and height of the bounding box, and Score det The defect detection confidence score is output by the AI ​​vision detection module. Based on the diagonal pixel coordinates (x1, y1) and (x2, y2) of the bounding box in the effective data, the defect region is accurately cropped from the original detection image after standardization by the data preprocessing module. During the cropping process, the edge outside the bounding box is expanded by 10 pixels to generate a defect region sub-image of uniform size (uniformly scaled to 224×224 pixels), which serves as the core input of the classification network. Secondly, for the cropped defect region sub-image, the module performs feature extraction using a customized CNN-Transformer dual-branch fusion classification network. This network is optimized for the feature differences of five types of metal defects. The CNN branch focuses on extracting local detail features of the defect, adapting to defects with obvious local features such as linear textures of cracks, elongated edges of scratches, and gray-level gradient changes in depressions. The Transformer branch captures the global semantic and spatial distribution features of the defect through a multi-head self-attention mechanism, adapting to defects with obvious global features such as overall contour anomalies due to size deviations and regional distribution features of surface attachments. Furthermore, the features extracted by both branches are based on the defect regions located by the AI ​​visual detection module, without background feature interference. The CNN branch uses a 4-layer convolution-pooling stacked structure. The specific feature mapping calculation method for the i-th convolution layer is as follows: ; in, Represented as the convolutional feature map of the i-th layer, Let b be the convolutional feature map of the (i-1)th layer.i Represented as the bias term of the i-th layer; The convolutional feature map is then subjected to max pooling and flattening to generate a local feature vector F with dimension 512. CNN ; The Transformer branch uses a 6-layer encoder structure, and the multi-head self-attention calculation method is as follows: ; Where Q represents the query matrix, K represents the key matrix, and V represents the value matrix; The query matrix, key matrix, and value matrix are encoded and flattened to generate a global feature vector F with a dimension of 512. Transformer Subsequently, the module adaptively weights and fuses the feature vectors extracted from the two branches. The fusion weights are dynamically learned and updated during model training using a backpropagation algorithm combined with metal defect samples. During the online inference phase on the actual production line, optimized fusion weights ω are fixed and applied based on the feature differences among the five types of defects. CNN =0.65、ω Transformer =0.35, the specific calculation method for the fused feature vector is as follows: ; Among them, F fusion This is represented as a fused feature vector; the final generated fused feature vector has a dimension of 512, which serves as the core basis for determining the defect type. The fused feature vector is input into a fully connected classification network with two hidden layers. After linear transformation and activation, the conditional probabilities of five defect types are output through the Softmax function for preliminary defect type determination. The probability calculation method is as follows: ; in, c is the original score of the nth type defect output by the fully connected network. n The defects are classified into five categories: cracks, scratches, dents, dimensional deviations, and surface deposits, respectively. k=5 represents the total number of defect categories. The category with the highest probability is taken as the preliminary classification result, and this highest probability is recorded as the classification confidence score. cls This forms a preliminary classification type plus classification confidence result; Finally, the module performs a double-reset reliability verification and outputs the results, setting the classification confidence score. cls The detection confidence score output by the AI ​​visual inspection module det Perform fusion validation and calculate the overall confidence level. The specific calculation method is as follows: ; When Score comIf the score is ≥0.85, it is considered a valid classification result, and the final defect type is directly output; if 0.7≤Score com If the value is less than 0.8, then secondary feature extraction and classification are triggered, and the weights of the two-branch fusion process (ω) are adjusted. CNN =0.7、ω Transformer =0.3) Re-infer, and take the maximum probability after the second inference as the final result; if Score com If the score is less than 0.7, it is considered a suspected defect and is pushed to the manual review stage. The final module integrates the raw detection data from the AI ​​visual inspection module with the final defect type and overall confidence score, outputting a structured classification result in the format {x,y;x1,y1,x2,y2,w,h;Score}. det Defect Type; Score cls Score com This data can be directly accessed by downstream early warning response modules, providing a clear and reliable basis for the execution of tiered early warning strategies.

[0020] Early warning response module: used to generate graded early warning signals based on defect levels, and control the processing equipment to perform shutdown, marking, and diversion operations; The early warning response module receives structured data output by the defect classification module, including "defect type" and "overall confidence score". com "Based on a pre-defined 'defect severity weighting table,' a multi-dimensional defect level determination model is established; defect levels are divided into three levels, each corresponding to a different handling strategy: Level 1 warning (fatal defect), Level 2 warning (serious defect), and Level 3 warning (minor defect). The level determination formula is: Level=f(T,Score)" com The score is calculated as follows: T is the defect type, and W is the defect severity weight (preset by the user based on the application scenario of the processed part, such as aerospace metal parts having a higher weight than ordinary mechanical parts); the specific judgment rule is: when the defect type is crack (W=0.9) or dimensional deviation (W=0.85) and Score is... com ≥0.85, or the defect type is dent (W=0.7) and the score is... com When the score is ≥0.9, it is judged as a Level 1 warning; when the defect type is scratch (W=0.6) or dent and 0.8≤Score com <0.9, or surface deposits (W=0.5) and Score com When the score is ≥0.9, it is judged as a level 2 warning; when the defect type is surface attachment and 0.8≤Score com <0.9, or 0.7≤Score for other defect types com A score <0.8 is considered a Level 3 warning; all scores comSuspected defects with a value <0.7 are directly marked as "pending re-inspection" without triggering tiered warnings; only manual re-inspection notifications are sent. The defect severity weighting table is shown in Table 1: Table 1 ; Secondly, the module generates tiered early warning signals based on the determined defect level. These signals encompass three main dimensions: audible and visual alarms, terminal notifications, and system logs. The signal characteristics of different warning levels are significantly distinct, ensuring rapid identification by management personnel and equipment systems: Level 1 warnings trigger a "high-frequency audible and visual alarm (frequency 2Hz, volume ≥85dB) + emergency notification via management personnel's mobile app / SMS + red system log marker," while simultaneously displaying the defect location coordinates, type, and judgment criteria in real-time on the industrial control screen; Level 2 warnings trigger a "medium-frequency audible and visual alarm (frequency 1Hz, volume ≥75dB) + notification via management personnel's desktop client + orange system log marker." The system records the following: Level 3 warnings trigger a low-frequency audible and visual alarm (0.5Hz, 265dB) + a yellow log marker, without actively pushing notifications to personnel; they are only recorded in the system background. Defects awaiting re-inspection trigger no audible and visual alarm + a dedicated notification to quality inspectors + a blue log marker. All warning signals contain a unique identifier, which is associated with the structured data output by the defect classification module for easy traceability. Finally, the module executes corresponding equipment control operations based on the defect level, communicating with the PLO control system of the metal processing equipment via the Modbus TCP protocol to output standardized control commands. The control logic and execution flow are as follows: Level 1 Warning (Critical Defect): For defects that directly affect product safety performance (such as cracks in load-bearing structural components or out-of-tolerance critical dimensions), the module immediately outputs an "emergency stop command." Upon receiving the command, the PLC cuts off the power supply to the processing equipment, and the production line completely stops operating within ≤500ms. At the same time, the marking unit of the control equipment prints a unique identification code (including defect level, inspection time, and equipment number) near the defect location for easy traceability and disposal. After the shutdown, management personnel must manually reset and confirm that the defect has been resolved before the production line can be restarted. Level 2 Warning (Severe Defect): For defects that affect product performance but can be repaired by rework (such as deep scratches or obvious dents), the module outputs a "marking + diversion command". First, it controls the marking unit to make a visual mark in the defect area (such as spraying an erasable mark). Then, it sends a command to the diversion device of the production line (such as a pneumatic shift fork or conveyor belt turning mechanism) to divert the defective workpiece from the main production line to the rework line. During the diversion process, the photoelectric sensor confirms that the workpiece is in place. Level 3 warning (minor defects): For defects that do not affect the core performance of the product, can be corrected through subsequent processes or meet the factory standards (such as minor surface deposits or shallow scratches), the module only outputs a "continuous monitoring instruction" and does not interfere with the normal operation of the production line. At the same time, a dynamic monitoring threshold is set. If the number of Level 3 warning defects in the same production batch exceeds 5%, it will be automatically upgraded to Level 2 warning and trigger diversion and sampling inspection. Defects awaiting re-inspection: The module outputs a "manual re-inspection instruction," which controls the slow-stop device of the production line to send the processed part to the re-inspection station. At the same time, it notifies the quality inspectors to view the defect image and inspection data through a dedicated terminal, and to manually determine whether it is a real defect and its corresponding level.

[0021] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An online intelligent detection and early warning system for metal processing quality based on AI vision, characterized in that, include: Image acquisition module: Composed of a multi-view industrial camera, a light source adjustment unit and a synchronization control component, used to acquire surface and dimensional image data of metal parts in real time; Data preprocessing module: used to perform noise reduction, grayscale normalization, distortion correction and data enhancement on the acquired images. The data enhancement adopts a combination strategy of random flipping, brightness adjustment and Gaussian blur. AI visual inspection module: It adopts the improved YOLOv3 detection model to perform feature extraction and defect detection on standardized inspection images, and outputs the defect location coordinates, bounding box parameters and confidence scores; Defect classification module: used to determine the type of detected defects, including cracks, scratches, dents, dimensional deviations and surface deposits; Early warning response module: Used to generate graded early warning signals based on defect levels, and control the processing equipment to perform shutdown, marking and diversion operations.

2. The AI ​​vision-based online intelligent detection and early warning system for metal processing quality according to claim 1, characterized in that: In the image acquisition module, the multi-view industrial cameras adopt a distributed layout scheme. The industrial cameras are configured according to the shape characteristics of the metal workpiece. The surface inspection cameras are deployed at the top, bottom, left, and right key viewpoints of the workpiece, while the size inspection cameras are orthogonally arranged to acquire the contour of the workpiece size. The synchronization control component consists of a photoelectric sensor, a pulse encoder, and an FPGA synchronization controller. The photoelectric sensor is used to detect the entry and exit of the workpiece and trigger the camera to capture the signal.

3. The AI ​​vision-based online intelligent detection and early warning system for metal processing quality according to claim 1, characterized in that: In the data preprocessing module, during the denoising process, an adaptive median filtering algorithm is used to address common industrial dust interference, camera sensor noise, and light reflection noise in metal processing scenarios. The grayscale normalization process maps the pixel grayscale values ​​of the original image to the [0,1] interval through linear transformation. The distortion correction process performs inverse transformation correction based on the camera intrinsic parameter matrix and distortion coefficients, which are pre-obtained using the Zhang Zhengyou calibration method.

4. The AI ​​vision-based online intelligent detection and early warning system for metal processing quality according to claim 1, characterized in that: The AI ​​vision inspection module receives standardized metal part image data, uses an optimized and improved YOLOv3 inspection model to complete the entire process of reasoning from image feature extraction, multi-scale defect prediction to bounding box regression and confidence calculation, and finally outputs accurate defect location coordinates, standardized bounding box parameters and defect confidence according to industrial inspection standards. The improved YOLOv3 model uses Darknet-53 as the backbone feature extraction network and completes triple core optimization for metal defect detection scenarios. The traditional convolutional layers in the backbone network are replaced with residual convolutional blocks. Residual mapping is used to alleviate gradient vanishing in deep networks. The specific method for calculating the feature mapping of a single residual unit is as follows: ； Among them, F out Represented as the output feature map, F in The input feature map is represented by the residual unit, W1 and W2 are the 3×3 convolution kernel weight matrices, b1 and b2 are the bias terms, and the slope of the LeakyReLU activation function is set to 0.

1.

5. The AI ​​vision-based online intelligent detection and early warning system for metal processing quality according to claim 4, characterized in that: A bidirectional feature pyramid is constructed using a fusion path aggregation network, which fused feature maps at three scales: 13×13, 26×26, and 52×52. Deep semantic features are transferred through top-down upsampling, while shallow detail features are enhanced through bottom-up path aggregation. The specific calculation method for the multi-scale feature fusion formula is as follows: ； in, Represented as the fused feature map, F l+1 Let F be the feature map of the (l+1)th layer, and Upsample represent upsampling by a factor of 2 using bilinear interpolation. l The original feature map of layer l is represented as , and Conv represents a 1×1 convolutional dimensionality reduction operation. This is represented as element-wise addition; The traditional IOU loss is removed, and the DIOU loss function is used to optimize the bounding box regression. This considers both the overlap between the predicted and ground truth boxes and the distance between their center points, significantly improving the localization accuracy of the metal defect bounding box. The specific DIOU loss calculation method is as follows: ； Among them, Loss DIOU This is represented by DIOU loss, IOU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, and q is the center coordinate of the predicted bounding box. gt Let be the center coordinates of the ground truth bounding box, and let c be the diagonal length of the smallest bounding rectangle enclosing both the predicted and ground truth bounding boxes. It is expressed as the square of the Euclidean distance; The model uses three detectors of different scales to predict defects in the fused feature map. Each detector outputs a set of feature tensors containing bounding box prediction parameters, class prediction scores, and target confidence scores.

6. The AI ​​vision-based online intelligent detection and early warning system for metal processing quality according to claim 4, characterized in that: For the raw prediction parameters output by the model, the module calculates the defect location coordinates and bounding box parameters using a standardized decoding algorithm, and calculates the defect confidence score through two-layer score fusion. The specific output logic and calculation process are as follows: In the calculation output of the bounding box parameters and defect location coordinates, the original bounding box prediction value output by the improved YOLOv3 model is t. x t y t w t h The parameters are decoded and converted into standardized bounding box parameters and defect center coordinates in the image pixel coordinate system; let the upper left pixel coordinate of a certain prediction grid in the model detection feature map be (c x ,c y The width of the prior box corresponding to this grid is p. w Height is p h ; Based on the center coordinates of the defect, the bounding box parameters are further decoded. First, the width w and height h of the predicted box in the pixel coordinate system are calculated. Then, the top left pixel coordinates (x1, y1) and bottom right pixel coordinates (x2, y2) of the bounding box are obtained by converting the center coordinates with the width and height. The final bounding box parameters output by the module include two sets of parameters: bounding box width w, height h, and diagonal pixel coordinates (x1, y1) and (x2, y2). In the calculation output of the defect confidence score, the defect confidence score output by the module is a fusion score of the comprehensive target existence confidence score and the defect category confidence score. The specific calculation method is as follows: ； When the module outputs the confidence score, it will synchronously associate the location coordinates of the corresponding defect with the bounding box parameters. The output format is {x,y;x1,y1,x2,y2,w,h;Score}. det } 7. The AI ​​vision-based online intelligent detection and early warning system for metal processing quality according to claim 1, characterized in that: In the defect classification module, standardized defect detection data output by the AI ​​visual detection module is received. Based on the data, the defect area is cropped and features are extracted. The type of defect in the metal processing part is determined by a customized dual-branch fusion classification network, and the classification confidence score is output. First, the module receives structured output data. Based on the diagonal pixel coordinates (x1, y1) and (x2, y2) of the bounding box in the data, it performs precise cropping of the defect region on the original detection image after standardization by the data preprocessing module. During the cropping process, the edge outside the bounding box is expanded by 10 pixels to generate a defect region sub-image of uniform size, which serves as the core input of the classification network. Secondly, for the cropped defective region sub-image, the module performs feature extraction through a customized CNN-Transformer dual-branch fusion classification network; The Transformer branch captures the global semantic features and spatial distribution features of defects through a multi-head self-attention mechanism; the CNN branch adopts a 4-layer convolution-pooling stacked structure, and the feature mapping calculation method of the i-th convolution layer is as follows: ； in, Represented as the convolutional feature map of the i-th layer, Let b be the convolutional feature map of the (i-1)th layer. i Represented as the bias term of the i-th layer; The convolutional feature map is then subjected to max pooling and flattening to generate a local feature vector F with dimension 512. CNN .

8. The AI ​​vision-based online intelligent detection and early warning system for metal processing quality according to claim 7, characterized in that: The Transformer branch employs a 6-layer encoder structure. The query matrix, key matrix, and value matrix are encoded and flattened to generate a global feature vector F with a dimension of 512. Transformer Subsequently, the module adaptively weights and fuses the feature vectors extracted from the two branches. The fusion weights are dynamically learned and updated during model training using a backpropagation algorithm combined with metal defect samples. During the online inference phase on the actual production line, optimized fusion weights ω are fixed and applied based on the feature differences among the five types of defects. CNN =0.65、ω Transformer =0.35, the specific calculation method for the fused feature vector is as follows: ； Among them, F fusion This is represented as a fused feature vector; the final generated fused feature vector has a dimension of 512, which serves as the core basis for determining the defect type. The fused feature vector is input into a fully connected classification network with two hidden layers. After linear transformation and activation, the conditional probabilities of five defect types are output through the Softmax function for preliminary defect type determination. The probability calculation method is as follows: ； in, c is the original score of the nth type defect output by the fully connected network. n These correspond to five types of defects in sequence: cracks, scratches, dents, dimensional deviations, and surface deposits. The module performs double-reset reliability verification and outputs the results, including the classification confidence score. cls The detection confidence score output by the AI ​​visual inspection module det Perform fusion validation and calculate the overall confidence level. The specific calculation method is as follows: ； Among them, Score com This is expressed as the overall confidence level.

9. The AI ​​vision-based online intelligent detection and early warning system for metal processing quality according to claim 1, characterized in that: The early warning response module receives structured data output by the defect classification module, including "defect type" and "overall confidence score". com "Based on a pre-defined 'defect severity weighting table,' a multi-dimensional defect level determination model is established; defect levels are divided into three levels, each corresponding to a different handling strategy: Level 1 warning, Level 2 warning, and Level 3 warning. The level determination formula is: Level=f(T,Score)" com ,W), Where T represents the defect type and W represents the defect severity weight; the specific judgment rule is: when the defect type is a crack or dimensional deviation and the Score is... com ≥0.85, or the defect type is dent and the score is... com A score ≥0.9 is considered a Level 1 warning; when the defect type is a scratch or dent and 0.8 ≤ Score com <0.9, or surface deposits and Score com When the score is ≥0.9, it is judged as a level 2 warning; when the defect type is surface attachment and 0.8≤Score com <0.9, or 0.7≤Score for other defect types com A score <0.8 is considered a Level 3 warning; all scores com Suspected defects with a value <0.7 are directly marked as "pending re-inspection" and do not trigger tiered warnings; only manual re-inspection notifications are sent.

Images