A method for detecting surface defects of a weld of a component

Abstract

The application discloses a kind of based on ZYNQ platform's transmission tower steel member weld surface defect detection system and method, belong to industrial vision detection and embedded intelligent identification technical field.Pass through acquisition weld image and construct defect dataset, in YOLOV3 target detection network training stage introduction self-adapting residual network structure and channel attention mechanism, enhance shallow layer detail information transmission and defect relevant feature response, realize to crack, pore, slag inclusion, undercut and weld tumor and so on Defect Effective identification.Training optimized model faces ZYNQ heterogeneous hardware and is deployed, through hierarchical data access, convolution unit reuse, multi-level parallel computing and pipeline processing, reduce data movement overhead, improve convolution operation efficiency and on-chip resource utilization, reduce the hardware scale required for edge deployment, to improve the real-time performance and engineering application capability of weld surface defect detection.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection technology, and more specifically to a method for detecting surface defects in weld seams of components. Background Technology

[0002] In recent years, steel components of power transmission towers, as crucial load-bearing parts of transmission lines, have had their weld quality directly impacting the load-bearing capacity, structural stability, and long-term service reliability. During manufacturing, weld surfaces are prone to defects such as cracks, porosity, slag inclusions, undercut, and weld beads. These defects can lead to stress concentration, localized strength reduction, and abnormal load transfer. Under long-term wind and vibration loads, they can induce fatigue damage, crack propagation, and even connection failure. Therefore, efficient and accurate detection of weld quality defects is of great significance.

[0003] However, existing target detection schemes based on embedded platforms still suffer from the following problems in practical applications of weld defect detection: On the one hand, weld defects are typically characterized by small target size, irregular shape, blurred edges, and indistinct local details. Existing target detection networks are prone to shallow detail attenuation during multi-layer convolution and downsampling, resulting in insufficient feature extraction capabilities for small defect areas and thus affecting detection accuracy. On the other hand, weld surfaces are often accompanied by texture interference, lighting variations, and noise, making the distinction between defect features and the background low, further increasing the difficulty of detecting small defect targets. In addition, traditional embedded deployment schemes still fall short in balancing detection accuracy, real-time performance, and resource constraints, making it difficult to meet the requirements of online weld defect detection for small target recognition capabilities and system stability. Therefore, it is urgent to propose an improved weld defect detection method suitable for the Zynq platform to enhance the network's ability to extract and transmit detailed information about small weld defects, thereby improving detection accuracy and real-time processing capabilities on edge devices. Summary of the Invention

[0004] Existing embedded weld defect detection schemes based on YOLOv3 still have shortcomings in identifying minute defects. Weld defects are typically small in size, with blurred edges and weak features. During the multi-layer convolution and downsampling process of the network, shallow detail information is easily attenuated, resulting in insufficient feature extraction of minute defects and affecting detection accuracy and robustness. At the same time, the complex surface texture of welds and the large amount of background interference, along with the lack of obvious feature differences between defect areas and background areas, further increase the difficulty of defect identification.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] Step S1: Introduce adaptive residual connection units into the original YOLOv3 network training model to enhance the ability of shallow detailed features to be transferred to deep networks, and preserve the edge, texture and minor defect information of the weld area.

[0007] Step S2: Introduce an attention mechanism during model training to perform weighted optimization on the channel dimension or spatial dimension of the feature map, so as to enhance the response of defect-related features and suppress background interference.

[0008] Step S3: Train and optimize the improved YOLOv3 network based on the introduction of adaptive residual structure and attention mechanism to obtain a target detection model suitable for weld defect detection.

[0009] Step S4: Deploy the trained and optimized object detection model on the Zynq platform, where the PL end is responsible for accelerating convolution operations, and the PS end is responsible for data scheduling, post-processing of detection results, and display output.

[0010] Step S5: Apply the deployed target detection model to the weld defect detection scenario, and perform target detection inference on the weld image to improve the accuracy, real-time performance and system integration of weld micro-defect detection.

[0011] The adaptive residual network introduced in step S1 refers to adding a feature weight adjustment mechanism to the residual connection unit, so that after the input features are processed by convolution to extract semantic information, they can be fused with the shallow features passed by skip connections according to preset or adaptive weights. This method enhances the transmission of shallow edge information, texture information, and minor defect features to the deep network, reduces the attenuation of detail information during multi-layer convolution and downsampling, and improves the network's ability to identify and detect minor weld defects.

[0012] The attention mechanism introduced in step S2 is preferably a channel attention mechanism. By adaptively allocating the response weights of different channels in the feature map, the response of the feature channels related to weld defects is enhanced, and interference from irrelevant feature channels is suppressed, thereby improving the network's ability to identify small weld defect targets.

[0013] Step S4 refers to porting and deploying the trained and optimized object detection model to the Zynq platform, and constructing a hardware acceleration structure for convolutional computations during model inference. Specifically, a convolution acceleration IP core is generated using high-level synthesis tools and deployed on the PL (Plug-in Processor) to accelerate convolutional operations; the PS (Power Controller) is responsible for image acquisition control, data scheduling, post-processing of detection results, and display output. Through the collaborative work of the PS and PL, the inference efficiency, real-time performance, and system integration of the model on edge devices can be improved. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the steps of the method for detecting surface defects in welds of steel components for power transmission towers based on ZYNQ and YOLO, as described in this invention.

[0015] Figure 2 This invention provides a network deployment process.

[0016] Figure 3 The neural network structure of this invention

[0017] Figure 4 Hardware deployment for the present invention Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0019] The core of this embodiment lies in constructing a convolution acceleration IP core for weld defect detection tasks, such as... Figure 2 The system is deployed on the programmable logic (PL) side of the Zynq platform to handle the main convolutional operations in the target detection network. Simultaneously, the processing system (PS) side handles image acquisition control, peripheral management, task scheduling, post-processing of detection results, and display output, thus forming an embedded real-time detection architecture based on PS / PL collaboration. Compared to existing schemes that primarily rely on the processor to execute convolutional operations serially, this embodiment maps the convolutional layers to hardware acceleration modules and combines on-chip caching, DMA transfer, and HLS parallel pipeline optimization to reduce convolutional computation latency and data transfer overhead, thereby improving the system's real-time detection capability for weld defect images.

[0020] The hardware system used in this embodiment is based on the ZYNQ-7100 platform, including an OV5640 image acquisition module, a ZYNQ processing module, an SD card read / write module, and an LCD display output module. The ZYNQ platform includes a processing system (PS) and a programmable logic (PL) module. The PS is used for image acquisition control, system process scheduling, detection result parsing, SD card data read / write, and display management. The PL is used for the main computational tasks in convolutional neural network inference. The PL includes an AXI DMA module, a convolution acceleration IP core, an on-chip cache module, and related logic resources, including at least a DSP, BRAM, distributed RAM, and register resources.

[0021] In the dataset construction phase, images of the weld surfaces of steel components in power transmission towers were first acquired using an OV5640 camera, and the acquired raw images were preprocessed. This preprocessing included noise reduction, size adjustment, and normalization to improve the consistency of subsequent defect feature extraction. For the input image... Its normalization process can be expressed as:

[0022]

[0023] in, This represents the minimum number of pixels in the image. Indicates the maximum pixel value of the image. This represents the normalized image. After preprocessing, defects such as cracks, porosity, slag inclusions, undercut, and weld beads in the weld image are classified, labeled, and their locations are bounded. Based on this, a sample dataset of weld surface defects is constructed.

[0024] During the model training phase, the YOLO object detection network is trained based on the aforementioned sample dataset. The output of the convolutional layers in the network can be represented as:

[0025]

[0026] in, Indicates the input feature map, Indicates the convolution kernel weights, Indicates the bias term. Indicates the kernel size. This indicates the location of the output feature map of the convolutional layer. The value at that point. After the convolution output is processed by the activation function, the final feature response is obtained, and its expression is:

[0027]

[0028] in, This represents the activation function. Through the above convolutional feature extraction process, feature enhancement and target representation of the weld defect region are achieved.

[0029] Furthermore, such as Figure 3 To address the issues of insufficient extraction of minute defect features, inadequate response capability of effective feature channels, and difficulty in balancing accuracy and complexity under embedded deployment conditions in lightweight target detection networks for bearing surface defect identification, the YOLOv3-tiny network is improved by introducing residual units and channel attention units in the backbone feature extraction part to enhance the network's feature representation and selection capabilities for defect targets.

[0030] The improved network still employs a dual-detection branch architecture, with an input image size of 416×416×3. After multiple levels of convolution and pooling operations, the input image sequentially forms feature maps of different scales, including 208×208×16, 104×104×32, 52×52×64, 26×26×128, 26×26×256, 13×13×512, and 13×13×1024. Based on the above feature extraction process, residual units are introduced into the mid-to-late feature layers, and channel attention units are set at the output of the residual units to achieve enhanced representation of key defect features.

[0031] A first residual unit is set at the 26×26×256 feature layer position, and a second residual unit is set at the 13×13×512 feature layer position. Each residual unit includes a main transport branch and a bypass branch. The main transport branch includes a 1×1 convolutional layer and a 3×3 convolutional layer connected in sequence. The bypass branch is used for direct transport of the input feature map. The outputs of the main transport branch and the bypass branch are summed element-wise under the condition that the feature map size is the same to obtain the residual enhanced feature map. By setting the residual units, the ability of the mid-to-high-level feature maps to preserve defect edge information, texture information and local change information is improved without significantly increasing the network parameter scale and computational load, and the gradient transfer effect during network training is improved.

[0032] A channel attention unit is set at the output of the residual enhanced feature map. The channel attention unit adaptively calculates the importance of each channel of the input feature map and adjusts the weighted feature responses of each channel based on the calculation results to enhance effective channel features related to the defect target and suppress background noise channels and redundant channels. The channel attention unit includes a global information aggregation module, a channel weight generation module, and a channel weighting module. After global information aggregation, the input feature map forms statistical descriptors for each channel. These statistical descriptors are mapped by the weight generation module to form weight coefficients for the corresponding channels. Each channel weight coefficient is then weighted sequentially with the input feature map to obtain the channel-recalibrated output feature map.

[0033] The first residual unit corresponding to the 26×26×256 feature layer is connected to the first channel attention unit, and the second residual unit corresponding to the 13×13×512 feature layer is connected to the second channel attention unit. The 26×26 scale feature map retains more spatial detail information and participates in the feature fusion process of subsequent shallow detection branches. Setting the first channel attention unit at this position can improve the network's response strength to channels related to small-sized defects such as cracks, pitting, and scratches. The 13×13 scale feature map contains strong high-level semantic information. Setting the second channel attention unit at this position can improve the network's ability to filter and represent features for different defect categories.

[0034] The improved network detection part still employs a dual-scale detection mechanism. The deep feature map is convolved to form the output of the first detection branch; the other branch upsamples the deep feature map after dimensionality reduction via convolution and concatenates it with a 26×26 scale feature map from the shallow network to form the input of the second detection branch. Since the shallow feature map has undergone residual enhancement and channel recalibration before feature fusion, the fused feature map possesses higher defect representation capabilities, which is beneficial for improving the detection accuracy of small-sized defect targets, low-contrast defect targets, and edge-blurred defect targets.

[0035] The residual units and channel attention units are positioned to match the size and number of channels of the input and output feature maps, thus avoiding the introduction of additional feature size alignment operations and complex data transformation processes. This reduces hardware implementation complexity and improves structural feasibility. While maintaining the lightweight characteristics of YOLOv3-tiny, the improved network enhances defect feature extraction capabilities, effective feature selection capabilities, and overall network robustness by introducing a limited number of residual units and channel attention units, making it suitable for deployment on resource-constrained ZYNQ embedded hardware platforms.

[0036] Compared with the original YOLOv3-tiny network without residual units and channel attention units, this implementation can improve the recognition accuracy and detection stability in bearing surface defect detection tasks while maintaining high inference efficiency. It is especially suitable for target detection application scenarios with small-sized defects, low-contrast defects and complex background interference conditions.

[0037] To adapt to the hardware resources of the ZYNQ platform, the trained YOLO model is quantized before deployment to reduce computational complexity and parameter storage overhead. The quantization process can be represented as:

[0038] in, Represents floating-point parameters or feature data. Indicates the quantification scaling factor. This represents the fixed-point value after quantization. Correspondingly, the dequantization process can be represented as:

[0039] Through the above quantization process, the original floating-point model can be converted into a fixed-point model suitable for hardware implementation, thereby reducing the storage overhead and computing resource consumption of convolutional neural networks during edge deployment.

[0040] In the process of accelerating convolution, this embodiment uses Vitis HLS to perform high-level synthesis design on the convolution computation module and generate a convolution acceleration IP core. The convolution acceleration IP core includes an input feature map caching unit, a weight caching unit, a multiply-accumulate operation unit, and an output write-back unit. The computational complexity of the convolutional layer can be expressed as:

[0041]

[0042] in, and These represent the height and width of the output feature map, respectively. Indicates the number of input channels. Indicates the number of output channels. This represents the convolution kernel size. Regarding the above calculation process, the HLS design applies pipeline optimization to the inner convolution loop to ensure a streamlined execution of data reading, multiplication-accumulation operations, and result write-back; it applies unroll optimization to some channel dimensions and convolution kernel dimensions to construct parallel multiplication-accumulation units; it applies array partition optimization to the input buffer, weight buffer, and output buffer to improve the concurrent access capability of on-chip storage; and it applies dataflow optimization to the convolution calculation, data transmission, and result output processes to achieve task-level parallel processing. If the parallelism is... The hardware operating frequency is The computation delay of the convolutional layer can be approximated as follows:

[0043] Therefore, by increasing the parallelism It can effectively reduce the latency of convolution calculation and improve the efficiency of model inference.

[0044] During the target detection result parsing stage, the YOLO network outputs the bounding box parameters and category information of the defect target. The center coordinates and width and height of the bounding box can be expressed as:

[0045]

[0046]

[0047] in, , , , This represents the network's predicted output. Indicates the current grid offset. Indicates the prior frame size. This represents the Sigmoid function. , These represent the center coordinates and width and height of the predicted bounding box, respectively. Based on the bounding box parameters and class probabilities, the location and class information of the weld surface defect target are obtained.

[0048] In the hardware integration stage, such as Figure 4 The convolution acceleration IP core generated by Vitis HLS is imported into the Vivado design environment, and a Block Design based on the ZYNQ-7100 platform is built in Vivado. The Block Design includes at least a ZYNQ Processing System module, an AXI DMA module, a convolution acceleration IP core, a DDR memory interface, an OV5640 image acquisition interface, and LCD display-related video output modules. After system synthesis, implementation, and bitstream generation, the XSA hardware platform file is exported, and a bare-metal control program is written in the Vitis environment to implement functions such as image acquisition, data transfer, convolution inference, result parsing, SD card read / write, and display output.

[0049] During system operation, the OV5640 camera acquires images of the weld surface and writes them to the DDR. The PS end transmits the image to be inspected to the convolution acceleration IP core at the PL end via AXI DMA to complete the main convolution operations in the forward inference of the YOLO model. After the inference results are written back to the DDR, the PS end performs bounding box parsing and result overlay, and outputs the detection results to the LCD display terminal. At the same time, the PS end can read and write data to the SD card through the file system interface to save the image to be inspected, model parameters, and detection results.

[0050] This embodiment achieves real-time detection of surface defects in welds of steel components in power transmission towers by deploying the YOLO model on the ZYNQ-7100 heterogeneous platform and combining it with OV5640 image acquisition, Vitis HLS convolution acceleration IP core design, Vivado system integration, and PS-PL collaborative control. Compared with traditional host computer or GPU detection solutions, this embodiment meets the detection accuracy requirements while offering better real-time performance, lower power consumption, and stronger edge deployment capabilities.

[0051] The parts of this invention not described in detail are well-known to those skilled in the art.

Claims

1. A method for detecting surface defects in weld seams of components, characterized in that, Includes the following steps: S1: A training network is built based on the YOLOv3 target detection network. An adaptive residual connection unit is introduced into the feature extraction part of the training network, and a channel attention unit is introduced to weight the channel response of the feature map in order to enhance the learning of the features related to the small defects of the weld during the training stage. S2: Construct a weld defect training dataset, train the network using the training network, and select the optimal network model parameters; after obtaining the optimal network model parameters, convert the training network into an inference network for deploying inference; S3: Perform model optimization processing on the inference network, including fusing the parameters of the convolutional layer with its adjacent batch normalized layer to obtain an equivalent convolutional layer, and performing fixed-point quantization on the fused model parameters and / or feature data to generate a quantized inference model; save the quantized inference model parameters to an accessible non-volatile storage medium. S4: Deploy and detect defects in the quantized inference model: The processing system PS reads the surface image of the weld to be detected and the parameters of the quantized inference model, and transmits the input feature data and convolution weight data to the programmable logic PL via the AXI interface; The PL end calls the convolution acceleration IP core generated by high-level synthesis to perform hardware acceleration of the convolution operation in the quantization inference model, and sends the operation result back to the PS end via the AXI interface; the PS end performs bounding box decoding, confidence calculation and non-maximum suppression post-processing on the network output, and outputs the category and location of weld surface defects and displays them.

2. The method for detecting surface defects in weld seams of components according to claim 1, characterized in that: The adaptive residual connection unit includes a main branch and a skip branch. The main branch performs a convolutional transformation on the input feature map to obtain the main branch features, and the skip branch performs pass-through on the input feature map. The adaptive residual connection unit performs weighted fusion of the main branch features and the skip branch features, and the fused output satisfies the following form: ， Where x is the input feature map, F(x) is the output of the main branch convolution transformation, and α(x) and β(x) are learnable parameters or weight coefficients generated by the gating function.

3. The method for detecting surface defects in weld seams of components according to claim 1, characterized in that: The channel attention unit is a Squeeze-and-Excitation channel attention structure or an equivalent channel recalibration structure. The channel attention unit includes a global average pooling module, a channel weight generation module, and a channel weighting module. The global average pooling module performs channel statistical description on the input feature map, the channel weight generation module outputs the weights of each channel through a gating function, and the channel weighting module weights the input feature map by channel to obtain the channel recalibrated feature map.

4. The method for detecting surface defects in component welds according to claim 1, characterized in that: The construction of the weld defect training dataset involves preprocessing the collected weld surface images of components and defining the location and category of the defect targets.

5. The method for detecting surface defects in weld seams of components according to claim 1, characterized in that: Steps S1 and S2 further include: dividing the weld defect training dataset into training and validation sets; adjusting at least one training hyperparameter during training and selecting the optimal network model parameters based on a preset evaluation index, wherein the training hyperparameter includes at least one of learning rate, batch size, input resolution, and data augmentation strategy.

6. The method for detecting surface defects in weld seams of components according to claim 1, characterized in that: The parameter fusion of the convolutional layer and batch normalization layer in step S3 includes: folding the scaling and bias parameters of the batch normalization layer into the weights and biases of the convolutional layer to remove the batch normalization layer; the fixed-point quantization in step S4 includes: mapping the floating-point weights and / or activations to fixed-point representations based on the quantization scale factor and optional zero-point; the fixed-point quantization bit width is 8 bits.

7. The method for detecting surface defects in weld seams of components according to claim 1, characterized in that: Step 4: The convolution acceleration IP core is generated by high-level synthesis tools and deployed on the PL side. The convolution acceleration IP core is connected to the AXI DMA module through the AXI4 storage mapping interface or the AXI4-Stream interface to transfer data between the external memory on the PS side and the convolution acceleration IP core on the PL side. The convolution acceleration IP core applies pipeline optimization to the inner convolution loop, loop unrolling optimization to the convolution channel dimension and convolution kernel dimension, storage partitioning optimization to the on-chip cache, and task-level data flow optimization.

Images