A ZYNQ neural network acceleration system for aluminum surface micro-defect detection

Abstract

The invention discloses a ZYNQ neural network acceleration system for aluminum surface micro-defect detection. A LADFastDet network for aluminum surface micro-defect detection is deployed on a ZYNQ platform. The PL end of the ZYNQ platform includes an accelerator module, which includes a convolution IP core and a sampling IP core. All convolutions in the LADFastDet network structure are designed as convolution IP cores, and upsampling and maximum pooling are designed as sampling IP cores. The convolution IP core is provided with multiple read-write parallel channels and interfaces, and the sampling IP core is provided with multiple read-write interfaces. The PS end is responsible for the configuration of the LADFastDet network structure, the calling of the accelerator module and the management of the overall control process. The PL end calls and executes the functions of the convolution IP core and the sampling IP core according to the configuration of the PS end. The invention improves the accuracy and speed of aluminum surface micro-defect detection under a limited resource ZYNQ platform.

Description

Technical Field

[0001] The present invention relates to the field of neural network and hardware acceleration technology, and in particular to a ZYNQ neural network acceleration system for aluminum surface micro-defect detection. Background Art

[0002] The production process of aluminum is easily affected by external environmental and operational factors, which can lead to various surface defects. If these defects are not detected and repaired in a timely and effective manner, it will not only affect the quality of aluminum products and reduce their performance, but also increase production costs and may even cause safety hazards. Traditional non-destructive testing such as infrared testing, ultrasonic testing and eddy current flaw detection is slow, costly and complex to operate.

[0003] In recent years, deep learning technology has been widely used in aluminum defect detection. By utilizing the powerful feature extraction and fusion capabilities of convolutional neural networks (CNNs), features are learned and extracted from aluminum defect data. However, these high-performance models are often bulky and require extremely high computing resources, making it difficult to meet the needs of real-time detection on mobile devices. Therefore, in order to meet the needs of accelerating CNNs on edge computing platforms, it is necessary to study lightweight detection methods and hardware acceleration methods.

[0004] Existing model lightweighting methods can be summarized into two categories: the first category is to lightweight the existing network model through model pruning, quantization and knowledge distillation, and directly apply it to the trained model. This type of method can achieve fast lightweighting, but this process will introduce accuracy loss, and it is difficult to adjust the network according to the application scenario to achieve the optimal performance of hardware resources; the second category is to use the strategy of designing lightweight structure to achieve network lightweighting. It is common to make lightweight designs based on typical networks such as YOLO series, GhostNet, MobileViTv2, etc. Compared with the first type of method, the method of improving the existing network to make it lightweight can fundamentally reduce the redundancy of calculation and parameter amount, but this type of method often relies on the existing network architecture, and there is limited room for improvement in accuracy and speed.

[0005] FPGA is a commonly used hardware acceleration platform for convolutional neural networks. There are two main accelerator design methods: the first is to use a single FPGA platform to design an accelerator. Its advantage is that it can make full use of the parallel resources of FPGA, realize customized hardware acceleration logic, and improve computing efficiency. However, its design complexity is high and its adaptability to different CNN models is poor. Each time the model is updated, the accelerator needs to be redesigned and adjusted; the second is to implement ARM+FPGA heterogeneous design accelerators on the ZYNQ platform. This design combines the flexible control capabilities of ARM processors with the parallel computing capabilities of FPGAs, so that the system can handle complex control logic and efficiently perform parallel computing tasks. Compared with a single FPGA platform, the design of CNN accelerators on the ZYNQ platform allows resources to be reasonably allocated while maintaining high performance, enhancing the programmability and scalability of the system, and providing a more comprehensive and flexible solution for CNN hardware acceleration. However, most studies are committed to realizing direct mapping of CNN models on FPGAs. Although the speed and throughput of data computing flow can be improved through the optimization of computing architecture, the memory access efficiency and data parallelism of input and output data reading and writing are not considered, which affects data transmission efficiency and memory access speed.

[0006] Therefore, it is urgent to study a lightweight acceleration method for aluminum surface micro-defect detection in response to the needs of improving the accuracy and speed of aluminum surface micro-defects under the limited resource ZYNQ platform. Summary of the invention

[0007] 1. Technical issues to be resolved

[0008] Based on the above problems, the present invention provides a ZYNQ neural network acceleration system for aluminum surface micro-defect detection, which solves the problem that the accuracy and speed of aluminum surface micro-defect detection under the limited resource ZYNQ platform need to be improved.

[0009] (II) Technical solution

[0010] Based on the above technical problems, the present invention provides a ZYNQ neural network acceleration system for aluminum surface micro-defect detection, and deploys a LADFastDet network for aluminum surface micro-defect detection on a ZYNQ platform: all convolution modules in the LADFastDet network structure are designed as convolution IP cores, and upsampling modules and maximum pooling modules are designed as sampling IP cores; the ZYNQ platform adopts a heterogeneous computing architecture of a PL end and a PS end, the PL end includes an accelerator module, the accelerator module includes the convolution IP core and the sampling IP core, the convolution IP core is provided with multiple read-write parallel channels and interfaces, and the sampling IP core is provided with multiple read-write interfaces; the PS end is responsible for the configuration of the LADFastDet network structure, the calling of the accelerator module and the management of the overall control process, the PL end calls and executes the functions of the convolution IP core and the sampling IP core according to the configuration of the PS end, and the data communication between the PS end and the PL end is realized through an AXI bus;

[0011] The LADFastDet network structure includes an input layer, a feature extraction layer, a prediction layer and an output layer; the input layer sets the height*width*channel to be a 3*416*416 RGB image; the feature extraction layer includes two 3*3 convolutions with a step size of 2, three lightweight inverted residual structure blocks and a multi-scale global feature fusion structure, which are respectively used for feature extraction and dimensionality reduction, depth enhancement feature extraction and multi-level feature fusion, and the lightweight inverted residual structure block includes the first 3*3 depth convolution, the first 1*1 point-by-point convolution, the second 3*3 depth convolution, and the output of the first 1*1 point-by-point convolution is concat-merged and connected, and after the second 1*1 point-by-point convolution, it is concat-merged and connected with the output of the first 3*3 depth convolution and then M axpool maximum pooling; the multi-scale global feature fusion structure includes a 5*5 depth convolution, a third 3*3 depth convolution, and a third 1*1 point-by-point convolution, respectively. The output of the third 3*3 depth convolution is concat-merged with the output of the 5*5 depth convolution, and then concat-merged with the output of the third 1*1 point-by-point convolution after the fourth 1*1 point-by-point convolution and then output; the prediction layer includes two detection heads of different scales, one detection head includes 3*3 depth separable convolution and 3*3 convolution in sequence, and the other detection head includes upsampling, 3*3 depth separable convolution and 3*3 convolution in sequence; the output layer includes two output feature maps with height*width*channels of 27*13*13 and 27*26*26 respectively.

[0012] Furthermore, the convolution IP core reads three kinds of data, namely, input feature map, weight and bias parameter, and writes output feature map data; the sampling IP core reads input feature map data and writes output feature map data.

[0013] Furthermore, the convolution IP core adopts optimization strategies of loop expansion, pipelining, data blocking and ping-pong buffering of data flow.

[0014] Furthermore, the sampling IP core adopts optimization strategies of loop unrolling, pipelining, data blocking and ping-pong buffering of data stream.

[0015] Furthermore, it also includes: inputting an aluminum material defect data set to train the LADFastDet network, and the trained LADFastDet network outputs a detection result of a defect category.

[0016] Furthermore, the defect categories include pinholes, scratches, dirt, and wrinkles.

[0017] Furthermore, data enhancement processing is performed on the input aluminum material defect data set, and the data enhancement processing includes copying and pasting and Gaussian filtering.

[0018] Furthermore, the input aluminum material defect data set is also subjected to data augmentation processing, and the data augmentation processing includes random flipping, changing brightness, rotating, cropping, translating and mirroring.

[0019] (III) Beneficial effects

[0020] The above technical solution of the present invention has the following advantages:

[0021] (1) The present invention constructs an efficient and accurate LADFastDet network structure through a lightweight inverted residual structure and a multi-scale feature fusion module; at the same time, by adopting deep optimization of the computing architecture and designing multi-interface and multi-channel memory access methods in the input and output reading and writing stages, the LADFastDet high-performance accelerator module is realized, thereby realizing fast, accurate and efficient aluminum surface micro-defect detection on the ZYNQ platform;

[0022] (2) The present invention constructs a lightweight aluminum surface micro-defect detection model LADFastDet, in which the lightweight inverted residual structure block maintains the model depth through inverted residual connection and reverse bottleneck, avoids gradient vanishing, enhances cross-layer fusion of features, and deeply extracts image details and features, so that the model can reduce model parameters and calculation amount while maintaining high precision; the multi-scale feature fusion structure obtains different receptive field feature maps through modules with different convolution kernel sizes, obtains feature information of different scales, and enhances the network's ability to extract defects of different scales, especially the defect detection ability of two types of high-precision small targets such as scratches and pinholes. It is significantly improved; thereby improving the accuracy and speed of aluminum surface micro-defect detection under the limited resource ZYNQ platform;

[0023] (3) Based on the LADFastDet structure, the present invention designs high-performance convolution IP cores and sampling IP cores, disperses interface loads and reduces data transmission delays through a multi-channel multi-interface data read-write architecture, and combines optimization strategies such as loop unrolling and ping-pong buffering to improve accelerator computing parallelism, throughput, and data transmission speed, further improving the accuracy and speed of aluminum surface micro-defect detection;

[0024] (4) This invention has certain theoretical significance and application value for the deployment of neural networks on mobile platforms with limited resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0026] Figure 1 It is an overall schematic diagram of the ZYNQ neural network acceleration system for aluminum surface micro-defect detection according to an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of the overall structure of the LADFastDet network according to an embodiment of the present invention;

[0028] Figure 3 This is a feature extraction layer structure diagram of LADFastDet according to an embodiment of the present invention;

[0029] Figure 4 This is a diagram of a lightweight inverted residual structure according to an embodiment of the present invention;

[0030] Figure 5 It is a multi-scale feature fusion structure of an embodiment of the present invention;

[0031] Figure 6 The overall architecture of the aluminum surface defect detection acceleration system based on ZYNQ according to an embodiment of the present invention;

[0032] Figure 7 A schematic diagram of the optimized design of a convolution IP core according to an embodiment of the present invention;

[0033] Figure 8 A schematic diagram of a sampling acceleration IP core according to an embodiment of the present invention;

[0034] Fig. 9 The following is a visualization of the results of testing 8 algorithms on the AMDataset dataset. DETAILED DESCRIPTION

[0035] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0036] The embodiment of the present invention is a ZYNQ neural network acceleration system for aluminum surface micro-defect detection, such as Figure 1 As shown. In order to realize fast and accurate detection of micro defects on aluminum surface on ZYNQ-7020 platform with limited computing resources, it is necessary to abandon the method of fine-tuning based on the existing model, and design the network model from the perspective of high precision, high efficiency, and fewer network parameters and computational complexity; at the same time, design a hardware accelerator based on the network model to maximize resource utilization and optimize performance. The present invention studies the lightweight inverted residual structure and multi-scale feature fusion module to construct an efficient and accurate LADFastDet; at the same time, by adopting deep optimization of the computing architecture, multi-interface and multi-channel memory access methods such as input and output read and write stages are designed to realize LADFastDet high-performance accelerator, thereby realizing fast, accurate and efficient detection of micro defects on aluminum surface on ZYNQ platform. The details are as follows:

[0037] like Figure 2 As shown in the figure, the design idea of ​​the input layer, feature extraction layer, prediction layer and output layer of LADFastDet is as follows: the input layer is set to a 3*416*416 RGB image, the feature extraction layer is designed as a lightweight structure, including two 3*3 convolutions with a step size of 2, three lightweight inverted residual structure blocks and a multi-scale global feature fusion structure, which are used for feature extraction and dimensionality reduction, deep enhancement feature extraction and multi-level feature fusion respectively. The lightweight inverted residual structure block includes the first 3*3 depth convolution, the first 1*1 point-by-point convolution, the second 3*3 depth convolution, and the first 1*1 point-by-point convolution. After the output of the convolution is concat-merged and connected, after the second 1*1 point-by-point convolution, it is concat-merged and connected with the output of the first 3*3 depth convolution and then Maxpool is performed; the multi-scale global feature fusion structure includes a 5*5 depth convolution, a third 3*3 depth convolution, and a third 1*1 point-by-point convolution, respectively, and the output of the third 3*3 depth convolution is concat-merged and connected with the output of the 5*5 depth convolution, and after the fourth 1*1 point-by-point convolution, it is concat-merged and connected with the output of the third 1*1 point-by-point convolution and then output. For the detection of targets of different sizes, two detection heads of different scales are designed in the prediction layer, one detection head includes 3*3 depth separable convolution and 3*3 convolution in sequence, and the other detection head includes upsampling, 3*3 depth separable convolution and 3*3 convolution in sequence, corresponding to the output images of the two height*width*channels of the output layer of 27*13*13 and 27*26*26.

[0038] The feature extraction structure of LADFastDet is as follows Figure 3As shown. Two 3×3 convolutions with a stride of 2 are used for feature image extraction and dimensionality reduction, reducing image dimensions and removing redundant information to accelerate the extraction of key features. The lightweight inverted residual structure uses inverted residual connections and reverse bottlenecks to maintain the depth of the model while avoiding gradient vanishing, enhancing cross-layer fusion of features, and deeply extracting image details and features. The network uses channel-by-channel convolution and point-by-point convolution instead of standard convolution, which can reduce model parameters and computation while maintaining high accuracy. The multi-scale feature fusion structure obtains different receptive field feature maps through modules with different convolution kernel sizes, obtains feature information of different scales, and enhances the network's ability to extract defects of different scales, especially for high-precision small-sized targets such as pinholes and scratches.

[0039] The lightweight inverted residual structure in the feature extraction layer is shown in the figure Figure 4 As shown. Input X0∈R H×W×C , after the first depthwise convolution with a convolution kernel size of 3×3, X1∈R is generated H×W×C Next, in order to enable the network to expand feature expression and obtain richer features, 1×1 Pointwise convolution is used to increase the dimension to generate X2∈R H ×W×2C ,exist Figure 4 In the case of the lightweight inverted residual structure in (a), the features after the second 3×3 depthwise convolution Depthwise Conv are connected with X2 through Concat to obtain X3∈R H×W×4C ,exist Figure 4 In the case of (b), X2 generates X3∈R after the third 3×3 depthwise convolution Depthwise Conv H×W×C :

[0040]

[0041] In order to extract more critical and useful information from the features after dimensionality increase, 1×1 Pointwise convolution is used to reduce the dimension and then connected with X1 through Concat to obtain X4∈R H×W×2C :

[0042] X4=Concat[X1,X PW (X3)] (2)

[0043] Perform Maxpool operation on the feature map to obtain X out :

[0044] X out =Maxpool(X4) (3)

[0045] In the aluminum surface defect image, the scales of the defects vary greatly, and the feature information of small-sized targets is easily concealed. Therefore, a multi-scale feature fusion module is designed to fuse the feature information of receptive fields of different sizes and improve the detection capability of small-sized defect features on the aluminum surface.

[0046] The structure of the multi-scale feature fusion module is as follows: Figure 5 As shown. In order to obtain multi-scale information, input X∈R H×W×C , through multi-scale convolution, three mapping layers Y1∈R with different receptive field sizes are designed H×W×C , Y2∈R H×W×C , Y3∈R H×W×C , the convolution kernel sizes are 1×1, 3×3, and 5×5 respectively. Y2 and Y3 are connected through Concat and then convolved with 1×1 to generate Y PW Then perform the Concat channel connection with Y1 to obtain Y:

[0047] Y=Concat[Y1,Y PW (Concat[Y2,Y3])] (4)

[0048] In addition, the hardware accelerator of LADFastDet has been improved. When deploying the LADFastDet network on the ZYNQ platform, it is necessary to design a high-performance hardware accelerator because the hardware resource limitations affect the running speed. Common hardware accelerator designs mainly consider optimization methods such as data computing architecture optimization and PINGPONG buffering to improve throughput and reasoning speed, without considering the impact of the input and output read and write stages on memory access speed.

[0049] Therefore, the present invention designs all convolution modules in LADFastDet, including standard convolution, depth convolution, point-by-point convolution and depth-separable convolution modules, into convolution IP cores, and designs upsampling modules and maximum pooling modules into sampling IP cores. The convolution IP core adds multi-channel and multi-interface memory access design in the input and output read and write stages. The sampling IP core adds input and output read and write interfaces to improve memory access speed and efficiency. The convolution IP core designs three interfaces, each with 4 channels, and the sampling IP core designs two input and output interfaces. The acceleration system designed in this paper is as follows: Figure 6 As shown in the figure, the system adopts a heterogeneous computing architecture of PL and PS, in which the PS side is responsible for the configuration of the LADFastDet network structure, the call of the accelerator module and the management of the overall control process, and the PL side includes the accelerator module. According to the configuration of the PS side, it flexibly calls and executes the functions of the convolution IP core and the sampling IP core to ensure high-speed data transmission. Data communication between the PS side and the PL side is realized through the AXI bus.

[0050] The LADFastDet accelerator module needs to read and write a large amount of data in the input and output stages. For example, the convolution IP core calculation needs to read three types of data: input feature map, weight and bias parameters, and write output feature map data; the sampling IP core calculation needs to read input feature map data and write output feature map data. The common processing of these data relies on a single AXI interface to access off-chip memory, which limits the efficiency of data transmission and the parallel processing capability of the system. The present invention proposes a multi-channel multi-interface read-write parallel transmission optimization strategy, which designs multiple read-write parallel channels and interfaces for the convolution IP core, and designs multiple read-write interfaces for the sampling IP core to disperse the interface load and reduce data transmission delay.

[0051] Through multi-interface multi-channel read and write parallel transmission, compared with a single AXI interface, the acceleration ratio R of accelerator data transmission is shown in (5). in +T in+w , T w +T in+w , T out , T in+w They represent the time to read the input feature map, the time to read the weights and biases, the time to write the output feature map data, and the time to read the input feature map and weights at the same time. N represents the number of channels of each interface.

[0052]

[0053] like Figure 7 As shown in the figure, the convolution IP core adopts optimization strategies such as loop expansion, pipelining, data blocking and PINGPONG buffering of data flow, which promotes parallel processing and throughput improvement in the calculation process, reduces data transmission costs and time delays, and thus improves the calculation speed. However, although these loop strategies improve parallelism, they also increase the memory access volume and access frequency of the convolution IP core. At the same time, the ping-pong buffer also requires the convolution IP core to load, calculate and output data at the same time, but the single AXI interface is prone to become a bottleneck in a single-channel configuration, causing data read and write conflicts and delay accumulation. Therefore, the convolution IP core introduces a multi-channel and multi-interface design to achieve parallel and efficient access to memory, effectively alleviate access conflicts, shorten data transmission delays, and significantly improve the convolution calculation speed and overall system performance.

[0054] Similar to the convolution IP core design, the sampling IP core uses loop expansion, pipelining, blocking, and PINGPONG cache optimization strategies for data flow to improve parallel capabilities and core throughput, and optimize data access methods to increase data transmission speed. Figure 8As shown in the figure, the sampling IP core is designed to read and write the input interface and output interface in parallel, which reduces the input and output waiting time and improves the transmission efficiency. IN is used to read the input feature map data, and OUT is used to write the output feature map.

[0055] The aluminum material defect dataset (AMDataset) of the present invention is generated by using a Hikvision industrial camera model MV-CS050-10GC-PRO. The defect categories include pinholes, scratches, dirt, and wrinkles. In the dataset, the pixel size of each image is 1280×720, and the precision area of ​​the pinhole is about 1mm 2 , the highest accuracy of scratches is about 0.05mm. In the detection process, it is easy to miss small targets such as pinholes and scratches. In order to enhance the feature information of small targets, the pinhole and scratch defect data are enhanced by Copy-pasting and Gaussian Blur filtering. At the same time, in order to solve the problem of insufficient samples, the present invention enhances the data by random flipping, cutout, changing brightness, rotating, cropping, translating and mirroring to expand the data set. The data set is randomly divided into training set, validation set and test set.

[0056] The model training adopts the PyTorch framework, and the training GPU is an NVIDIA GeForce RTX 3060Ti with 8GB memory. The positioning loss during training adopts the EIoU loss function, and its calculation formula is shown in (6). Where b is the center point of the predicted box, b^gt is the center point of the real box, w and h are the width and height of the predicted box, w^gt and h^gt are the width and height of the real box, and w^c and h^c are the width and height of the minimum external box covering the two boxes.

[0057]

[0058] In the training experiment, the input image size is 416×416, the training BatchSize is set to 32, and the maximum number of Epochs is 1000. We choose the stochastic gradient descent algorithm (SGD) with momentum as the optimizer, and the momentum factor is set to 0.937 to accelerate convergence and reduce oscillation. The learning rate is initially set to 10 -2 , and use the cosine annealing strategy to gradually reduce the learning rate until the learning rate drops to 10 -4 At the same time, in order to prevent the model from overfitting, the weight decay coefficient is set to 5×10 -4 In the prediction, the post-processing method of non-maximum suppression algorithm is adopted, NMS_IoU is set to 0.3, and the confidence is set to 0.5.

[0059] In order to verify the technical effect of the present invention, Precision (P), Recall (R), Average Precision (AP), mean Average Precision (mAP) and F1-Score (F1) are used as evaluation indicators of the detection algorithm. Among them, Precision refers to the proportion of samples that are truly positive among the samples predicted by the model to be positive. Recall refers to the proportion of samples that are correctly predicted by the model to be positive among all true positive samples. AP (Average Precision) is the average of the precision at different recall levels, which measures the average performance of the model over the entire recall range. F1-Score is the harmonic mean of precision and recall, which is used to balance precision and recall. mAP is the mean of AP of all categories, which indicates the overall performance of the model on all categories. The higher the mAP value, the better the detection performance of the model on all categories.

[0060] In addition, the present invention uses Floating Point Operations (FLOPs), model parameters and Model Size to evaluate the lightweightness of the model.

[0061] The development board of the embodiment of the present invention is ZYNQ-7020, and the main chip model is XC7Z020-CLG400-2. ZYNQ-7020 is mainly composed of two parts: the processing system (PS) and the programmable logic (PL). The PS system integrates two ARM Cortex TM -A9 processor, the PL side includes 4.9Mb of BRAM (280 18Kb BRAMs), 220 multipliers (DSPs), 106,400 flip-flops (FFs), and 53,200 look-up tables (LUTs).

[0062] After a comprehensive evaluation of the test set, the detection results of various defects are summarized based on a series of evaluation indicators, as shown in Table 1. Among them, the scratch defect has an increased detection difficulty due to its extremely high precision, and its recall rate and AP are slightly lower than the average level, at 93.49% and 95.37% respectively. All other detection indicators reach 96% and above, indicating that the overall performance of the LADFastDet model is good.

[0063] Table 1 Experimental results of LADFastDet model on various defects

[0064]

[0065] We selected 7 typical lightweight models on AMDataset to conduct comparative experiments with LADFastDet, and the experimental results are shown in Table 2. As can be seen from Table 2, the Precision, Recall, F1 score and mAP values ​​of LADFastDet are better than other models, which are 98.03%, 97.16%, 0.975 and 97.51% respectively, and the number of parameters and Model Size are lower than other models. Although the FLOPs of LADFastDet is slightly higher than that of MobileNetv2, it is only 0.53G, which is much lower than other models, indicating that LADFastDet has certain advantages in the detection of micro-defects on aluminum surfaces.

[0066] From a qualitative perspective, we visualized the reasoning results of the LADFastDet network in the AMDataset dataset and compared it with the detection results of other advanced networks on the AMDataset dataset, inputting five photos for detection. Fig. 9 In the figure, we use different red, green, blue and purple color detection frames to distinguish and encode different types of defects. The information in the detection image contains data on the defect location, category and its detection accuracy.

[0067] Table 2 Experimental results of LADFastDet model compared with other models

[0068]

[0069]

[0070] The performance indicators of the hardware platform are mainly evaluated through system resource occupancy, power consumption, execution time, throughput, resource consumption efficiency, and energy efficiency ratio.

[0071] The LADFastDet model was experimented on the CPU, GPU and ZYNQ platforms, and the results are shown in Table 3. As can be seen from Table 3, the energy efficiency ratio of the ZYNQ-7020 platform to achieve LADFastDet acceleration is 11.58GOPS / W, which is 44.54 times that of the CPU and 28.24 times that of the GPU, showing the significant advantage of the ZYNQ platform in energy efficiency ratio.

[0072] Table 3 Performance comparison of different platforms

[0073]

[0074] In summary, the ZYNQ neural network acceleration system for aluminum surface micro-defect detection has the following beneficial effects:

[0075] (1) The present invention constructs an efficient and accurate LADFastDet network structure through a lightweight inverted residual structure and a multi-scale feature fusion module; at the same time, by adopting deep optimization of the computing architecture and designing multi-interface and multi-channel memory access methods in the input and output reading and writing stages, the LADFastDet high-performance accelerator module is realized, thereby realizing fast, accurate and efficient aluminum surface micro-defect detection on the ZYNQ platform;

[0076] (2) The present invention constructs a lightweight aluminum surface micro-defect detection model LADFastDet, in which the lightweight inverted residual structure block maintains the model depth through inverted residual connection and reverse bottleneck, avoids gradient vanishing, enhances cross-layer fusion of features, and deeply extracts image details and features, so that the model can reduce model parameters and calculation amount while maintaining high precision; the multi-scale feature fusion structure obtains different receptive field feature maps through modules with different convolution kernel sizes, obtains feature information of different scales, and enhances the network's ability to extract defects of different scales, especially the defect detection ability of two types of high-precision small targets such as scratches and pinholes. It is significantly improved; thereby improving the accuracy and speed of aluminum surface micro-defect detection under the limited resource ZYNQ platform;

[0077] (3) Based on the LADFastDet structure, the present invention designs high-performance convolution IP cores and sampling IP cores, disperses interface loads and reduces data transmission delays through a multi-channel multi-interface data read-write architecture, and combines optimization strategies such as loop unrolling and ping-pong buffering to improve accelerator computing parallelism, throughput, and data transmission speed, further improving the accuracy and speed of aluminum surface micro-defect detection;

[0078] (4) This invention has certain theoretical significance and application value for the deployment of neural networks on mobile platforms with limited resources.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the embodiments of the present invention are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations shall fall within the scope defined by the appended claims.

Claims

1. A ZYNQ neural network acceleration system for aluminum surface micro-defect detection, characterized in that: Deploy the LADFastDet network for aluminum surface micro-defect detection on the ZYNQ platform: design all convolution modules in the LADFastDet network structure into convolution IP cores, and design the upsampling module and the maximum pooling module into sampling IP cores; The ZYNQ platform adopts a heterogeneous computing architecture of the PL side and the PS side. The PL side includes an accelerator module, and the accelerator module includes the convolution IP core and the sampling IP core. The convolution IP core is provided with multiple read-write parallel channels and interfaces, and the sampling IP core is provided with multiple read-write interfaces. The PS side is responsible for the configuration of the LADFastDet network structure, the calling of the accelerator module and the management of the overall control process. The PL side calls and executes the functions of the convolution IP core and the sampling IP core according to the configuration of the PS side. The data communication between the PS side and the PL side is realized through the AXI bus. The LADFastDet network structure includes an input layer, a feature extraction layer, a prediction layer and an output layer; the input layer sets the height*width*channel to be a 3*416*416 RGB image; the feature extraction layer includes two 3*3 convolutions with a step size of 2, three lightweight inverted residual structure blocks and a multi-scale global feature fusion structure, which are respectively used for feature extraction and dimensionality reduction, depth enhancement feature extraction and multi-level feature fusion, and the lightweight inverted residual structure block includes the first 3*3 depth convolution, the first 1*1 point-by-point convolution, the second 3*3 depth convolution, and the output of the first 1*1 point-by-point convolution is concat-merged and connected, and after the second 1*1 point-by-point convolution, it is concat-merged and connected with the output of the first 3*3 depth convolution and then M axpool maximum pooling; the multi-scale global feature fusion structure includes a 5*5 depth convolution, a third 3*3 depth convolution, and a third 1*1 point-by-point convolution, respectively. The output of the third 3*3 depth convolution is concat-merged with the output of the 5*5 depth convolution, and then concat-merged with the output of the third 1*1 point-by-point convolution after the fourth 1*1 point-by-point convolution and then output; the prediction layer includes two detection heads of different scales, one detection head includes 3*3 depth separable convolution and 3*3 convolution in sequence, and the other detection head includes upsampling, 3*3 depth separable convolution and 3*3 convolution in sequence; the output layer includes two output feature maps with height*width*channels of 27*13*13 and 27*26*26 respectively.

2. The ZYNQ neural network acceleration system for aluminum surface micro-defect detection according to claim 1 is characterized in that: The convolution IP core reads three kinds of data, namely, input feature map, weight and bias parameter, and writes output feature map data; the sampling IP core reads input feature map data and writes output feature map data.

3. The ZYNQ neural network acceleration system for aluminum surface micro-defect detection according to claim 1 is characterized in that: The convolution IP core adopts the optimization strategies of loop unrolling, pipelining, data blocking and ping-pong buffering of data flow.

4. The ZYNQ neural network acceleration system for aluminum surface micro-defect detection according to claim 1 is characterized in that: The sampling IP core adopts optimization strategies of loop unrolling, pipelining, data blocking and ping-pong buffering of data stream.

5. The ZYNQ neural network acceleration system for aluminum surface micro-defect detection according to claim 1 is characterized in that: Also includes: An aluminum material defect data set is input to train the LADFastDet network, and the trained LADFastDet network outputs detection results of defect categories.

6. The ZYNQ neural network acceleration system for aluminum surface micro-defect detection according to claim 5 is characterized in that: The defect categories include pinholes, scratches, dirt, and wrinkles.

7. The ZYNQ neural network acceleration system for aluminum surface micro-defect detection according to claim 5 is characterized in that: The input aluminum material defect data set is further subjected to data enhancement processing, wherein the data enhancement processing includes copying and pasting and Gaussian filtering.

8. The ZYNQ neural network acceleration system for aluminum surface micro-defect detection according to claim 5 is characterized in that: The input aluminum material defect data set is further subjected to data augmentation processing, wherein the data augmentation processing includes random flipping, brightness change, rotation, cropping, translation and mirroring.

Images