Smelting zinc film residue detection method based on self-supervised learning

Abstract

The invention relates to the technical field of detection, in particular to a smelting zinc film residue detection method based on self-supervised learning, and by introducing methods of self-supervised learning, feature enhancement and the like, the performance of a zinc film residue target detection algorithm based on deep learning is obviously improved.

Description

[Technical field]

[0001] The present invention belongs to the field of detection technology, and in particular relates to a smelting zinc film residue detection method based on self-supervised learning. [Background technology]

[0002] With the rapid development of artificial intelligence technology, various intelligent devices have gradually been integrated into people's daily lives. In addition to their applications in daily life, intelligent devices can also perform various tasks such as operations, inspections, sorting, and security, thereby liberating humans from dangerous, repetitive, and harsh working environments, and maximizing the safety of production operations. Although industrial robots have developed rapidly and are widely used, their application in the field of non-ferrous metal smelting is still in its early stages. Many repetitive tasks still rely on manual labor, lacking automation and intelligence, which makes it difficult to effectively improve production efficiency and quality. To solve this problem, it is urgent to design and develop industrial robots suitable for specific tasks for the actual tasks of non-ferrous metal smelting. According to data from the National Bureau of Statistics, the output of industrial robots in my country continued to increase from 2015 to 2022. In 2022, the output of industrial robots reached 443,000 sets, an increase of 11% over the same period in 2021. Although the growth rate has slowed down slightly compared with 2016 and 2017, with growth rates of 120% and 81% respectively, it still shows a stable growth trend overall. This shows that my country's industrial robot market still has huge development potential.

[0003] In recent years, deep learning algorithms have been deeply studied in the field of artificial intelligence, and target detection algorithms based on deep learning have gradually become well-known. Deep learning technology not only improves the accuracy of the algorithm through automated feature extraction, but also simplifies the feature design process. Deep learning algorithms represented by convolutional neural networks (CNN), such as R-CNN and YOLO, have broken through the traditional detection process. R-CNN uses CNN to extract features, avoiding the manual design of target pre-selection boxes, greatly improving the accuracy and speed of target detection; YOLO simplifies target detection into an end-to-end regression problem, creating a one-stage detection framework, and significantly speeding up the detection speed.

[0004] However, these deep learning-based target detection algorithms still face some bottlenecks in practical applications. The dependence of supervised learning on labeled data, the imbalance of categories, and the poor generalization ability of new data have become problems that restrict their further development. In the actual factory smelting metal scene, collecting liquid metal data sets with residues and manually annotating the collected data will consume a lot of manpower. In addition, in the actual data collection process, the residue targets on the surface of liquid zinc will be interfered by a lot of background noise, making it more difficult to detect the residue. [Summary of the invention]

[0005] In view of the above, it is necessary to provide a smelting zinc film residue detection method based on self-supervised learning. By introducing self-supervised learning and feature enhancement methods, the performance of the zinc film residue target detection algorithm based on deep learning is significantly improved.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for detecting zinc film residue in smelting based on self-supervised learning, the method comprises the following steps:

[0008] (1) Collect images of the production process, obtain unlabeled zinc film residue image data, and perform data enhancement processing on these image data;

[0009] (2) Using the collected data as the algorithm input, a momentum-based contrastive self-supervised learning algorithm is used to construct positive and negative sample pairs and perform feature extraction. By constructing a dynamic encoder queue, the key visual features of the zinc film residue are effectively obtained;

[0010] (3) Adjust the aspect ratio and size of the AnchorBox;

[0011] (4) The confidence scores of these boxes are dynamically downgraded according to the IoU value. At the same time, the detection frames of zinc film residue features in different parts of the same image are overlapped and dynamically adjusted using Soft-NMS.

[0012] (5) The image processed in step (4) is subjected to feature extraction by ResNet50 to obtain a feature vector;

[0013] (6) The feature vector of step (5) is processed by the channel attention module to generate a feature map M with enhanced channel attention. c (F), and multiply the initial intermediate feature map F by elements to generate a channel-optimized feature map F′; F′ is sent to the spatial attention module, and the spatial attention map M generated by the spatial attention module s (F′) is fused with F′ using the same element-wise multiplication method to further refine the feature representation and generate the final refined feature output F″;

[0014] (7) Perform prediction processing on the feature output generated by each layer of ResBlock to obtain the final calibration prediction result;

[0015] (8) Use PyQt5 to link and improve the model algorithm and develop a human-computer interaction interface for the zinc film residue detection system.

[0016] In the present invention, further, the step (3) specifically sets the aspect ratio of the AnchorBox to 1:3, 1:1, and 3:1 respectively, and then adjusts the original size of the AnchorBox from 128, 256, and 512 to 256, 512, and 1024.

[0017] In the present invention, further, the human-computer interaction interface of step (8) integrates multiple functional modules, including: 1) importing a single or multiple images, 2) detection, 3) quantification and 4) result storage.

[0018] The present invention has the following beneficial effects:

[0019] The present invention is based on the Momentum Contrast (MoCo) algorithm to construct positive and negative sample pairs. The MoCo algorithm can learn the similarities and differences between samples, thereby obtaining a more discriminative feature representation. The MoCo algorithm is used for feature extraction. By constructing a dynamic encoder queue, the key visual features of zinc film residues can be effectively obtained, avoiding the high computational cost required to calculate a large number of negative samples in each iteration.

[0020] In terms of target detection algorithms, the present invention proposes three major improvement strategies for the Mask R-CNN model. In view of the large difference in aspect ratios of target objects, the Anchor Box generation mechanism of the RPN network is optimized to better adapt to targets with extreme aspect ratios in the image. In view of the problem of overlapping feature extraction frames of zinc film residues, the Soft-NMS algorithm is used instead of the original NMS algorithm to reduce the phenomenon of missed detection caused by overlapping feature extraction frames. In view of the problem of uneven illumination distribution, the CBAM attention mechanism is integrated into the Backbone network of the MaskR-CNN model to enhance the focus and accuracy of feature extraction. Through these improvement measures, the average accuracy of zinc film residue detection is increased to 93.64%, an increase of 6.72% compared to the basic model.

[0021] In addition, the present invention also designs an interactive graphical user interface for displaying zinc film residue detection results through Python's PyQt library. This design not only improves the usability and human-computer interaction efficiency of the zinc film residue detection system based on feature self-supervised learning, but also allows users to easily select single or batch image files as input to facilitate the detection of zinc film residues. Users can easily upload images and view zinc film residue detection results, realizing intelligent zinc film residue detection.

Brief Description of the Drawings

[0022] Figure 1 This is a schematic diagram of the improvements to the Anchor Box;

[0023] Figure 2Schematic diagram of the convolutional attention module (CBAM) structure;

[0024] Figure 3 Schematic diagram of the fusion of CBAM and ResBlock in ResNet;

[0025] Figure 4 Loss curves before and after improvement for MaskR-CNN;

[0026] Figure 5 AP curves before and after MaskR-CNN improvement;

[0027] Figure 6 Improved the object detection performance of the Mask R-CNN model;

[0028] Figure 7 The improved target detection effect of MaskR-CNN model;

[0029] Figure 8 The design flow chart of the entire self-supervised learning zinc film residue target detection system;

[0030] Fig. 9 Design of software functional modules;

[0031] Fig.10 Draw a window interface diagram for Qt Designer;

[0032] Fig.11 It is the user login interface of the zinc film residue target detection system developed using self-supervised learning technology;

[0033] Fig.12 It is the interactive main interface of the zinc film residue detection system developed based on PyQt5;

[0034] Fig.13 Schematic diagram of the successful software packaging process. [Specific implementation method]

[0035] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific implementation disclosed below.

[0036] Example:

[0037] This embodiment provides a method for detecting smelting zinc film residue based on self-supervised learning, which comprises the following steps:

[0038] (1) Collect images of the production process, obtain unlabeled zinc film residue image data, and perform data enhancement processing on these image data;

[0039] (2) Using the collected data as the algorithm input, a momentum-based contrastive self-supervised learning algorithm is used to construct positive and negative sample pairs and perform feature extraction. By constructing a dynamic encoder queue, the key visual features of the zinc film residue are effectively obtained;

[0040] (3) In order to detect and extract the target area more efficiently and accurately, this embodiment adjusts the aspect ratio and size of the AnchorBox (anchor box); specifically, the aspect ratio of the AnchorBox is set to 1:3, 1:1, and 3:1, and then the original size of the AnchorBox of 128, 256, and 512 is adjusted to 256, 512, and 1024; the improved schematic diagram of the AnchorBox is shown in FIG. Figure 1 As shown;

[0041] (4) The confidence scores of these boxes are dynamically downgraded according to the IoU value. At the same time, the detection frames of zinc film residue features in different parts of the same image are overlapped and dynamically adjusted using Soft-NMS.

[0042] NMS aims to select the best detection bounding box from many overlapping candidate boxes. Due to the characteristics of NMS, anchor boxes are missed due to overlap. During the detection process, each detection box is equipped with a confidence score that reflects the possibility of containing the target. Detection boxes with high confidence are more likely to correctly indicate the exact location of the target. The NMS algorithm first sorts all detection boxes in descending order according to confidence. Then, the algorithm takes the detection box with the highest confidence score, adds it to the final detection result, and removes it from the candidate list. Then, NMS calculates the IoU value of this highest-scoring bounding box with all other bounding boxes in the list. If the IoU of other bounding boxes with the highest-scoring bounding box exceeds the preset threshold, it indicates that the bounding box overlaps too much with the highest-scoring bounding box and points to different parts of the same target, so it needs to be removed from the candidate list. In this way, NMS effectively suppresses non-maximum detection boxes, leaving only the detection box with the largest local response.

[0043] The NMS algorithm is a discontinuous binary weight function, which is as follows:

[0044]

[0045] In the formula, s i is the confidence score corresponding to the i-th prediction box, M is the candidate box with the highest score, and b i is the frame to be detected, N tThe hyperparameter is the overlap threshold, and IoU is b i The ratio of the intersection and union of M.

[0046] From the above analysis, it is obvious that the NMS algorithm is very sensitive to the set overlap threshold. If the threshold is set too low, some important detections will be missed, resulting in missed detections; on the contrary, if the threshold is set too high, the detection system tends to retain redundant bounding boxes, resulting in false detections. In order to effectively balance the problems of missed detection and false detection, this section uses the Soft-NMS algorithm to replace the original NMS algorithm.

[70] Soft-NMS draws on the shortcomings of NMS in dealing with overlapping detection boxes. Compared with the traditional NMS, which directly deletes detection boxes with an IoU greater than a certain threshold, Soft-NMS adopts a more moderate strategy, appropriately reducing the confidence scores of these detection boxes instead of completely eliminating them. This method reduces missed detection during target detection and retains those detection boxes that may contain valid information.

[0047] The linear weighting function used in the Soft-NMS algorithm is as follows:

[0048]

[0049] Soft-NMS no longer simply deletes candidate boxes whose IoU exceeds a certain threshold, but dynamically downgrades the confidence scores of these boxes according to the IoU value. This method allows candidate boxes with lower confidence to have the opportunity to be identified as valid targets without interfering with the main candidate box, significantly reducing the missed detection rate. When generating detection frames, the residual features of the zinc film in different parts of the same image overlap with the detection frames, resulting in some target images being unable to be detected. The dynamic adjustment of Soft-NMS provides a more flexible recognition process, ensuring that these targets can be captured to the greatest extent even in complex backgrounds.

[0050] (5) The image processed in step (4) is subjected to feature extraction by ResNet50 to obtain a feature vector;

[0051] (6) Figure 2 As shown in the figure, the CBAM module includes two submodules: channel attention and spatial attention, which are responsible for identifying important features in the image and the specific locations of these features respectively. The whole process first processes the feature vector of step (5) through the channel attention module to generate a feature map M with channel attention enhancement. c (F), and multiply the initial intermediate feature map F by elements to generate a channel-optimized feature map F′; F′ is sent to the spatial attention module, and the spatial attention map M generated by the spatial attention module s(F′) is fused with F′ using the same element-wise multiplication method to further refine the feature representation and generate the final refined feature output F″;

[0052] The whole process above can be summarized as follows:

[0053]

[0054]

[0055] (7) Perform prediction processing on the feature output generated by each layer of ResBlock to obtain the final calibration prediction result;

[0056] The feature extraction network of Mask R-CNN is composed of ResNet and FPN, and the CBAM module is integrated into the ResNet50 network. Figure 3 The location where the CBAM module is integrated into the re-control block is shown in the figure; the improved model is denoted as Improved Mask R-CNN;

[0057] (8) The improved model algorithm was linked with PyQt5 to develop a human-computer interaction interface for the zinc film residue detection system; the human-computer interaction interface integrates multiple functional modules, including: 1) importing single or multiple images, 2) detection, 3) quantification, and 4) result saving.

[0058] In order to fully demonstrate the effectiveness and practicality of the improved model (improved Mask R-CNN), after the pre-training model was obtained by momentum contrastive learning on 30,000 samples, the Backbone network of the improved object detection model was initialized and trained on a dataset of 300 zinc film images.

[0059] The experiment was conducted under the Ubuntu 18.04 operating system. The CPU used in the experiment was Intel Xeon Silver, the GPU was NVIDIA GeForce RTX 3090, the programming language was Python 3.9, and the deep learning framework was Pytorch 1.9.1. The specific settings are shown in the following table.

[0060] Table 1 Experimental configuration

[0061]

[0062] like Figure 4 The results show the loss function curve used by the improved algorithm and compare it with the original Mask R-CNN model. At the same Epoch, the Loss value of the improved model is significantly smaller than that of the original model, indicating that the closer the model's prediction results are to the actual situation, the better the model's performance.

[0063] In order to deeply understand and verify the specific impact of each improvement on the model performance, a series of ablation experiments were continued to be performed, and each improvement was trained separately. The standard detection model is the result of training on a dataset of 300 zinc film residue images after self-supervised learning with 30,000 zinc film residue samples in the previous chapter;

[0064] The results are shown in Table 2 and Figure 5 As shown;

[0065] Table 2 Comparison of ablation experiment training results

[0066]

[0067]

[0068] Depend on Figure 5 As shown in Table 2, the improved MaskR-CNN model achieves an accuracy of 93.64% for zinc film residues by incorporating the CBAM attention mechanism module. This ablation experiment verifies the effectiveness of the model proposed in this chapter.

[0069] Figure 6 and Figure 7 The visualization results of the detection of zinc film residue targets before and after the improvement of the MaskR-CNN model are shown respectively. It can be clearly seen from the figure that the improved model has significantly improved the detection accuracy and missed detection. The improved model is more precise and accurate in detecting zinc film residue targets, and the number of missed detections is greatly reduced, indicating that the improvements in this chapter have effectively improved the performance of the model to a certain extent, making it more suitable for the detection task of zinc film residue targets. From these visualization results, it can be concluded that the improved model can indeed achieve positive results.

[0070] Next, a self-supervised learning zinc film residue detection system is developed according to step (8).

[0071] The software tool is designed and developed based on the Visual Studio Code integrated development environment. Throughout the design and development phase, this chapter adopts a strategy of separating interface design from background logic programming to build the software.

[0072] The first step is to configure the environment. This step mainly includes installing corresponding development tools and plug-ins such as PyQt5, Qt Designer, and Pyuic plug-ins. The installation and configuration of these tools lays the foundation for subsequent interface design and development work.

[0073] The second step is interface design. When designing the interface, you need to reasonably choose the layout of the main window and build the necessary control elements on this basis. Then, arrange these controls reasonably to achieve a clear and user-friendly interface design. Connecting the control's signal with the corresponding slot is another key step, which facilitates the writing of logic code and ensures smooth interaction and response between the interface and program logic.

[0074] The third step is to convert the designed interface file into a Python code file by using the Pyuic plug-in. According to the specific properties and naming conventions of the interface controls, as well as the user's operating habits and experience, start writing logic code to achieve complete software functions and dynamic response of the interface.

[0075] The fourth step is to use the PyInstaller tool to package the previous program files, carefully designed interface files, and logic codes to generate an executable application.

[0076] Figure 8 It is the design flow chart of the entire self-supervised learning zinc film residue target detection system.

[0077] An image processing application with a multi-module architecture was designed, which includes a series of key functional modules such as reading images, zinc film residue detection and quantitative analysis, displaying results and saving files. Fig. 9 The first is the login module of the application, where users need to enter the preset account and password to enter the main page of the application. On the main page, the layout and functions of the main modules are pre-designed based on the algorithm model. The user then selects the target type to be detected through the input module and then enters the output module. In the output module, the application displays the detection, segmentation and quantitative analysis results of zinc film residues and saves these results.

[0078] In PyQt5, Qt Designer provides a more intuitive and efficient way to layout graphical user interfaces. Designers can select from a variety of basic window controls using only the mouse, and can easily place the selected components in the appropriate position in the main window of QMainWindow by dragging and dropping.

[0079] The QLabel control is used to display "input image", "prediction result" and "quantification result". Then, the QPushButton control is used to set the function keys such as "import image", "import folder", "detection", "quantification" and "save". Finally, the QTextBrowser control is used to display the program's running information and the specific data of the detection results in real time.

[0080] like Fig.10As shown in the figure, the left area of ​​the interface design is used to place controls, while the right area displays the properties of these controls, allowing for quick modification. This chapter defines several key slot functions to implement the signal and slot mechanism to achieve communication between objects in order to respond to corresponding user operation events. The completed interface design is saved, and the interface file is converted into a Python code file using the Pyuic tool. The resulting Python code file is the final interface layout template.

[0081] Fig.11 The figure shows the user login interface of the zinc film residue target detection system developed using self-supervised learning technology. The interface is designed to be simple and clear, and users can enter the account and password they created during the registration process. After completing this step, the user clicks the login button and the system will verify the entered credential information. Once the verification is passed, the user can smoothly jump to the main window interface of the application, which is the interface provided by QMainWindow.

[0082] Fig.12 It is the interactive main interface of the zinc film residue detection system developed based on PyQt5. The logic coding ideas are summarized as follows:

[0083] (1) Users can browse the local file system and select the images they wish to analyze by clicking the "Import Images" button on the main interface of the application. This function not only allows users to detect a single image, but also allows users to batch select multiple images for analysis by selecting the "Import Folder" button, which greatly improves processing efficiency and is especially suitable for scenarios where large amounts of image data need to be processed in batches. Once selected, these images will be immediately displayed in the first QLabel control on the main interface of the application.

[0084] (2) After the user clicks the "Detect" button, the program will trigger the built-in zinc film residue detection model and immediately start in-depth analysis of the zinc film residue image selected by the user, using advanced image processing technology to accurately identify and analyze the zinc film residue. After the analysis is completed, the detection results will be displayed in the second QLabel control of the interface.

[0085] (3) After the user clicks the "Quantify" button, the system will immediately use the data obtained from the previous detection stage to accurately extract the segmentation mask of the zinc film residue. After the quantitative analysis is completed, the results will be intuitively displayed on the third QLabel control of the application interface, providing a clear and visible quantitative result.

[0086] (4) By clicking the “Save” button, the user can save the detection and quantification results to a specified path.

[0087] When the system performs the detection process, all relevant operation information and detection data will be displayed in real time in the QTextBrowser control at the bottom of the interface, providing users with instant feedback and a detailed overview of the detection results. This interactive method improves the user's operational convenience and makes the zinc film residue detection and analysis process more intuitive and efficient.

[0088] In order to avoid complicated installation steps for users, this application also provides a complete independent operating environment. Users can run applications directly in this environment without configuring a complicated development environment or installing additional packages. This method simplifies the use process while enhancing the convenience and efficiency of the application.

[0089] As a program packaging tool, Pyinstaller can convert Python-based programs into independent executable files and supports multiple platforms such as Windows, Linux, and Mac OS. Pyinstaller first analyzes the Python code to identify the modules and libraries required for operation. Next, these dependent files and the Python interpreter itself are collected and packaged together with the source code. The packaged program can be stored as a collection in the same directory or packaged into a separate executable file. In this way, the Python code file is converted into a direct executable file that does not require source code, and finally an "Zinc Film Residue Detection System" EXE file suitable for the Windows operating system is generated. The packaging process is as follows: Fig.13 shown.

[0090] In summary, by introducing methods such as self-supervised learning and feature enhancement, the performance of zinc film residue target detection algorithm based on deep learning has been significantly improved.

[0091] The above-mentioned embodiments merely express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it should not be understood as limiting the scope of the present invention.

Claims

1. A method for detecting zinc film residues from smelting based on self-supervised learning, characterized in that: The method comprises the following steps: (1) Collect images of the production process, obtain unlabeled zinc film residue image data, and perform data enhancement processing on these image data; (2) Using the collected data as the algorithm input, a momentum-based contrastive self-supervised learning algorithm is used to construct positive and negative sample pairs and perform feature extraction. By constructing a dynamic encoder queue, the key visual features of the zinc film residue are effectively obtained; (3) Adjust the aspect ratio and size of the AnchorBox; (4) The confidence scores of these boxes are dynamically downgraded according to the IoU value. At the same time, the detection frames of zinc film residue features in different parts of the same image are overlapped and dynamically adjusted using Soft-NMS. (5) The image processed in step (4) is subjected to feature extraction by ResNet50 to obtain a feature vector; (6) The feature vector of step (5) is processed by the channel attention module to generate a feature map M with enhanced channel attention. c (F), and multiply the initial intermediate feature map F by elements to generate a channel-optimized feature map F′; F′ is sent to the spatial attention module, and the spatial attention map M generated by the spatial attention module s (F′) is fused with F′ using the same element-wise multiplication method to further refine the feature representation and generate the final refined feature output F″; (7) Perform prediction processing on the feature output generated by each layer of ResBlock to obtain the final calibration prediction result; (8) Use PyQt5 to link and improve the model algorithm and develop a human-computer interaction interface for the zinc film residue detection system.

2. The method according to claim 1, characterized in that The step (3) specifically sets the aspect ratio of the Anchor Box to 1:3, 1:1, and 3:1, respectively, and then adjusts the original size of the AnchorBox from 128, 256, and 512 to 256, 512, and 1024.

3. The method according to claim 1, characterized in that The human-computer interaction interface of step (8) integrates multiple functional modules, including: 1) importing a single or multiple images, 2) detection, 3) quantification and 4) result saving.

Images