A multi-target automatic temperature measurement and early warning method for components of steel cold rolling annealing furnace

Abstract

The present invention discloses a multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace, which relates to the field of metal smelting technology, and includes the following steps: collecting image data of key components of the annealing furnace, developing an algorithm for target detection of components of the annealing furnace, developing an algorithm for semantic segmentation of images of components of the annealing furnace, analyzing the historical temperatures of each component, and designing an algorithm for detecting abnormal temperatures. An automatic temperature monitoring system based on a combination of infrared thermal imaging cameras and artificial intelligence technology is used, and target detection and image semantic segmentation algorithms based on machine vision technology are used to obtain real-time images of equipment components. On the basis of obtaining on-site measured data, real-time temperature data of key components are obtained based on infrared thermal imaging cameras, and an algorithm for detecting abnormal temperatures is developed to judge the operating status of the equipment and to promptly warn of equipment abnormalities.

Description

Technical Field

[0001] The invention relates to the technical field of metal smelting, and in particular to a multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace. Background Art

[0002] Since there are no open-source images of key components of annealing furnaces in steel companies on the Internet, it is necessary to obtain images through manual photography as the basic data set for model training, and the data set needs to be manually annotated, and then the data set needs to be expanded through data enhancement methods. Because the generalization and accuracy of the model need to be enhanced, it is necessary to collect images of components at multiple angles and different lighting intensities; the component image data set obtained through the previous data collection work is relatively less obvious in key target features than some open-source machine vision training data sets on the Internet. In the process of using self-constructed training data sets that conform to real scenes to complete the training of target detection algorithms and image semantic segmentation algorithms, how to choose a basic model or network that has advantages in accuracy, speed, etc., to build a robust target detection and image semantic segmentation model;

[0003] Combined with the actual working conditions on site, it is necessary to use the image information of the key components of the annealing furnace collected by the infrared thermal imaging camera, and based on multi-source image analysis technologies such as real-time target detection and image semantic segmentation, calculate the real-time temperature of each component and realize the temperature anomaly detection function. How to combine the target and component contour results output by target detection and image semantic segmentation with the image collected by the infrared thermal imaging camera to obtain the pixel area of ​​each component and calculate the temperature of the corresponding component; in view of this, we proposed a multi-target automatic temperature measurement and early warning method for steel cold rolling annealing furnace components. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace, which solves the problems mentioned in the above background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace, the automatic temperature measurement and early warning method comprising the following steps:

[0006] S1: Collect image data of key components of the annealing furnace, and obtain more effective feature information by shooting the components to be detected from multiple angles; Based on the labelme open source annotation tool, determine the sample annotation method suitable for the present invention, formulate the criteria for the annotation of components such as motors, reducers, and bearings, and perform refined annotation of the features required for data set target recognition and semantic segmentation tasks;

[0007] S2: Annealing furnace component target detection algorithm development process, annealing furnace component target detection algorithm is developed based on the YOLOv3 model;

[0008] S3: The process of developing the semantic segmentation algorithm for annealing furnace component images, using the FCN model to classify images at the pixel level, and then perform semantic level image segmentation on component images;

[0009] S4: Deploy a slide rail in the channel on one side of the furnace roller, and move the infrared thermal imaging camera on the slide rail at a constant speed to capture the infrared thermal image of each furnace roller. According to the temperature of each pixel provided by the infrared thermal image, combined with the pixel area of ​​each component obtained by the target detection and image segmentation algorithm, the real-time temperature of each component is calculated by the average weighted algorithm;

[0010] S5: Design a temperature anomaly detection algorithm by analyzing the historical temperature of each component; analyze and count the historical temperature of the component, calculate the highest, lowest and average temperature in the past period of time, and then calculate the normal temperature range of the component at the current moment, and then determine whether the current temperature of the component is within the normal temperature range. If not, the current temperature of the component is determined to be abnormal and an alarm is issued.

[0011] Preferably, the data collection and preprocessing process in step S1 further includes:

[0012] S11. Manually take photos of annealing furnace components at multiple angles and in different light and dark conditions, covering all key inspection components, and simulating the shooting angles of various infrared cameras after installation;

[0013] S12. Through business research and analysis, formulate annotation criteria for key features such as key targets in target detection and component coverage areas in image semantic segmentation;

[0014] S13. Based on the business research analysis and the target characteristics of the annealing furnace components, the component image dataset is preprocessed using data augmentation techniques such as flipping, random pruning, translation transformation, scale change, and noise disturbance.

[0015] Preferably, the annealing furnace component target detection algorithm development process in step S2 further includes:

[0016] S21. According to the characteristics of the number of targets, target size and positioning requirements required for target identification of annealing furnace components, and the requirements for model reasoning speed and result accuracy in actual business, the target detection model or network is selected.

[0017] Preferably, step S2 further includes:

[0018] The S22 and YOLOv3 models use only convolutional layers and adopt the Darknet-53 feature extraction network, which contains 53 convolutional layers, each followed by a batch normalization layer and a leaky ReLU layer;

[0019] The S23 and YOLOv3 algorithms also added residual modules to the Darknet-53 network.

[0020] Preferably, the process of developing the semantic segmentation algorithm of the annealing furnace component image in step S3 further includes:

[0021] S31. According to the requirements of the multi-source image analysis task for annealing furnace components, and considering the requirements for model inference speed and result accuracy in actual business, the image semantic segmentation model is selected, and the test results are compared. The FCN model is selected as the basic model for the annealing furnace component image semantic segmentation task.

[0022] Preferably, step S3 further includes:

[0023] S32. In the FCN model, the input image of the model passes through multiple convolutional layers and a maximum pooling layer, and becomes feature map pool1, and the width and height become 1 / 2; feature map pool1 passes through multiple convolutional layers and a maximum pooling layer feature map pool2, and the width and height become 1 / 4; the feature map pool5 is generated, and its width and height become 1 / 32. The 6th and 7th layers of the network are respectively a one-dimensional vector of length 4096, and the 8th layer is a one-dimensional vector of length 1000, corresponding to the probabilities of 1000 different categories;

[0024] S33. After the heat map is generated, it needs to be upsampled to enlarge the image multiple times until it is restored to the size of the original image.

[0025] Preferably, the process of real-time temperature detection of components in step S4 further includes:

[0026] S41. Integrate the target detection scheme and the image semantic segmentation algorithm to realize the multi-image analysis scheme of annealing furnace components, realize the detection and positioning of annealing furnace components by the target detection algorithm, and confirm the accurate outline of each component by the image semantic segmentation algorithm;

[0027] S42, performing image matrix alignment between the instance segmentation image generated by the target detection and instance segmentation algorithm in the multivariate image analysis and the image transmitted back by the infrared thermal imaging camera.

[0028] Preferably, step S4 further includes:

[0029] The S43 infrared thermal imaging camera takes infrared thermal images. The camera's internal algorithm calculates and processes the image and outputs the temperature T_p of each pixel, where p represents the pixel. The target detection and image segmentation algorithm performs target detection and image segmentation on the image to obtain the pixel area obj of each component. i , represents the pixel area of ​​the i-th component, and then the real-time temperature of each component is calculated through the average weighted algorithm. The calculation formula is as follows:

[0030]

[0031] Where: m i Represents the number of pixels in the i-th component area.

[0032] Preferably, the process of determining the real-time temperature status of components in step S5 further includes:

[0033] S51, calculate and record the real-time temperature of each component in step S4, and statistically analyze the temperature C of each component at the past time t it , calculate the highest, lowest and average temperatures in the past period of time, recorded as Max i 、Min i and Aver i , thereby calculating the normal temperature range of the component at the current moment.

[0034] S52. By comparing the temperature of each component in real time with the calculated normal temperature range, it is determined whether the current temperature of the component is normal, and an alarm needs to be issued for abnormal conditions.

[0035] The present invention provides a multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace. It has the following beneficial effects:

[0036] The multi-target automatic temperature measurement and early warning method for steel cold rolling annealing furnace components is based on an automatic temperature monitoring system that combines infrared thermal imaging cameras and artificial intelligence technology, and uses target detection and image semantic segmentation algorithms based on machine vision technology to obtain real-time equipment component images. On the basis of obtaining on-site measured data, the real-time temperature data of key components is obtained based on infrared thermal imaging cameras, and a temperature anomaly detection algorithm is developed to determine the operating status of the equipment and promptly warn of equipment anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The network structure diagram of the YOLOv3 model for target detection of the present invention;

[0038] Figure 2 This is the network structure diagram of the FCN model for image semantic segmentation of the present invention;

[0039] Figure 3 This is the image analysis effect of the present invention and the actual picture taken by the infrared thermal imaging camera;

[0040] Figure 4 It is a schematic diagram of the component position and temperature inference process of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] See also Figure 1-Figure 4 The present invention provides a technical solution: a multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace, the automatic temperature measurement and early warning method comprising the following steps:

[0043] S1: Collect image data of key components of the annealing furnace, and obtain more effective feature information by photographing the components to be detected from multiple angles; determine the sample labeling method suitable for the present invention based on the labelme open source labeling tool, formulate the criteria for labeling components of motors, reducers, and bearings, and perform refined labeling of features required for data set target recognition and semantic segmentation tasks; preprocess the data set using data augmentation techniques such as flipping, random pruning, translation transformation, scale change, and noise disturbance to improve the generalization ability of the model.

[0044] Specifically, step S1 includes:

[0045] S11. Manually take photos of annealing furnace components at multiple angles and in different light and dark conditions, covering all key inspection components, and simulating the shooting angles of various infrared cameras after installation;

[0046] S12. Through business research and analysis, formulate the labeling criteria for key features such as key targets in target detection and component coverage areas in image semantic segmentation; use labelme software to label more than a thousand images of annealing furnace components;

[0047] S13. Based on the business research analysis and the target characteristics of annealing furnace components, the component image dataset is preprocessed using data augmentation techniques such as flipping, random pruning, translation transformation, scale change, and noise disturbance to improve the generalization ability of the model during model training.

[0048] S2: According to the target features of the annealing furnace key component data set, an annealing furnace component target detection algorithm is developed based on the YOLOv3 model; YOLOv3 is a small object target detection algorithm that combines model inference speed and accuracy; the algorithm uses a single neural network to act on the image, divide the image into multiple regions and predict the bounding box and the probability of each region.

[0049] Specifically, step S2 includes:

[0050] S21. According to the characteristics of the number of targets, target size and positioning requirements required for target identification of annealing furnace components, and the requirements for model reasoning speed and result accuracy in actual business, the target detection model or network is selected; by comparing the test results, the YOLOv3 model is finally selected as the basic model, which further improves the recognition accuracy of small targets while maintaining the speed advantage of the YOLO family algorithm; YOLO creatively proposed one-stage, that is, object classification and object positioning are completed in one step, and the position of the border and the category of the border are directly regressed in the output layer, thereby realizing one-stage; in this way, a computing speed of 45 frames / second can be achieved, which can meet the real-time requirements;

[0051] The S22 and YOLOv3 models only use convolutional layers and adopt the Darknet-53 feature extraction network, which contains 53 convolutional layers, each followed by a batch normalization layer and a leaky ReLU layer. There is no pooling layer, and a convolutional layer with a stride of 2 is used instead of the pooling layer to downsample the feature map, which can effectively prevent the loss of low-level features caused by the pooling layer.

[0052] S23 and YOLOv3 algorithms also add residual modules to the Darknet-53 network, which is helpful to solve the gradient problem of deep networks. Each residual module consists of two convolutional layers and a direct link. If a 416*416*3 image is input, three prediction results of different scales will be obtained through the Darknet-53 network. Each scale corresponds to N channels, which contain prediction information. YOLOv3 has a total of 13*13*3+26*26*3+52*52*3 predictions; each prediction corresponds to 85 dimensions, namely 4 (coordinate value), 1 (confidence score), and 80 (number of coco categories); the network structure of the YOLOv3 model is as follows Figure 1 shown.

[0053] S3: According to the target features of the annealing furnace key component data set, the FCN model is used to classify the image at the pixel level, and then the component image is segmented at the semantic level. The FCN model uses a deconvolution layer to upsample the feature map of the last convolution layer to restore it to the same size as the input image, so that a prediction can be generated for each pixel while retaining the spatial information in the original input image. Then, through the upsampling process, the upsampled feature map is classified pixel by pixel. Finally, the softmax classification loss is calculated pixel by pixel. This process is equivalent to one training sample for each pixel, and finally the instance segmentation of each pixel is achieved.

[0054] Specifically, step S3 includes:

[0055] S31. According to the requirements of the multi-source image analysis task for annealing furnace components, and considering the requirements for model reasoning speed and result accuracy in actual business, the image semantic segmentation model is selected. By comparing the test results, the FCN model is selected as the basic model for the image semantic segmentation task of annealing furnace components. For general classification CNN networks, some fully connected layers are usually added at the end of the network, and the category probability information can be obtained after softmax; the FCN model replaces the backward fully connected layers of the network with convolutional layers, so that a two-dimensional feature map can be obtained, and finally the classification information of each pixel is obtained through the softmax layer, thereby obtaining two-dimensional probability information, thereby solving the semantic segmentation problem;

[0056] S32. In the FCN model, the input image of the model passes through multiple convolutional layers and a maximum pooling layer to become feature map pool1, and the width and height become 1 / 2; feature map pool1 then passes through multiple convolutional layers and a maximum pooling layer feature map pool2, and the width and height become 1 / 4; using the same processing process, the feature map pool5 is finally generated, and its width and height become 1 / 32. The 6th and 7th layers of the network are a one-dimensional vector of length 4096, and the 8th layer is a one-dimensional vector of length 1000, corresponding to the probabilities of 1000 different categories. FCN represents these 3 layers as convolutional layers, and the sizes of the convolution kernels (number of channels, width, height) are (4096, 1, 1), (4096, 1, 1), (1000, 1, 1), and the weights and biases that have been trained by CNN before are used, but the difference is that the weights and biases have their own ranges and belong to their own convolution kernels. Therefore, all layers in the FCN network are convolutional layers, so it is called a fully convolutional network. After multiple convolutions and pooling, the resulting image becomes smaller and smaller, and the resolution becomes lower and lower. The high-dimensional feature map generated after the last layer is called a heat map. The network structure of the image semantic segmentation FCN model is as follows Figure 2 As shown;

[0057] S33, after the heat map is generated, it needs to be upsampled, that is, the image is enlarged multiple times until it is restored to the size of the original image. The final output is an image of the original image size generated by the upsampled heat map, in which each pixel has been classified and predicted, and pixel-level semantic segmentation is achieved through this process.

[0058] S4: A slide rail is deployed in the channel on one side of the furnace roller, and the infrared thermal imaging camera is moved at a constant speed on the slide rail to capture the infrared thermal image of each furnace roller. According to the temperature of each pixel provided by the infrared thermal image, combined with the pixel area of ​​each component obtained by the target detection and image segmentation algorithm, the real-time temperature of each component is calculated by the average weighted algorithm.

[0059] Specifically, step S4 includes:

[0060] S41. Integrate the target detection scheme and the image semantic segmentation algorithm to implement the multi-image analysis scheme for annealing furnace components. Use the target detection algorithm to detect and locate the annealing furnace components, and use the image semantic segmentation algorithm to confirm the accurate outline of each component. Input the image transmitted by the infrared camera into the image semantic segmentation network to obtain the semantic segmentation graph of the input image; then combine the semantic segmentation graph with the positioning obtained by the component target detection, select the intersection of the target detection position and the semantic segmentation area after scaling to optimize the selected range of the component recognition result, and obtain the instance segmentation image of a single component;

[0061] S42, align the instance segmentation image generated by the target detection and instance segmentation algorithm in the multivariate image analysis with the image sent back by the infrared thermal imaging camera. Figure 3 As shown in the figure, the position and outline of each component in the infrared thermal imaging camera image are accurately identified, and the pixel area where the outline is located is then calculated using an average weighted algorithm to calculate the real-time temperature of each component. The process of inferring the position and temperature of components is shown in the figure. Figure 4 As shown;

[0062] S43, infrared thermal imaging camera takes infrared thermal images, the camera internal algorithm will calculate and process the image, and output the temperature T_p of each pixel, where p represents the pixel. The target detection and image segmentation algorithm will perform target detection and image segmentation on the image to obtain the pixel area obj of each component. i , represents the pixel area of ​​the i-th component, and then the real-time temperature of each component is calculated through the average weighted algorithm. The calculation formula is as follows:

[0063]

[0064] Where: m i Represents the number of pixels in the i-th component area.

[0065] S5: Design a temperature anomaly detection algorithm by analyzing the historical temperature of each component. Analyze and count the historical temperature of the component, calculate the highest, lowest and average temperature in the past period of time, and then calculate the normal temperature range of the component at the current moment, and then determine whether the current temperature of the component is within the normal temperature range. If not, the current temperature of the component is determined to be abnormal and an alarm is issued.

[0066] Specifically, step S5 includes:

[0067] S51, calculate and record the real-time temperature of each component in step S4. Statistically analyze the temperature C of each component at the past time t it , calculate the highest, lowest and average temperatures in the past period of time, recorded as Max i 、Min i and Aver i , thereby calculating the normal temperature range of the component at the current moment;

[0068] S52: By comparing the temperature of each component in real time with the calculated normal temperature range, it is determined whether the current temperature of the component is normal. An alarm needs to be issued for abnormal conditions.

[0069] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace, characterized in that: The automatic temperature measurement and early warning method comprises the following steps: S1: Collect image data of key components of the annealing furnace, and obtain more effective feature information by shooting the components to be detected from multiple angles; Based on the labelme open source annotation tool, determine the sample annotation method suitable for the present invention, formulate the criteria for the annotation of components such as motors, reducers, and bearings, and perform refined annotation of the features required for data set target recognition and semantic segmentation tasks; S2: Annealing furnace component target detection algorithm development process, annealing furnace component target detection algorithm is developed based on the YOLOv3 model; S3: The process of developing the semantic segmentation algorithm for annealing furnace component images, using the FCN model to classify images at the pixel level, and then perform semantic level image segmentation on component images; S4: A slide rail is deployed in the channel on one side of the furnace roller, and an infrared thermal imaging camera is moved at a uniform speed on the slide rail to obtain an infrared thermal image of each furnace roller. According to the temperature of each pixel provided by the infrared thermal image, combined with the pixel area of ​​each component obtained by the target detection and image segmentation algorithm, the real-time temperature of each component is calculated by an average weighted algorithm; S5: Design a temperature anomaly detection algorithm by analyzing the historical temperature of each component; analyze and count the historical temperature of the component, calculate the highest, lowest and average temperature in the past period of time, and thus calculate the normal temperature range of the component at the current moment, and then determine whether the current temperature of the component is within the normal temperature range. If not, the current temperature of the component is determined to be abnormal and an alarm is issued.

2. The multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace according to claim 1 is characterized by: The process of data collection and preprocessing in step S1 also includes: S11. Manually take photos of annealing furnace components at multiple angles and in different light and dark conditions, covering all key inspection components, and simulating the shooting angles of various infrared cameras after installation; S12. Through business research and analysis, formulate annotation criteria for key features such as key targets in target detection and component coverage areas in image semantic segmentation; S13. Based on the business research analysis and the target characteristics of the annealing furnace components, the component image dataset is preprocessed using data augmentation techniques such as flipping, random pruning, translation transformation, scale change, and noise disturbance.

3. The multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace according to claim 1 is characterized in that: Step S2: The annealing furnace component target detection algorithm development process also includes: S21. According to the characteristics of the number of targets, target size and positioning requirements required for target identification of annealing furnace components, and the requirements for model reasoning speed and result accuracy in actual business, the target detection model or network is selected.

4. A multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace according to claim 3, characterized in that: Step S2 further includes: The S22 and YOLOv3 models use only convolutional layers and adopt the Darknet-53 feature extraction network, which contains 53 convolutional layers, each followed by a batch normalization layer and a leaky ReLU layer; The S23 and YOLOv3 algorithms also added residual modules to the Darknet-53 network.

5. The multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace according to claim 1 is characterized in that: Step S3: The process of developing the semantic segmentation algorithm for the annealing furnace component image also includes: S31. According to the requirements of the multi-source image analysis task for annealing furnace components, and considering the requirements for model inference speed and result accuracy in actual business, the image semantic segmentation model is selected, and the test results are compared. The FCN model is selected as the basic model for the annealing furnace component image semantic segmentation task.

6. A multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace according to claim 5, characterized in that: Step S3 further includes: S32. In the FCN model, the input model image passes through multiple convolutional layers and a maximum pooling layer to become feature map pool1, and the width and height become 1 / 2; feature map pool1 passes through multiple convolutional layers and a maximum pooling layer feature map pool2, and the width and height become 1 / 4; feature map pool5 is generated, and its width and height become 1 / 32, while the 6th and 7th layers of the network are respectively a one-dimensional vector with a length of 4096, and the 8th layer is a one-dimensional vector with a length of 1000, corresponding to the probabilities of 1000 different categories respectively; S33. After the heat map is generated, it needs to be upsampled to enlarge the image multiple times until it is restored to the size of the original image.

7. The multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace according to claim 1 is characterized in that: The process of real-time temperature detection of components in step S4 also includes: S41. Integrate the target detection scheme and the image semantic segmentation algorithm to realize the multi-image analysis scheme of annealing furnace components, realize the detection and positioning of annealing furnace components by the target detection algorithm, and confirm the accurate outline of each component by the image semantic segmentation algorithm; S42, performing image matrix alignment between the instance segmentation image generated by the target detection and instance segmentation algorithm in the multivariate image analysis and the image transmitted back by the infrared thermal imaging camera.

8. The multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace according to claim 7, characterized in that: Step S4 further includes: The S43 infrared thermal imaging camera takes infrared thermal images. The camera's internal algorithm calculates and processes the image and outputs the temperature T_p of each pixel, where p represents the pixel. The target detection and image segmentation algorithm performs target detection and image segmentation on the image to obtain the pixel area obj of each component. i , represents the pixel area of ​​the i-th component, and then the real-time temperature of each component is calculated through the average weighted algorithm. The calculation formula is as follows: Where: m i Represents the number of pixels in the i-th component area.

9. The multi-target automatic temperature measurement and early warning method for components of a steel cold rolling annealing furnace according to claim 1, characterized in that: The process of determining the real-time temperature status of components in step S5 also includes: S51, calculate and record the real-time temperature of each component in step S4, and statistically analyze the temperature C of each component at the past time t it , calculate the highest, lowest and average temperatures in the past period of time, recorded as Max i 、Min i and Aver i , thereby calculating the normal temperature range of the component at the current moment; S52. By comparing the temperature of each component in real time with the calculated normal temperature range, it is determined whether the current temperature of the component is normal, and an alarm needs to be issued for abnormal conditions.

Images