A detection method for measuring temperature of tuyere based on dual-optical image fusion matrix

Abstract

The application discloses a kind of based on double light image fusion matrix's tuyere temperature measurement detection method, image video stream is collected by pan-tilt camera, visible light and thermal infrared image are read, image is preprocessed using Gaussian filter, target object ROI region in image is extracted, according to ROI region feature, it is scaled to the same resolution under bilinear interpolation, feature point detection is carried out using AKAZE algorithm, digital transformation matrix is solved, image information fusion is carried out;YOLOV5 algorithm is used to identify temperature measurement object again, visible light image is projected into infrared image using image fusion matrix again, the outline coordinates of object identified are projected into infrared image, the outline of measured object is extracted from infrared image, and the temperature data of infrared image are combined to realize the temperature measurement of direct blowing pipe, tuyere, gun bag related high temperature area.The application uses computer vision technology, solves the problem that infrared general measurement cannot find high temperature point, improves the efficiency of on-site worker inspection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of blast furnace tuyere temperature measurement, in particular to a detection method for tuyere temperature measurement based on a double-light image fusion matrix. BACKGROUND

[0002] The blast furnace tuyere is on the blast furnace of the ironworks, high-temperature hot air blown in by the blast furnace tuyere and coke at the bottom of the furnace are oxidized to generate CO, and CO is reduced to iron in the form of oxide in the process of rising at high temperature, which is the most important link in steelmaking and ironmaking. The material of the blast furnace tuyere is mainly high-purity red copper, and the material state is forged, copper plate rolled, and cast in different states according to the strength, rigidity, and anti-cracking performance of the tuyere. The copper plate rolled tuyere has low weight and low cost, but if the wall thickness is too thin, the rigidity is insufficient and deformation is easy to occur. Since the blast furnace tuyere is cooled by water to keep the tuyere body operating at a low temperature, the strength and rigidity are maintained, so the purity of copper is crucial. The traditional method for checking the tuyere is to use a handheld temperature gun to measure the temperature of the tuyere on site, and the on-site environment is harmful to the human body. SUMMARY

[0003] The purpose of the present application is to provide a detection method for tuyere temperature measurement based on a double-light image fusion matrix, which can automatically patrol and detect the temperature of the tuyere on site by cooperating with the algorithm of the robot autonomous navigation, and can automatically patrol and detect the high-temperature area of the straight blow pipe, tuyere, gun bag and related positions regularly or as needed, so as to realize no dead angle temperature measurement and eliminate the blind area of manual point temperature measurement, thereby solving the problems in the above background technology.

[0004] To achieve the above purpose, the present application provides the following technical scheme: a detection method for tuyere temperature measurement based on a double-light image fusion matrix, comprising the following steps:

[0005] S1: calling the image video stream collected by the patrol robot cloud platform camera, reading the visible light and thermal infrared images;

[0006] S2: using Gaussian filtering to pre-process the image, improving the image quality, and used for subsequent ROI region extraction;

[0007] S3: extracting the ROI region of the target object in the image, performing bilinear interpolation scaling to the same resolution according to the ROI region characteristics, and used for subsequent feature point detection and image information fusion;

[0008] S4: using the AKAZE algorithm to detect the feature points, calculating the digital transformation matrix, and performing image information fusion;

[0009] S5: using the YOLOV5 algorithm to identify the temperature measurement object, including the straight blow pipe, tuyere, and gun bag;

[0010] S6: Projecting the object contour coordinates recognized by the visible light image into the infrared image by using the image fusion matrix, extracting the contour of the measured object from the infrared image, and combining the temperature data of the infrared image to realize temperature measurement of the high-temperature areas of the straight blow pipe, the tuyere and the gun package.

[0011] Further, the inspection robot in S1 is provided with a self-navigation programming program for pre-setting the inspection task of the robot.

[0012] Further, the specific method in S2 is as follows: traversing the image by using Gaussian filtering, taking a pixel point in the image as the center of the convolution kernel, using the convolution kernel to make a weighted average of the neighborhood pixels around the pixel point, and taking the calculation result as the new pixel value of the current pixel point to realize Gaussian filtering denoising of the image.

[0013] Further, the method for extracting the ROI region features of the image in S3 is as follows:

[0014] S301: performing OTSU binary threshold segmentation and Canny contour extraction on the image, and then screening by using the shape scale features of the aspect ratio to obtain the required target ROI region;

[0015] S302: extracting the contour shape features of the ROI region, fitting the minimum circumscribed rectangle, determining the scaling size according to the size of the circumscribed rectangle, and using the bilinear interpolation method for image scaling.

[0016] Further, the calculation method of the bilinear interpolation in S3 is as follows:

[0017] S301': defining four points Q11(x1, y1), Q12(x1, y2), Q21(x2, y1), and Q22(x2, y2) of the image, and the pixel values are f(Q11), f(Q12), f(Q21), and f(Q22);

[0018] S302': performing linear interpolation in the x direction and then performing linear interpolation in the y direction on the pixel value f(P) of any point P(x, y) between the four points to obtain the following relationship:

[0019]

[0020] The pixel value of the point P can be solved by the relationship.

[0021] Further, in S4, the FED algorithm is used to solve the nonlinear diffusion equation when using the AKAZE algorithm, and the FED algorithm solves the nonlinear diffusion equation: L i+1 = (I + tauA(L i ))L i , i = 0, 1,..., n-1.

[0022] where I is the identity matrix, A(L i ) is the conduction matrix of the image in dimension i, constructed by the scale-influenced gradient histogram after Gaussian filtering, τ is the time step, and t i-1 -t i .

[0023] Compared with the prior art, the present application has the following advantages:

[0024] 1. The blast hole temperature detection method based on the dual-light image fusion matrix of the present application, in cooperation with the autonomous navigation of the robot, can timely detect the abnormality of the straight blowing pipe, the reducing pipe, the upper corrugation and the lower corrugation of the hot blast stove, and realize the dead angle-free detection of the blast hole temperature of the blast furnace.

[0025] 2. The blast hole temperature detection method based on the dual-light image fusion matrix of the present application can greatly reduce the labor intensity of workers, lock the high-temperature position and high-temperature point, improve the work efficiency, reduce the safety risk of construction operation, and improve the economic benefit of enterprises.

[0026] 3. The blast hole temperature detection method based on the dual-light image fusion matrix of the present application uses two different types of light sources or wavelengths to provide different types of information, and this multi-modal information acquisition method can more comprehensively understand the state of the measured object. By combining visual recognition and infrared temperature measurement, high-resolution and high-precision temperature measurement can be performed in a complex environment, and specific high-temperature points can be found, such as pipeline cracks, holes and other small problem points that are not easily found by the human eye.

[0027] 4. The blast hole temperature detection method based on the dual-light image fusion matrix of the present application uses a deep learning model to first identify the measured object, thereby increasing the accuracy of the method and providing more picture information to help more accurate temperature measurement. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is the logic block diagram of the detection method of the present application;

[0029] Figure 2 is the pixel point definition diagram of the bilinear interpolation algorithm of the present application;

[0030] Figure 3 is the framework schematic diagram of the YOLOV5 algorithm of the present application;

[0031] Figure 4 is the case implementation state diagram in the embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] like Figure 1 As shown, this embodiment of the invention provides a method for detecting air vent temperature based on a dual-light image fusion matrix, comprising the following steps:

[0034] S1: Call the image and video streams collected by the inspection robot's pan-tilt camera and read the visible light and thermal infrared images; the inspection robot has a built-in autonomous navigation programming program that can pre-set the robot's inspection tasks, realizing the inspection and temperature monitoring of the blast furnace tuyeres without blind spots.

[0035] S2: Gaussian filtering is used to preprocess the image to improve image quality for subsequent ROI region extraction. Specifically, Gaussian filters are used to eliminate noise in the original RGB and thermal infrared image data. Gaussian filtering is a weighted average filter, and its convolution kernel has a coefficient to achieve averaging. The reciprocal of the sum of all values ​​in the matrix is ​​the coefficient of the convolution kernel. In actual filtering, the image is traversed, and a certain point in the image is used as the center of the convolution kernel. The convolution kernel is used to calculate the weighted average of the neighboring pixels around the pixel, and the result is used as the new pixel value of the current pixel. Finally, Gaussian filtering is used to remove noise from the image, providing high-quality image data for subsequent ROI extraction and feature point detection.

[0036] The Gaussian distribution can be weighted according to the Gaussian function. The one-dimensional and two-dimensional forms of the function when the mean μ = 0 are shown in the formula: where σ is the standard deviation of the normal distribution, and its value determines the rate of decay of the function.

[0037] One-dimensional form when μ = 0:

[0038] Two-dimensional form when μ = 0:

[0039] S3: Extract the ROI region of the target object in the image, and perform bilinear interpolation to scale it to the same resolution based on the ROI region features for subsequent feature point detection and image information fusion;

[0040] The method for extracting ROI region features from an image is as follows:

[0041] S301: OTSU threshold segmentation and Canny contour extraction are performed on the image, and a shape scale feature of an aspect ratio is used for screening to obtain a required target ROI region;

[0042] S302: The contour shape feature of the ROI region is extracted, a minimum circumscribed rectangle is fitted, the size of the circumscribed rectangle is determined to determine the scaling size, and a bilinear interpolation method is used for image scaling.

[0043] The calculation method of the bilinear interpolation is as follows:

[0044] S301': define four points Q11(x1, y1), Q12(x1, y2), Q21(x2, y1), and Q22(x2, y2) of the image, and the pixel values are f(Q11), f(Q12), f(Q21), and f(Q22);

[0045] S302': the pixel value f(P) of any point P(x, y) between the four points is first linearly interpolated in the x direction, and then linearly interpolated in the y direction to obtain the following relationship:

[0046]

[0047] The pixel value of the P point is solved through the relationship to ensure the quality of the scaled image, as shown in the following formula: Figure 2

[0048] S4: AKAZE algorithm is used for feature point detection, a digital transformation matrix is solved, and image information fusion is performed; specifically, the FED (fast display algorithm) algorithm is used to solve the nonlinear diffusion equation, the FED algorithm can converge at any step size to obtain a stable scale space, improve the repeatability and specificity, and improve the calculation speed in the description of the feature direction while improving the calculation accuracy.

[0049] The FED algorithm solves the nonlinear diffusion equation: L i+1 =(I+τA(L i ))L i , i=0, 1,..., n-1

[0050] Where I is the unit matrix, A(L i ) is the conduction matrix of the image in dimension i, which is constructed by the scale-influenced gradient histogram after Gaussian filtering, τ is the time step, and t i-1 -t i is the difference value of evolution time.

[0051] S5: YOLOV5 algorithm is used for temperature measurement object target recognition, including straight blow pipe, air port, and gun bag; as shown in the following formula: Figure 3 ​As shown, YOLOv5 is a single-stage target detection algorithm that adds some new improvement ideas on the basis of YOLOv4, so that its speed and accuracy are greatly improved; the main improvement ideas are as follows:

[0052] Input: In the model training stage, some improvement ideas are proposed, mainly including Mosaic data enhancement, adaptive anchor box calculation, and adaptive picture scaling;

[0053] Benchmark network: some new ideas in other detection algorithms are integrated, mainly including Focus structure and CSP structure;

[0054] Neck network: some layers are often inserted between the BackBone and the last Head output layer of the target detection network, and FPN+PAN structure is added in YOLOv5;

[0055] Head output layer: the anchor box mechanism of the output layer is the same as that of YOLOv4, and the main improvement is the loss function GIOU_Loss during training and the DIOU_nms of the prediction box screening.

[0056] S6: using the image fusion matrix, projecting the object contour coordinates recognized from the visible light image into the infrared image, extracting the measured object contour from the infrared image, and combining the temperature data of the infrared image to realize the temperature measurement function of the high-temperature area of the direct blowing pipe, tuyere and gun bag.

[0057] In order to further better explain the embodiment of the present application, the method of the present application is implemented in the blast furnace area of the general plant, as shown in Figure 4 The high-temperature area of the direct blowing pipe, tuyere, gun bag and related positions can be automatically inspected regularly or as needed, realizing no dead angle temperature measurement and eliminating the blind area of manual point temperature measurement. When an abnormal condition is found, the system immediately sends an alarm automatically to prevent safety accidents.

[0058] In summary: the blast furnace temperature measurement detection method based on the double-light image fusion matrix, in view of the temperature measurement demand of the on-site environment, sets the inspection task of the robot in cooperation with the algorithm of autonomous navigation of the robot, calls the image video stream collected by the robot pan-tilt camera, reads the visible light and thermal infrared images, performs image preprocessing, extracts the contour features in the image, uses the feature point detection to obtain the transformation matrix, and performs image registration and information fusion based on the transformation matrix to obtain new image data; the visible light image is used to identify and extract the contour of the direct blowing pipe, tuyere, gun bag and related positions, and the temperature information of the thermal infrared image is combined to realize the tuyere inspection temperature measurement algorithm based on the visible light and thermal infrared images, realize the automatic inspection of the high-temperature area of the direct blowing pipe, tuyere, gun bag and related positions, realize the no dead angle temperature measurement, and eliminate the blind area of manual point temperature measurement.

[0059] The above description is only the preferred embodiment of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can make equivalent replacements or changes according to the technical solution and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A method for detecting air vent temperature based on a dual-light image fusion matrix, characterized in that, Includes the following steps: S1: Call the image and video streams collected by the inspection robot's gimbal camera and read the visible light and thermal infrared images; S2: Gaussian filtering is used to preprocess the image to improve image quality for subsequent ROI region extraction; S3: Extract the ROI region of the target object in the image, and perform bilinear interpolation to scale it to the same resolution based on the ROI region features for subsequent feature point detection and image information fusion; S4: Use the AKAZE algorithm to detect feature points, obtain the digital transformation matrix, and perform image information fusion; S5: Uses the YOLOv5 algorithm to identify temperature measurement targets, including direct blowing pipes, air vents, and gun bags; S6: Using an image fusion matrix, the outline coordinates of the object identified in the visible light image are projected onto the infrared image. The outline of the object under test is extracted from the infrared image and then combined with the temperature data of the infrared image to achieve temperature measurement of the high-temperature areas of the direct blow pipe, air outlet, and gun bag. The specific method in S2 is as follows: Gaussian filtering is used to traverse the image. A certain pixel in the image is used as the center of the convolution kernel. The convolution kernel is used to calculate the weighted average of the neighboring pixels around the pixel. The result is used as the new pixel value of the current pixel, thereby realizing Gaussian filtering to remove noise from the image. The method for extracting ROI region features from an image using S3 is as follows: S301: Perform OTSU binarization thresholding segmentation on the image and Canny contour extraction, then use aspect ratio shape scale features for filtering to obtain the required target ROI region; S302: Extract the contour shape features of the ROI region, perform minimum bounding rectangle fitting, determine the scaling ratio based on the size of the bounding rectangle, and use bilinear interpolation to scale the image. In S4, the FED algorithm is used to solve the nonlinear diffusion equation when using the AKAZE algorithm. The FED algorithm solves the nonlinear diffusion equation as follows: ; Where I is the identity matrix, A(L) i ) is the transmission matrix of the image in dimension i, constructed from the gradient histogram of the scale effect after Gaussian filtering. Let be the time step, and t be the difference in evolution time. i-1 - t i .

2. The method for detecting air vent temperature based on a dual-light image fusion matrix according to claim 1, characterized in that, The S1 inspection robot has a built-in autonomous navigation programming program for pre-setting the robot's inspection tasks.

3. The method for detecting air vent temperature based on a dual-light image fusion matrix according to claim 1, characterized in that, The calculation method for bilinear interpolation in S3 is as follows: S301': Define four points Q11(x1,y1), Q12(x1,y2), Q21(x2,y1), Q22(x2,y2) of the image, with pixel values ​​f(Q11), f(Q12), f(Q21), f(Q22); S302': For the pixel value f(P) of any point P(x,y) between the four points, first perform linear interpolation in the x-direction, then perform linear interpolation in the y-direction, and obtain the following relationship: The pixel value of point P can be calculated using this relationship.

Images