Method and system for on-line detection of temperature field of molten metal fluid under dust interference

Abstract

The application discloses a kind of dust interference under molten metal fluid temperature field on-line detection method and system, by based on the feature point and mutual information of visible light image and infrared thermal image and based on the optical flow registration method of Kalman filtering, respectively visible light image and infrared thermal image are spatially registered and time registration, and establish dust transmissivity estimation model based on color consistency under visible light band, obtain dust transmissivity under visible light band, and by establishing dust transmissivity mapping model between visible light and infrared band, obtain dust transmissivity under infrared band;Finally according to dust transmissivity under infrared band and molten metal fluid temperature field detection model, realize on-line detection to molten metal fluid temperature field, solve the technical problem that due to dust interference leads to molten metal fluid temperature field on-line detection precision is low, overcome the influence of dust on infrared temperature measurement, realizes on-line accurate detection to molten metal fluid temperature field.

Description

TECHNICAL FIELD

[0001] The present application mainly relates to the technical field of blast furnace smelting, in particular to a molten metal fluid temperature field online detection method and system under dust interference. BACKGROUND

[0002] The molten metal is an intermediate product in the metal smelting process, mainly composed of metal melt and a small amount of slag. The temperature of the molten metal is not only an important indicator reflecting the quality of the metal melt, but also an important parameter reflecting the internal situation of the metal smelting furnace. Therefore, the accurate detection of the temperature of the molten metal is a problem to be solved in the smelting process.

[0003] The current detection methods of the molten metal mainly include contact temperature measurement and non-contact temperature measurement. The contact temperature measurement method includes fast thermocouple and blackbody cavity sensor; the non-contact measurement mainly uses infrared thermal imager and infrared thermometer. The fast thermocouple can only obtain a limited number of data each time and consumes a thermocouple; even the blackbody cavity sensor can only work continuously for ten or so hours, and cannot achieve the purpose of long-term online detection. The infrared thermal imager as a non-contact temperature measurement device based on infrared vision can realize long-term online detection of the temperature of the molten metal, but the intermittent and non-uniform distribution of dust in the actual production site seriously affects the temperature measurement accuracy of the infrared thermal imager.

[0004] After analyzing the advantages and disadvantages of the existing temperature measurement methods, the infrared thermal imager is still the most likely device to realize online temperature detection. Therefore, how to reduce the interference of dust on the infrared thermal imager becomes the key difficulty to realize online detection of the temperature of the molten metal. Considering that the infrared thermal image has less texture details and poor color perception ability, the present application introduces visible light information on the basis of the previous infrared temperature measurement system, proposes a molten metal fluid temperature field online detection method based on multi-source vision on the basis of the spatio-temporal registration of the visible light image and the infrared thermal image. The present application specifically analyzes the causes of the reduction of the brightness of the molten metal in the visible light image, estimates the dust transmittance in the visible light band, establishes a mapping model between the dust transmittance in the visible light band and the dust transmittance in the infrared band, and a molten metal fluid temperature field detection model based on the infrared radiation mechanism, overcomes the interference of dust on the infrared temperature measurement, and realizes online accurate detection of the temperature field of the molten metal fluid.

[0005] The patent with the patent publication number CN108998608A discloses a blast furnace tap hole molten iron temperature measurement method and system based on infrared machine vision, which proposes a tap hole molten iron temperature measurement method and system based on infrared machine vision, extracts the texture features of the infrared thermal image, describes the interference degree of dust on the infrared thermal image, constructs an infrared temperature measurement result compensation model, and realizes online detection of the molten iron temperature, but the compensation model constructed by data driving has a relatively long system deployment cycle, and is more suitable for blast furnace ironmaking sites with high-quality, well-labeled and rich-class data sets.

[0006] The patent with the patent publication number CN113834575A discloses a molten metal bath temperature device composed of an optical core wire, a sleeve and a separation element, which measures the molten metal temperature by means of a molten metal bath, reduces the consumption of the device in the molten metal, and achieves the purpose of multiple measurements. However, this method can only obtain a limited number of data each time, and it is difficult to realize online detection of the molten metal temperature, and there is a certain risk of manual operation. SUMMARY

[0007] The present application provides a molten metal fluid temperature field online detection method and system under dust interference, which solves the technical problem of low online detection accuracy of molten metal fluid temperature field caused by dust interference.

[0008] To solve the above technical problems, the molten metal fluid temperature field online detection method under dust interference provided by the present application comprises:

[0009] The visible light image and the infrared thermal image of the molten metal fluid are collected at the same time.

[0010] Based on the feature points and mutual information of the visible light image and the infrared thermal image, the visible light image and the infrared thermal image are spatially registered.

[0011] The optical flow registration method is used to time-register the visible light image and the infrared thermal image.

[0012] According to the visible light image after spatial matching and time matching, a dust transmittance estimation model based on color consistency in the visible light band is established, so as to obtain the dust transmittance in the visible light band.

[0013] A dust transmittance mapping model between the visible light band and the infrared band is established, and the dust transmittance in the infrared band is obtained according to the dust transmittance in the visible light band.

[0014] According to the dust transmittance in the infrared band and the molten metal fluid temperature field detection model based on the infrared radiation mechanism, the online detection of the molten metal fluid temperature field is realized.

[0015] Furthermore, based on the feature points and mutual information of the visible light image and the infrared thermal image, spatial registration of the visible light image and the infrared thermal image includes:

[0016] The visible light image is downsampled to obtain a visible light image with the same resolution as the infrared thermal image.

[0017] Bilateral filtering is applied to both the infrared thermal image and the downsampled visible light image.

[0018] The Canny operator is used to extract the edges of molten metal fluid from bilaterally filtered infrared thermal images and visible light images.

[0019] Feature point descriptors are established based on the edges of molten metal fluid in infrared thermal images and visible light images using the SIFT algorithm.

[0020] Based on the feature point descriptors, the RANSAC algorithm is used to perform feature point matching between infrared thermal images and visible light images. At the same time, based on the mutual information between visible light images and infrared thermal images, mutual information matching between infrared thermal images and visible light images is performed.

[0021] Calculate the registration error loss function during the feature point and mutual information matching process, and iteratively obtain the optimal spatial registration parameters based on the registration error loss function. The specific formula for the registration error loss function is as follows:

[0022] e pre =ω|e c |+ρ|e m |+ζ(0.3*e c -0.7*e m ) 2 ,

[0023] Where e pre Where represents the spatial registration error, ω, ρ, and ζ represent the first, second, and third penalty weights, respectively, and e c e represents the feature point registration error term. m This represents the mutual information registration error term.

[0024] Furthermore, the optical flow registration method is used to perform time registration between the visible light image and the infrared thermal image, including:

[0025] Harris corner detection is used to detect feature points in visible light and infrared thermal images.

[0026] The optical flow information of feature points in visible light images and infrared thermal images is calculated, and combined with their respective color intensity information, the visible light sparse optical flow sequence information and infrared sparse optical flow sequence information of molten metal fluid are obtained respectively.

[0027] Using the frame rate difference between the infrared thermal imager and the visible light camera as the theoretical value and the calculation result obtained from optical flow registration as the observation value, the state equation and observation equation of the time-series registration system are established.

[0028] Based on the state equation and observation equation of the time-series registration system, the Kalman filter equation set is determined, and based on the Kalman filter equation set, visible light sparse optical flow sequence information, and infrared sparse optical flow sequence information, time registration is performed on the visible light image and the infrared thermal image.

[0029] Furthermore, based on the state equation and observation equation of the time-series registration system, the Kalman filter equation set is determined. Then, based on the Kalman filter equation set, visible light sparse optical flow sequence information, and infrared sparse optical flow sequence information, time registration of the visible light image and infrared thermal image is performed, including:

[0030] Step 1: Based on the state equation and observation equation of the time-series registration system, determine the Kalman filter equation set, where the specific equations of the Kalman filter equation set are as follows:

[0031]

[0032] P i - =P i-1 +w i ,

[0033] K i =P i - (P i - +v i ) -1 ,

[0034]

[0035] P i = (1-K) i )P i - ,

[0036] in, P i - w i and v i Let K represent the registration result predicted by the i-th Kalman filter, the prior variance of the prediction, the system noise, and the observation noise, respectively. i , P i and y i Let represent the Kalman gain calculated by the i-th Kalman filter, the optimal registration result, the updated optimal variance, and the registration result obtained by sparse optical flow registration, respectively. and P i-1 Represent the optimal registration result and optimal variance calculated by the (i-1)th Kalman filter, respectively, where Δt represents the interval between two adjacent frames in the infrared thermal image, in fps. c This indicates the frame rate of the visible light camera used to acquire visible light images.

[0037] Step 2: Settings Both P1 and P1 are initialized to 1, w i and v i It is then initialized using historical registration errors.

[0038] Step 3: Obtain the observed value y through sparse optical flow registration. i Information on the sparse optical flow sequence in visible light The range of matching is Where t_start and t_end represent the start and end times of image acquisition, respectively. Step 4: Run the parameter prediction part of the Kalman filter to predict... and P i - The value of .

[0039] Step 5: Run the state update part of the Kalman filter and update the Kalman gain K. i With variance P i and output the prediction results.

[0040] Step 6: Return to Step 2 until registration is complete.

[0041] Furthermore, the observed value y is obtained through sparse optical flow registration. i include:

[0042] Select the visible light sparse optical flow sequence information CLK from the m-th visible light image. m Infrared sparse optical flow sequence information ILK of the nth frame infrared thermal image n ,in Constructing CLK using the RANSAC algorithm m With ILK n The correspondence between them.

[0043] According to CLK m With ILK n To determine the correspondence between them and obtain CLK m With ILK n The spatial registration parameters between the two time points are determined, and the corresponding temporal registration error is calculated. The m corresponding to the minimum temporal registration error is taken as the registration result.

[0044] Furthermore, the specific formula for calculating the time registration error is as follows:

[0045]

[0046] wherein E represents the time registration error, λ, η and are the first, second and third time matching weights respectively, u m (p), v m (p) and g m (p) respectively represent the horizontal vector, the vertical vector and the color intensity of the pth feature point in the visible light sparse optical flow sequence information of the mth frame of visible light image, u n (p), v n (p) and g n (p) respectively represent the horizontal vector, the vertical vector and the color intensity of the pth feature point in the infrared sparse optical flow sequence information of the nth frame of infrared thermal image.

[0047] Further, the specific formula of the dust transmissivity estimation model based on color consistency under the visible light band established according to the visible light image after spatial matching and time matching is:

[0048]

[0049] wherein represents the dust transmissivity under the visible light band at the coordinate y after the maximum operation of the visible light image in the rectangular region with the size of Ω, represents the pixel value at the coordinate y after the maximum operation of the visible light image in the rectangular region with the size of Ω and the color channel c respectively, A c represents the glow, J max represents the maximum value of the brightness of the molten metal fluid region.

[0050] Further, the calculation formula of the dust transmissivity under the infrared band obtained according to the dust transmissivity under the visible light band is:

[0051]

[0052] wherein, represents the dust transmissivity under the infrared band at the position y in the infrared thermal image, represents the w-th power of w represents the fitting coefficient corresponding to the w-th power of .

[0053] Further, the specific formula of the online detection of the molten metal fluid temperature field realized according to the dust transmissivity under the infrared band and the molten metal fluid temperature field detection model based on the infrared radiation mechanism is:

[0054]

[0055]

[0056] wherein, T ob represents the detected value of the temperature field of the molten metal fluid after removing the dust interference, ε ob represents the emissivity of the molten metal fluid, T rd represents the temperature detected by the infrared thermal imager, T u represents the ambient temperature, ε a represents the atmospheric emissivity, T a represents the atmospheric temperature, T de represents the temperature of the infrared thermal imager itself, δ represents a fitting coefficient, and represent the n-th power of T rd , T u , T a and T de respectively, t Inf represents the dust transmittance field in the infrared band after optimization, f guided_filter is a guided filter function, is the dust transmittance field in the infrared band estimated after local spatial maximization of the visible light image, Inf n is the nth infrared thermal image corresponding to the mth visible light image after time registration.

[0057] The present application provides a molten metal fluid temperature field online detection system under dust interference, comprising:

[0058] a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the molten metal fluid temperature field online detection method under dust interference provided by the present application when executing the computer program.

[0059] The application provides a dust interference molten metal fluid temperature field online detection method and system, first, aiming at the inconsistent resolution, field of view and angle of view of the visible light camera and the infrared thermal imager, a feature point and mutual information based cooperative registration method is proposed, so that the visible light image and the infrared thermal image are registered in the spatial dimension. Secondly, considering the inconsistency of the visible light image and the infrared thermal image in the time dimension, a light flow registration method based on Kalman filtering is proposed, which realizes the registration of the visible light image and the infrared thermal image in time and greatly improves the registration efficiency. Moreover, for the phenomenon that the color of the molten metal region becomes dark and the contrast is reduced under the dust interference, a dust transmittance estimation method based on color consistency in the visible light band is proposed. Finally, aiming at the different transmittance of the dust in different bands, a mapping model between the dust transmittance in the visible light band and the dust transmittance in the infrared band is established, and according to the infrared radiation mechanism, a molten metal fluid temperature field detection model is established, so as to overcome the influence of dust on infrared temperature measurement and realize the online accurate detection of the molten metal fluid temperature field. The multi-source vision online detection method and system proposed by the application can accurately detect the molten metal fluid temperature field for a long time, and has the advantages of simple deployment, safe operation, long service life, high temperature measurement precision and the like.

[0060] The beneficial effects of the application specifically include:

[0061] (1) For the first time in the molten metal temperature detection method based on infrared vision, visible light information is introduced to assist in depicting the infrared temperature measurement error caused by dust, overcoming the problem that the dust transmittance field in the infrared thermal image is difficult to accurately estimate, and realizing accurate detection of the molten metal fluid temperature field disturbed by dust;

[0062] (2) A feature point and mutual information based cooperative registration method is proposed, which introduces global mutual information registration on the basis of local region feature point registration, and designs a corresponding target optimization function, comprehensively considers local registration and global registration, reduces the influence of object motion at different times on spatial registration, and realizes spatial registration at different times;

[0063] (3) A light flow time sequence registration method based on Kalman filtering is designed. Sparse light flow is used to estimate the object motion trajectory, and color intensity information is introduced, and then the corresponding relationship between different video sequences is established. The sparse light flow registration result and the theoretical registration result are effectively combined through Kalman filtering, which ensures the time sequence registration accuracy, narrows the search range of light flow registration, and improves the registration efficiency;

[0064] (4) A dust transmittance estimation model based on color consistency in the visible light band is established. According to the characteristics of high brightness and high temperature of the molten metal fluid, the color consistency prior is first proposed, the transmittance estimation model based on color consistency in the visible light band is established, and the corresponding model parameter estimation method is designed, so that the influence of the dust on the molten metal fluid region is accurately quantified from the visible light image.

[0065] (5) A dust transmittance mapping model between the visible light and infrared bands and a molten metal fluid temperature field detection model based on the infrared radiation mechanism are constructed, the molten metal fluid temperature field online detection based on multi-source vision is realized, and the interference of the dust on the temperature measurement result of the infrared thermal imager is effectively overcome. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 It is a multi-source vision online detection system schematic diagram of the second embodiment of the present application.

[0067] Figure 2 It is an implementation step diagram of the molten metal fluid temperature field online detection method based on multi-source vision of the second embodiment of the present application.

[0068] Figure 3 (a) is a visible light image of the molten iron flow under the interference of the dust in the second embodiment of the present application, and (b) is a transmittance field obtained by using the dust transmittance estimation method based on color consistency in the visible light band in the second embodiment of the present application.

[0069] Figure 4 It is a molten iron flow temperature field image after removing the dust interference by using the second embodiment of the present application.

[0070] Figure 5 It is a structure block diagram of the molten metal fluid temperature field online detection system under the interference of the dust in the embodiment of the present application.

[0071] Reference signs:

[0072] 1, molten metal fluid; 2, dust appearing in metal smelting; 3, multi-source vision detection device; 4, comprehensive cable; 5, computer; 10, storage; 20, processor. DETAILED DESCRIPTION

[0073] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings and preferred embodiments, but the protection scope of the present application is not limited to the following specific embodiments.

[0074] The embodiments of the present application will be described in detail below with reference to the accompanying drawings, but the present application can be implemented in various different ways limited and covered by the claims.

[0075] Embodiment one

[0076] The method for on-line detection of temperature field of molten metal fluid under dust interference provided by the embodiment one of the present application comprises:

[0077] In step S101, visible light images and infrared thermal images of the molten metal fluid are collected simultaneously.

[0078] In step S102, the visible light images and the infrared thermal images are spatially registered based on feature points and mutual information of the visible light images and the infrared thermal images.

[0079] In step S103, the visible light images and the infrared thermal images are temporally registered by using an optical flow registration method.

[0080] In step S104, a dust transmittance estimation model based on color consistency in the visible light band is established according to the visible light images after spatial matching and temporal matching, so as to obtain the dust transmittance in the visible light band.

[0081] In step S105, a dust transmittance mapping model between the visible light band and the infrared band is established, and the dust transmittance in the infrared band is obtained according to the dust transmittance in the visible light band.

[0082] In step S106, on-line detection of the temperature field of the molten metal fluid is realized according to the dust transmittance in the infrared band and a molten metal fluid temperature field detection model based on infrared radiation mechanism.

[0083] The method for on-line detection of temperature field of molten metal fluid under dust interference provided by the embodiment of the present application collects visible light images and infrared thermal images of the molten metal fluid simultaneously, spatially registers the visible light images and the infrared thermal images based on feature points and mutual information of the visible light images and the infrared thermal images, temporally registers the visible light images and the infrared thermal images by using an optical flow registration method, establishes a dust transmittance estimation model based on color consistency in the visible light band according to the visible light images after spatial matching and temporal matching, so as to obtain the dust transmittance in the visible light band, establishes a dust transmittance mapping model between the visible light band and the infrared band, and obtains the dust transmittance in the infrared band according to the dust transmittance in the visible light band, and realizes on-line detection of the temperature field of the molten metal fluid according to the dust transmittance in the infrared band and a molten metal fluid temperature field detection model based on infrared radiation mechanism, thereby overcoming the influence of dust on infrared temperature measurement and realizing on-line and accurate detection of the temperature field of the molten metal fluid.

[0084] Embodiment two

[0085] The embodiment two of the present application provides a method for on-line detection of temperature field of molten metal fluid based on multi-source vision, Figure 1It is a schematic diagram of a multi-source vision online detection system, which comprises a molten metal fluid 1, dust 2 appearing in the process of metal smelting, a multi-source vision detection device 3, a comprehensive cable 4 and a computer 5, wherein the multi-source vision detection device specifically comprises a visible light camera, an infrared thermal imager and a water cooling device and the like.

[0086] Figure 2 It is an implementation step diagram of a molten metal fluid temperature field online detection method based on multi-source vision, comprising the following steps:

[0087] (1) A collaborative registration method based on feature points and mutual information is used to realize the registration of visible light images and infrared thermal images in space.

[0088] (2) A light flow registration method based on Kalman filtering is used to realize the registration of visible light images and infrared thermal images in time.

[0089] (3) A dust transmittance estimation method based on color consistency in the visible light band is proposed for the phenomenon of color darkening of the light source area in the visible light image under the interference of dust.

[0090] (4) Considering that the transmittance of dust changes with the change of wavelength, a dust transmittance mapping model between the visible light and infrared bands is established, the dust transmittance in the infrared band is obtained, and a molten metal fluid temperature field detection model based on infrared radiation mechanism is constructed, realizing the online accurate detection of the molten metal fluid temperature field.

[0091] The specific implementation scheme is as follows:

[0092] (1) A collaborative registration method based on feature points and mutual information is used to realize the registration of visible light images and infrared thermal images in space.

[0093] Single sensor is often limited by imaging angle, waveband range and lighting environment, resulting in that the obtained image only contains part of the feature information in the scene, and there is a limitation of insufficient data amount. Therefore, the visible light camera and the infrared thermal imager are used to analyze the molten metal fluid in the present application, but due to the different resolutions between the two, the difference in spatial position of the camera and the non-parallelism of the camera optical axis, there is often a geometric transformation (translation, rotation, scaling, etc.) between the two images. Therefore, the present application proposes a collaborative registration method based on feature points and mutual information to realize the registration of the two images in space, so as to facilitate the extraction and matching of useful information by subsequent algorithms. The specific method is as follows:

[0094] Firstly, considering the difference in resolution between the infrared thermal imager and the visible light camera, the present application divides the batch collected visible light images and infrared thermal images into visible light image groups and infrared thermal image groups and the visible light image group Down-sampling the visible light image group so that the infrared thermal image group is consistent with the resolution of the visible light image group .

[0095]

[0096] where the max-pooling operator f Maxpooling has a kernel size of [w Col / w Inf , h Col / h Inf ], w Col and h Col are the width and height of the visible light image, and w Inf and h Inf are the width and height of the visible light image. The subscript t_start and superscript t_end represent the start time and end time of the image group acquisition time period, respectively.

[0097] Secondly, since the infrared thermal imager and the visible light camera have not been registered in time series, spatial registration needs to be achieved under different time series. There are two methods to achieve spatial registration in different time, one is to ensure that the motion state of the object does not change, and the other is to remove the feature information brought by the change of the motion state of the object. Since the molten metal fluid has the characteristics of high temperature, high speed and high corrosion, and there is high contrast between the molten metal fluid region and the background region in the image, in order to remove the motion information brought by the high-speed motion of the molten metal fluid while keeping the edge information as much as possible, the visible light image group and the infrared thermal image group are respectively subjected to bilateral filtering to obtain the image group and The mathematical form of the bilateral filtering is shown in equation (2).

[0098]

[0099] where the weight p is the position of the calculated pixel value, S is the convolution kernel coverage area, is the calculated pixel value, I(p) is the calculated pixel value, I(q) is the pixel value in the convolution coverage area, G c and G s represent the distance and color weight, respectively.

[0100] Because the texture information inside the molten metal fluid in high-speed motion state will interfere with the calculation accuracy of the feature point registration method, after the image is subjected to bilateral filtering, the edge of the molten metal fluid in the image is obtained by using a Canny operator and , then a feature point descriptor is established by using a Sift algorithm, the feature points are matched by using a RANSAC algorithm, and finally, the coarse spatial registration parameter (pre-registration) is calculated by using formula (3).

[0101]

[0102] wherein and represent a plurality of groups of feature point position coordinates established by the RANSAC algorithm, and M represents the coarse spatial registration parameter.

[0103] Finally, because the visible light image group and the infrared thermal image group are not registered in the time dimension, the feature point descriptor is extracted by using the Sift algorithm, and the corresponding feature point relationship is established by using the RANSAC algorithm, which has defects in the bottom layer, that is, the feature point matching between the visible light image group and the infrared thermal image group has the problems of displacement, misplacement and even mismatching. Therefore, in order to improve the registration accuracy, the application provides a spatial matching method for cooperative fine registration of feature points and mutual information (cooperative registration), and the specific steps are as follows:

[0104] Step 1: selecting m as an initial spatial registration parameter, wherein m=M.

[0105] Step 2: calculating the jth registration error j according to the spatial registration parameter m wherein the loss function of the registration error is represented as follows:

[0106] e pre = ω|e c | + ρ|e m | + ζ (0.3 * e c - 0.7 * e m ) 2 (4)

[0107] wherein ω, ρ and ζ all represent penalty weights, e c represents a feature point registration error term, and the calculation method is as follows:

[0108]

[0109] e m represents a mutual information registration error term, and the calculation method is as follows:

[0110]

[0111] Step3: use the Powell algorithm to optimize the loss function , and calculate the spatial registration parameters m j+1 of the j+1th iteration. When the error change is greater than 0.001, return to Step 2; when it is less than or equal to 0.001, end the loop and output the precise registration calculation result.

[0112] (2) Design a light flow time sequence registration method based on Kalman filtering to realize the registration of visible light images and infrared thermal images in the time dimension.

[0113] Since the molten metal fluid is in a high-speed motion state, and the infrared thermal imager and the visible light camera each have different clock sources inside, it cannot be ensured that the two cameras capture the molten metal fluid in the same motion state at the same time. Considering that there is a certain amount of offset in the motion information extracted from the visible light images and the infrared thermal images, directly using the optical flow method will cause system errors. Therefore, the present application proposes a light flow time sequence registration method based on Kalman filtering, and the specific steps are as follows:

[0114] Step 1: Detect feature points from the visible light image group using Harris corner points, calculate the optical flow information of these feature points, and combine the corresponding color intensity information to obtain the visible light sparse optical flow sequence information of the molten metal fluid (wherein each feature point is composed of a horizontal vector u, a vertical vector v, and a color intensity g). Perform the same operation on the infrared thermal image group to obtain the infrared sparse optical flow sequence information of the molten metal fluid

[0115] Step 2: Take the frame rate difference between the infrared thermal imager and the visible light camera as the theoretical value, and the calculation result obtained by the optical flow registration as the observed value, to obtain the state equation and the observation equation of the time sequence registration system as shown in formula (7).

[0116]

[0117] Wherein, x i represents the frame number of the visible light image group corresponding to the i-th frame of the infrared thermal image group, x i-1 represents the frame number of the visible light image group corresponding to the i-1-th frame of the infrared thermal image group, Δt represents the interval time length of adjacent two frames in the infrared thermal image group, fps c represents the acquisition frame rate of the visible light camera, w i and v i represent system noise and observation noise, both with a mean of zero, and y irepresents the frame number of the visible light image group corresponding to the i-th frame in the infrared thermal image group calculated by using the optical flow method.

[0118] Step 3: According to the state equation and the observation equation of the system, the Kalman filter equation set (see formula (8-1)-(8-5)) is determined.

[0119]

[0120] P i - = P i-1 + w i (8-2)

[0121] K i = P i - (P i - + v i ) -1 (8-3)

[0122]

[0123] P i = (1-K i ) P i - (8-5)

[0124] wherein formula (8-1) and (8-2) are the parameter prediction part, formula (8-3), (8-4) and (8-5) are the state update part, P i - , w i and v i respectively represent the predicted registration result, the predicted prior variance, the system noise and the observation noise of the i-th Kalman filter, K i , P i and y i respectively represent the Kalman gain, the optimal registration result, the updated optimal variance and the registration result obtained by the sparse optical flow registration calculated by the i-th Kalman filter, and P i-1 respectively represent the optimal registration result and the optimal variance calculated by the i-1-th Kalman filter, Δt represents the interval time length of the adjacent two frames in the infrared thermal image, fps c represents the acquisition frame rate of the visible light camera used for acquiring the visible light image.

[0125] Step 4: Set and P1 are both 1, w i and v iIt is then initialized using historical registration errors.

[0126] Step 5: Obtain the observed value y through sparse optical flow registration. i Information on the sparse optical flow sequence in visible light The range of matching is

[0127] Step 6: Run the parameter prediction part of the Kalman filter to predict... and P i - The value of .

[0128] Step 7: Run the state update part of the Kalman filter and update the Kalman gain K. i With variance P i and output the prediction results.

[0129] Step 8: Return to Step 5 until registration is complete.

[0130] The specific steps for sparse optical flow registration are as follows:

[0131] step <1> : Select the visible light sparse optical flow sequence information CLK from the m-th visible light image. m Infrared sparse optical flow sequence information ILK of the nth frame infrared thermal image n ,in

[0132] step <2> : Constructing CLK using the RANSAC algorithm m With ILK n The correspondence between them.

[0133] step <3> According to CLK m With ILK n To determine the correspondence between them and obtain CLK m With ILK n The spatial registration parameters are determined, and then the corresponding temporal registration error is calculated using equation (9). The m corresponding to the minimum temporal registration error is taken as the registration result.

[0134]

[0135] (3) Establish a dust transmittance estimation model based on color consistency in the visible light band and obtain the corresponding transmittance field.

[0136] With the operation of the metallurgical reaction furnace, the molten metal fluid ejected at the outlet of the reaction furnace will cause a large amount of dust when falling into the channel. The dust diffused between the multi-source visual detection device and the molten metal fluid will not only cause the color of the molten metal fluid in the visible light image to be dark, the contrast to be reduced, the texture to be blurred and the like, but also will seriously interfere with the temperature field of the molten metal fluid estimated from the infrared thermal image. Therefore, the application proposes a dust transmittance estimation model based on color consistency in the visible light band, estimates the dust transmittance field from the visible light image, and applies it to the infrared thermal image, so as to compensate for the temperature field of the molten metal fluid under the interference of the dust.

[0137] Through the analysis of the molten metal fluid image under the interference of the dust, it is found that, unlike the color being bright in the daytime haze scene, the color of the molten metal fluid region under the interference of the dust is seriously dark and the color tone is uneven. Therefore, according to the atmospheric scattering model, the application reanalyzes the reason for the reduced visibility of the molten metal fluid region under the daytime haze and the interference of the dust, and finds that the biggest difference between the molten metal fluid region under the interference of the dust and the daytime environment is whether the atmospheric light exists. Instead of the atmospheric light, the molten metal is high-brightness, which leads to the fact that the scattering of the dust cannot improve the brightness of the molten metal, and finally the color of the molten metal fluid under the interference of the dust is dark and the color tone is uneven.

[0138] The atmospheric scattering model is a model for simulating the color and brightness of light after single or multiple scattering in the atmosphere to the observer (camera), and its specific form is as follows:

[0139] I(x) = J(x)t + A(1-t) (10)

[0140] Where I(x) represents the observed color and brightness, J(x) represents the color and brightness emitted by the object itself, t represents the transmittance, and A represents the glow generated after the light emitted by the object acts on the surrounding atmosphere.

[0141] According to the atmospheric scattering model, when there are suspended particles in the atmosphere, the transmittance of the transmission medium between the target and the camera will be reduced due to the scattering of the particles, thereby causing the target brightness to be reduced, the contrast to be reduced, the texture details to be blurred and the like. In the research on the reason for the reduced visibility in the daytime haze, most of the analysis is based on the atmospheric scattering model. Although the color of the molten metal fluid region is dark and the color tone is uneven, which can also be explained by the atmospheric scattering model, it is different from some important prior conditions in the daytime environment, which leads to the fact that the previous daytime defogging method cannot perform the original performance in this kind of environment.

[0142] To this end, the embodiment of the present application firstly assumes that the transmittance of the dust is constant in a local space with a size of Ω(x), which is specifically represented as The atmospheric scattering model is maximized in the local space Ω(x) and the color channel c∈(R, G, B) respectively to obtain formula (11).

[0143]

[0144] According to the color consistency prior (formula (12)) proposed by the embodiment of the present application, the color maximum J max of the light source object in the local space under the interference of the dust tends to be constant.

[0145]

[0146] After substituting it into formula (12) and performing operations such as combining like terms and moving terms, the dust transmittance estimation model (formula (13)) based on color consistency in the visible light band can be obtained.

[0147]

[0148] wherein J max represents the maximum brightness of the molten metal fluid region, A c represents the glow, that is, the light emitted by the molten metal fluid to the surrounding region, represents the transmittance at the position y, represents the maximum value of each color channel in the region with the size of Ω.

[0149] Moreover, when the observed tends to the maximum brightness J max of the molten metal fluid region, the transmittance tends to 1, as shown in formula (14); when the observed tends to the glow A c , the transmittance tends to 0, as shown in formula (15), which is consistent with the interference of the dust to the light source region observed by the camera.

[0150]

[0151]

[0152] For the selection of the maximum brightness J max of the molten metal fluid region and the glow A c , the specific steps are as follows:

[0153] Step 1: Firstly, the molten metal fluid region is filtered to remove abnormal light spots;

[0154] Step2: Histogram equalization is performed on the molten metal fluid region, denoted as I IHE ;

[0155] Step3: Multi-threshold Otsu method is performed on I IHE , two threshold values are extracted, and the molten metal fluid region is divided into a highlight area, a glow area, and a background area;

[0156] Step4: The first 1% of pixel values in the highlight area and the glow area are selected respectively, and the mean values are calculated, the mean value of the highlight area is J max , and the mean value of the glow area is A c .

[0157] After J max and A c are determined, the transmittance field at this moment can be calculated from the visible light image by formula (13).

[0158] (4) A dust transmittance mapping model between visible light and infrared wave bands is established, combined with a molten metal fluid temperature field detection model based on infrared radiation mechanism, to realize online detection of the molten metal fluid temperature field.

[0159] Light, as an electromagnetic wave, has its penetration ability determined by wavelength. When the wavelength is shorter, the penetration ability is also enhanced. Therefore, even under the same thickness of dust interference, different wave bands of light will show different transmittance.

[0160] In order to remove the interference of the penetration ability of light itself on the dust transmittance estimation, the present application establishes a dust transmittance mapping model between visible light and infrared wave bands by the least square fitting method. The specific steps are as follows: in the laboratory, an infrared thermal imager, a visible light camera, a black body furnace and metallurgical dust are used for experiments.

[0161] Step1: Set the temperature of the black body furnace to T b , and keep the ambient temperature and the glow (ambient light) constant, and use the infrared thermal imager and the visible light camera to obtain the images of the black body furnace in this state;

[0162] Step2: Place metallurgical dust with a thickness of h dust between the infrared thermal imager, the visible light camera and the black body furnace;

[0163] Step3: Use the infrared thermal imager and the visible light camera to obtain the images of the black body furnace in this state;

[0164] Step4: Increase the thickness of the dust Δh dust in turn, and use the infrared thermal imager and the visible light camera to obtain the images of the black body furnace under different dust thicknesses;

[0165] Step5: Calculate the dust transmittance in infrared and visible light images using the blackbody furnace images taken at different dust thicknesses;

[0166] Step6: According to the obtained infrared and visible light transmittance at different dust thicknesses, use the least squares method to construct a dust transmittance mapping model between visible and infrared bands, as shown in equation (16), where represents the transmittance in the infrared band, represents the transmittance in the visible light, a and w are experimental fitting coefficients and order.

[0167]

[0168] where, represents the dust transmittance in the infrared band at position y in the infrared thermal image, represents w order of a w represents the fitting coefficient corresponding to the w order of .

[0169] After obtaining the dust transmittance mapping model between visible and infrared bands, first, through the model, convert the estimated dust transmittance field in the visible light image to the dust transmittance field in the infrared band Second, through the guided filtering technology, take the registered infrared thermal image Inf n at the same time as the guide image, optimize the dust transmittance field in the infrared band to obtain t Inf , and realize accurate characterization of the dust transmittance in the infrared thermal image.

[0170]

[0171] where f guided_filter is the guided filtering function, t Inf is the optimized dust transmittance field in the infrared band, is the estimated dust transmittance field in the infrared band after local spatial maximization of the visible light image, Inf n is the nth frame of infrared thermal image corresponding to the mth frame of visible light image after time registration.

[0172] In order to realize the online detection of the temperature field of molten metal fluid, the embodiment of the present application establishes a temperature compensation model according to the infrared temperature measurement principle. When there is no dust interference in the optical path between the measured object and the infrared thermal imager, the infrared thermal imager receives infrared radiation W rd mainly from the infrared radiation brightness ε ob τa W ob , the infrared radiation brightness of the surroundings emitted and reflected from the object ob τ a W u , the infrared radiation brightness of the surroundings a W a and the infrared radiation brightness W of the infrared thermal imager itself de as shown in equation (17).

[0173] W rd = ε ob τ a W ob + p ob τ a W u + ε a W a - W de (17)

[0174] wherein W ob , W u , W a and W de represent the infrared radiation brightness of the measured object, the infrared radiation brightness of the surroundings, the radiation brightness of the atmosphere and the infrared radiation brightness of the infrared thermal imager itself. ε ob and p ob represent the emissivity and reflectivity of the measured object, ε a and τ a represent the atmospheric emissivity and transmissivity.

[0175] When dust interference exists in the optical path between the measured object and the infrared thermal imager, the infrared radiation brightness ε ob τ a W ob of the measured object itself and the infrared radiation brightness p ob τ a W u of the surroundings emitted and reflected from the object can only be received by the infrared thermal imager after attenuation by the dust transmissivity τ dust and the received infrared radiation W rd is as shown in equation (18).

[0176] W rd = ε ob τ a τ dust W ob + p ob τ a τ dust W u + ε a W a - Wde (18)

[0177] By comparing equation (17) and (18), it can be found that the dust interferes with the infrared temperature measurement by changing the transmissivity of the light path between the measured object and the infrared thermal imager. Therefore, after assuming that the transmissivity of the light path between the measured object and the infrared thermal imager is τ, the infrared radiation received by the infrared thermal imager is shown in equation (19).

[0178] W rd = ε ob τW ob + ρ ob τW u + ε a W a -W de (19)

[0179] Wherein when there is no dust interference between the light path of the measured object and the infrared thermal imager, τ = τ a ; when there is dust interference between the light path, τ = τ a τ dust .

[0180] According to the principle of infrared radiation temperature measurement, f(T) can be used to represent the relationship between the infrared radiation brightness and the temperature. Therefore, the voltage signal output by each detector in the infrared thermal imager can be represented by equation (20).

[0181] f(T rd ) = ε ob τf(T ob ) + ρ ob τf(T u ) + ε a f(T a ) - f(T de ) (20)

[0182] Since the sum of the absorption, transmissivity and reflectivity of the object at the same wavelength is 1, and when the measured object is an opaque molten metal fluid, its transmissivity can be considered as 0. Therefore, the relationship between the emissivity and reflectivity of the molten metal fluid can be established by equation (21).

[0183] ε ob = 1 - ρ ob (21)

[0184] By combining equation (20) and (21), the voltage signal corresponding to the infrared radiation emitted by the molten metal object in the infrared thermal imager can be obtained, as shown in equation (22).

[0185]

[0186] According to the infrared radiation mechanism, within a certain wavelength range, the voltage signal detected by the infrared thermal imager has a certain relationship with the temperature of the object being measured, as shown in equation (23).

[0187] f(T) = αT δ (twenty three)

[0188] In the formula, α and δ are fitting coefficients, which depend on the aperture, filter, and detector materials of the infrared thermal imager.

[0189] Substituting equation (23) into equation (22) yields the molten metal fluid temperature field detection model based on the infrared radiation mechanism, as shown in equation (24).

[0190]

[0191] The infrared thermal imager's own temperature T in the temperature field mapping model de Fitting coefficient δ, ambient temperature T u Atmospheric temperature T a Emissivity ε of molten metal fluid ob and atmospheric emissivity ε a These parameters can all be obtained on-site through detection or calibration. After obtaining these parameters, the dust transmittance field t in the infrared band estimated from the image will be used. Inf (The transmittance in the dust transmittance field is 1 when there is no dust) and infrared thermal image Inf n As τ and T respectively rd By substituting the data into the molten metal fluid temperature field detection model based on infrared radiation mechanism, the molten metal fluid temperature field after removing dust interference can be obtained.

[0192] The embodiment of the present application takes molten metal fluid as the research object, and proposes a molten metal fluid temperature field online detection method and system based on multi-source vision. The infrared thermal imager and the visible light camera are used to capture the molten metal fluid image in real time. A collaborative registration method based on feature points and mutual information is proposed for the differences in resolution, viewing angle and field of view between the infrared thermal imager and the visible light camera, so as to realize the spatial registration between the infrared thermal image and the visible light image at different time sequences. A light flow time sequence registration method based on Kalman filtering is proposed for the problem of different internal clock sources of the infrared thermal imager and the visible light, so as to realize the unification of the infrared thermal image and the visible light image in the time dimension. A dust transmittance estimation model based on color consistency in the visible light band is established for the problem that the temperature measurement method based on infrared vision is easily disturbed by dust in the light path, and the influence of the dust on the molten metal fluid region is described from the perspective of visible light. Finally, a dust transmittance mapping model between visible light and infrared bands and a molten metal fluid temperature field detection model based on infrared radiation mechanism are constructed, the influence of the dust on the molten metal fluid temperature field detection result in the infrared band is quantified, and the online detection of the molten metal fluid temperature field is realized.

[0193] The embodiment takes the 1050m 3 high blast furnace of a certain iron mill as the test platform, and applies the molten metal fluid temperature multi-state detection method and system to the online detection of the temperature of the molten iron stream at the No. 1 tapping hole of the blast furnace. The multi-source vision detection system is installed at the tapping hole to obtain the image of the iron stream during tapping. Figure 3 In (a), the visible light image of the molten iron stream under dust interference is obtained by the embodiment of the present application, and (b) is the transmittance field obtained by the dust transmittance estimation method based on color consistency in the visible light band. It can be known from Figure 3 that the dust transmittance is obviously reduced in the area obviously disturbed by dust, which shows that the dust transmittance estimation method based on color consistency in the visible light band can effectively measure the interference of the dust on the molten metal. Figure 4 The molten iron stream temperature field image after removing the dust interference is shown. It can be known from Figure 4 that the temperature in the dust interference area is obviously improved, the infrared temperature measurement error caused by the dust is effectively compensated, and the effectiveness of the present application in detecting the molten metal fluid temperature field is shown.

[0194] With reference to Figure 5 , the molten metal fluid temperature field online detection system under dust interference proposed by the embodiment of the present application includes a memory 10, a processor 20, and a computer program stored in the memory 10 and executable on the processor 20, wherein the processor 20 realizes the steps of the molten metal fluid temperature field online detection method under dust interference proposed by the embodiment when executing the computer program.

[0195] The specific working process and working principle of the molten metal fluid temperature field on-line detection system under dust interference of the embodiment can refer to the working process and working principle of the molten metal fluid temperature field on-line detection method under dust interference of the embodiment.

[0196] The above only is the preferred embodiment of the present application, and is not used to limit the present application, for the person skilled in the art, the present application can have various changes and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for on-line detection of temperature field of molten metal fluid under dust disturbance, characterized in that, The method comprises: Simultaneously collecting visible light images and infrared thermal images of the molten metal fluid; Based on the feature points and mutual information of the visible light images and the infrared thermal images, spatially registering the visible light images and the infrared thermal images; Using an optical flow registration method, temporally registering the visible light images and the infrared thermal images; According to the visible light images after spatial matching and temporal matching, establishing a dust transmittance estimation model based on color consistency in the visible light band, so as to obtain the dust transmittance in the visible light band; Establishing a dust transmittance mapping model between the visible light band and the infrared band, and obtaining the dust transmittance in the infrared band according to the dust transmittance in the visible light band; According to the dust transmittance in the infrared band and a molten metal fluid temperature field detection model based on infrared radiation mechanism, realizing online detection of the molten metal fluid temperature field.

2. The method according to claim 1, wherein Based on the feature points and mutual information of the visible light images and the infrared thermal images, spatially registering the visible light images and the infrared thermal images comprises: Down-sampling the visible light images to obtain visible light images with the same resolution as the infrared thermal images; Bilateral filtering the infrared thermal images and the down-sampled visible light images; Using a Canny operator to extract the molten metal fluid edges in the bilateral filtered infrared thermal images and visible light images; Based on the molten metal fluid edges in the infrared thermal images and the visible light images, establishing feature point descriptors through a Sift algorithm; According to the feature point descriptors, using a RANSAC algorithm to match the feature points of the infrared thermal images and the visible light images, and simultaneously matching the mutual information of the infrared thermal images and the visible light images according to the mutual information of the visible light images and the infrared thermal images; Calculating a registration error loss function in the process of matching the feature points and the mutual information, and iteratively obtaining optimal spatial registration parameters according to the registration error loss function, wherein the specific formula of the registration error loss function is: e pre = ω |e c | + ρ |e m | + ζ (0.3 * e c - 0.7 * e m ) 2 , where e pre where e c represents the feature point registration error term, e m represents the mutual information registration error term.

3. The method according to claim 2, wherein Using an optical flow registration method, temporally registering the visible light images and the infrared thermal images comprises: Detecting feature points of the visible light images and the infrared thermal images using a Harris corner point; Calculating optical flow information of the feature points of the visible light images and the infrared thermal images, and combining respective color intensity information to respectively obtain visible light sparse optical flow sequence information and infrared sparse optical flow sequence information of the molten metal fluid; Taking the frame rate difference between the infrared thermal imager and the visible light camera as a theoretical value and the calculation result obtained by the optical flow registration as an observation value, establishing a state equation and an observation equation of a time sequence registration system; According to the state equation and the observation equation of the time sequence registration system, determining a Kalman filter equation set, and according to the Kalman filter equation set and the visible light sparse optical flow sequence information and the infrared sparse optical flow sequence information, temporally registering the visible light images and the infrared thermal images.

4. The method according to claim 3, wherein According to the state equation and the observation equation of the time sequence registration system, determining a Kalman filter equation set, and according to the Kalman filter equation set and the visible light sparse optical flow sequence information and the infrared sparse optical flow sequence information, temporally registering the visible light images and the infrared thermal images comprises: Step 1: According to the state equation and observation equation of the time sequence registration system, determine the Kalman filter equation set, wherein the specific equation of the Kalman filter equation set is: wherein, w i and v i respectively represent the predicted registration result, the predicted prior variance, the system noise and the observation noise of the i-th Kalman filtering, K i , P i and y i respectively represent the Kalman gain, the optimal registration result, the updated optimal variance and the registration result obtained by the sparse optical flow registration calculated by the i-th Kalman filtering, and P i-1 respectively represent the optimal registration result and the optimal variance calculated by the i-1-th Kalman filtering, Δt represents the interval time length of adjacent two frames in the infrared thermal image, fps c represents the acquisition frame rate of the visible light camera used for acquiring the visible light image; Step 2: Set and the initial value of P1 is 1, w i and v i initialized by historical registration error; Step 3: Obtain observation value y by sparse optical flow registration i wherein the range matched in the visible light sparse optical flow sequence information wherein t_start and t_end represent the start time and the end time of image acquisition respectively;​ Step 4: Run the parameter prediction part of the Kalman filter to predict and P i - values; Step 5: Run the state update part of the Kalman filter to update the Kalman gain K i with the variance P i and output the prediction result this time Step 6: Return to Step 2 until the registration is completed.

5. The method according to claim 4, wherein Obtaining observations y by sparse optical flow registration i comprising: selecting a visible light sparse optical flow sequence information CLK of the mth frame of visible light image m and an infrared sparse optical flow sequence information ILK of the nth frame of infrared thermal image n wherein Constructing CLK using RANSAC algorithm m correspondence between ILK n and CLK; According to the correspondence relationship between CLK m and ILK n , the spatial registration parameters between CLK m and ILK n are calculated, and the corresponding time registration error is calculated, and the m corresponding to the minimum value of the time registration error is taken as the registration result.

6. The method according to claim 5, wherein The specific formula for calculating the time registration error is: wherein E represents the time registration error, λ, η and l are the first, second and third time matching weights, respectively, and u m (p), v m (p) and g m (p) respectively represent the horizontal vector, the vertical vector and the color intensity of the pth feature point in the visible light sparse optical flow sequence information of the mth frame of visible light image, u n (p), v n (p) and g n (p) respectively represent the horizontal vector, the vertical vector and the color intensity of the pth feature point in the infrared sparse optical flow sequence information of the nth frame of infrared thermal image.

7. The method according to any one of claims 1 to 6, wherein the method is characterized by, According to the visible light image after spatial matching and time matching, the specific formula of the dust transmittance estimation model based on color consistency in the visible light band is established as: wherein represents the dust transmittance in the visible light band at the coordinate y after a maximum operation is performed on the visible light image in a rectangular region of size Ω, represents the pixel value at the coordinate y after a maximum operation is performed on the visible light image in a rectangular region of size Ω and in the color channel c, respectively, A c represents the glow, J max represents the molten metal fluid region brightness maximum.

8. The method according to claim 7, wherein According to the dust transmittance in the visible light band, the calculation formula of the dust transmittance in the infrared band is obtained as: wherein, represents the dust transmittance in the infrared wave band at position y in the infrared thermal image, represents of w-th order, a w represents a fitting coefficient corresponding to the w-th order of of w-th order.

9. The method according to claim 8, wherein According to the dust transmittance in the infrared band and the molten metal fluid temperature field detection model based on the infrared radiation mechanism, the specific formula for realizing online detection of the molten metal fluid temperature field is: wherein T ob represents the detected temperature field of the molten metal fluid after removing the dust interference, ε ob represents the emissivity of the molten metal fluid, T rd represents the temperature detected by the infrared thermal imager, T u represents the ambient temperature, ε a represents the atmospheric emissivity, T a represents the atmospheric temperature, T de represents the temperature of the infrared thermal imager itself, δ represents a fitting coefficient, and respectively represent the n-th power of T rd , T u , T a and T de , t Inf represents the dust transmittance field in the infrared band after optimization, f guided_filter is a guided filter function, is the dust transmittance field in the infrared band estimated after local spatial maximization of the visible light image, Inf n is the n-th infrared thermal image corresponding to the m-th visible light image after time registration.

10. A molten metal fluid temperature field online detection system under dust interference, the system comprising: A memory (10), a processor (20), and a computer program stored on the memory (10) and executable on the processor (20), characterized in that the processor (20) implements the steps of the method of any one of claims 1 to 9 when executing the computer program.

Images