Flotation foam flow velocity detection method based on NSST domain infrared target segmentation and SURF matching

Abstract

The present invention provides a method for detecting the surface velocity of flotation foam by NSST-domain infrared target segmentation and improved SURF matching. First, the NSST decomposition is performed on two adjacent frames of foam infrared images, and the boundary, brightness, and saliency constraint terms of graph cut are constructed in the multi-scale domain to achieve the segmentation of merged and broken bubbles. Then, SURF feature point detection and localization are performed on the segmented background region. The main direction of the feature points is determined by statistically calculating the scale correlation coefficient within the fan-shaped region, and the feature descriptor is constructed by using the multi-directional high-frequency coefficients in the neighborhood of the feature points. Finally, the feature points of two adjacent frames of foam infrared images are matched, and the magnitude, direction, acceleration, and disorder degree of the foam velocity are calculated according to the matching results. The application of this technical solution can reduce the influence of noise and improve the segmentation accuracy. The improvement of the feature direction determination and feature point description method of SURF in the NSST domain not only greatly improves the operation efficiency and matching accuracy, but also enhances the overall robustness of the algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method for detecting the flow rate of flotation foam by NSST-domain infrared target segmentation and SURF matching. Background Art

[0002] Flotation is a beneficiation method in which minerals in a flotation machine collide and adhere to microbubbles in the air. By utilizing the hydrophilic and hydrophobic properties of the surfaces of minerals and impurities, mineral particles with high floatability float to the liquid surface foam layer along with the bubbles, so as to separate the target minerals from ores with complex material compositions. The flotation production process is affected by various physical and chemical factors. Researchers have found that the flow characteristics on the foam surface are closely related to the flotation production process indicators. The fluidity of the bubbles reflects whether the adopted beneficiation process is of high quality and also effectively reflects the mineral content. The accurate extraction of the foam flow rate can objectively describe the fluidity of flotation bubbles and is of great significance for predicting flotation production indicators and improving the process.

[0003] In recent years, several methods for extracting foam flow characteristics have emerged. One method combines bubble highlight segmentation and phase correlation method to estimate the flow velocity of bubbles. However, the segmentation accuracy of this method is greatly affected by bubble deformation and illumination, and the relative displacement of two frames of images obtained by the phase correlation method calculation does not have globality. Another method uses the adjacent domain matching block search method to achieve foam flow velocity detection. This method has high operating efficiency, but is easily affected by external illumination factors and has low matching accuracy. Another method uses pixel point tracking technology to calculate the average flow velocity between consecutive frames. This method has high operating efficiency, but the pixel values are prone to change due to noise and illumination during the flow process, resulting in errors in pixel point tracking. Another method applies the SIFT (Scale Invariant Feature Transform) algorithm to the detection of foam flow velocity, calculates the flow velocity magnitude and direction according to the matching results, has high matching accuracy, and is less affected by noise and illumination, but SIFT is computationally complex and has poor real-time performance. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method for detecting the flow rate of flotation foam by NSST-domain infrared target segmentation and SURF matching, so as to reduce the influence of noise and improve the segmentation accuracy. By improving the feature direction determination and feature point description methods of SURF in the NSST domain, not only the operation efficiency and matching accuracy are greatly improved, but also the overall robustness of the algorithm is enhanced.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions: A method for detecting the flow rate of flotation foam by NSST-domain infrared target segmentation and SURF matching, including the following steps:

[0006] Step1: Extract two consecutive foam infrared images I with a time interval of Δtt and I t+1 , for I t and I t+1 Perform NSST multi-scale decomposition on them to obtain 1 low-pass sub-band image and k scale high-frequency sub-bands respectively. Each scale high-frequency sub-band is further decomposed into l directional sub-bands;

[0007] Step2: Construct a brightness constraint term for the low-frequency images of I t and I t+1 At the same time, perform fractional-order differential saliency detection on the low-frequency images, and then construct a saliency constraint term;

[0008] Step3: For the k scale high-frequency sub-bands of I t and I t+1 Remove noise according to the scale correlation of the coefficients, and then construct a boundary constraint term;

[0009] Step4: Construct a graph cut energy function that includes a brightness constraint term, a saliency constraint term, and a boundary constraint term, and then use the maximum flow / minimum cut algorithm to solve the minimum value of the energy function to achieve the segmentation of merged and broken bubbles;

[0010] Step5: Perform SURF scale-space extreme value extraction and feature point detection on the segmented background regions of I t and I t+1 ;

[0011] Step6: Calculate the sum of the scale correlation coefficients Corr(i,j) of all points within the sector with the position of the feature point as the center in the multi-scale high-frequency sub-bands, and then rotate the sector to traverse the entire circular region. Take the direction of the sector with the maximum sum of Corr(i,j) as the main direction of the feature point;

[0012] Step7: After rotating the coordinate axes to the main direction with the feature point as the center, divide the surrounding neighborhood into 16 sub-regions, calculate the sum C l (i,j) of the scale coefficients in 8 directions of each point, and then count the sum of the coefficients in 8 directions of each sub-region. Take the sum of the coefficients in each direction as the feature vector to obtain a 128-dimensional feature descriptor;

[0013] Step8: Perform feature point matching on the segmented background regions of I t and I t+1 , and use the RANSAC algorithm to remove mis-matched points;

[0014] Step9: For the N pairs of matching points of I t and I t+1 , their positions in I t and I t+1 are respectively and Calculate the horizontal flow velocity at this point through Equation (16). Vertical flow velocity Calculate the average horizontal flow velocity v x and the average vertical flow velocity v y ;

[0015] Step 10: Calculate the current average flow velocity and direction of the foam. Assume that the horizontal flow velocity detected at the previous moment is v x ', and the vertical flow velocity is v y '. Calculate the current horizontal flow acceleration a x and the vertical flow acceleration a y , and calculate the disorder degree of the flow velocity and direction, and the direction are the average flow velocity and direction in the previous period of time.

[0016] In a preferred embodiment, after the image is decomposed by the k-level non-sub-sampled pyramid NSP multi-scale, k + 1 sub-band images are obtained, including 1 low-frequency image and k high-frequency images with different scales. The high-frequency images are decomposed in the l-level multi-direction, and decomposed into 2l + 2 directional sub-band images; the noise of the foam low-frequency image is removed, and the bubble contour information is retained; the high-frequency sub-band images contain bubble edges, texture features, gradient information, and noise coefficients, providing boundary reference information for infrared target segmentation, and replacing the Haar wavelet response to establish a multi-scale and multi-directional feature description for SURF.

[0017] In a preferred embodiment, assume represents the coefficient at the (i, j) point in the high-frequency sub-band in the k-th scale and the l-th direction, represents the sub-band coefficient energy in the k-th scale and the l-th direction. Define the scale correlation coefficient of the pixel point (i, j) on the high-frequency sub-band in the k-th scale and the l-th direction as:

[0018]

[0019] where, represents the product of the coefficients at the (i, j) position in different scales, represents the coefficient energy of the sub-band in the k-th scale and the l-th direction, is the normalization process for facilitating coefficient comparison; after the foam image is decomposed by NSST, as the scale becomes finer and finer, the noise coefficient decays rapidly, and the edge coefficient is relatively stable, that is, the edge coefficient is strongly correlated, while the noise coefficient is weakly correlated. According to this feature, the noise coefficient will be removed:

[0020]

[0021] In a preferred embodiment, before detecting SURF feature points, the merged and broken bubbles are segmented first; infrared thermal imaging is performed on the foam on the surface of the flotation cell; when bubbles are generated, broken or merged, heat is released, and after thermal imaging, a highlighted yellow area appears;

[0022] After the NSST decomposition of the image, the merged and broken bubbles correspond to the significant regions in the low-frequency image, and the edge and gradient information contained in the high-frequency sub-band images provides boundary reference information for infrared target segmentation; first, an energy function containing a region term and a boundary term is constructed to establish a graph cut model:

[0023]

[0024] In the above formula, A is the set of pixel points of the image, is the region term of the graph cut energy function, F I (f a ) is the brightness constraint term, F s (v′ i ) is the saliency constraint term, α and β are the weight coefficients of the constraint terms, and α + β = 1, is the boundary constraint term. Finally, the minimum value of the energy function is solved using the maximum flow / minimum cut algorithm to obtain the segmentation result; the construction of the three constraint terms is as follows:

[0025] (1) Construction of the low-frequency image brightness constraint term

[0026] Assume that I is the brightness of the low-frequency image, and the brightness range is [I L , I H , and I a is the brightness value of pixel point a. The constructed brightness constraint term F I (f a ) is:

[0027]

[0028]

[0029]

[0030] In the above formula, f I (I) is the brightness function based on Gaussian fitting, and k is the contrast adjustment factor between the foreground target region and the background region;

[0031] (2) Low-frequency image saliency detection and construction of the constraint term

[0032] After the NSST decomposition of the image, the significant regions corresponding to the merged and broken bubbles in the low-frequency image are merged, and significant detection and constraint term construction are performed on the low-frequency image. For the significant detection of the low-frequency image, a fractional-order differential significant detection method is adopted, and the significant value calculation formula is:

[0033] S(x,y) = ||I u -I fd (x,y)|| (7)

[0034] In the above formula, I u is the mean value of the L, A, and B channels of the input image I, and I fd (x,y) is the mean value of the image in the L, A, and B channels after the input image I is enhanced by fractional-order differentiation. ||·|| is the Euclidean distance between the mean values of the three channels and the filtered image and then summed. According to the significant detection results, a significant constraint term is constructed, and the constraint term F s (v′ i ) is established as follows:

[0035]

[0036]

[0037]

[0038]

[0039] In the above formula: S′ i is the average significant value corresponding to each significant block, n is the number of significant region blocks, S′ is the overall average significant value, S′ Fi is the significant mean value of the foreground, and S′ Bi is the significant mean value of the background;

[0040] (3) Construction of high-frequency subband boundary constraint term

[0041] The edge and gradient information contained in the high-frequency subband image provides boundary reference information for graph cut. The boundary term B(f a , f b ) in the graph cut energy function is constructed as follows:

[0042]

[0043]

[0044] In the above formula, C a , C b respectively represent the average coefficient values of the K high-frequency scales at pixel point a and pixel point b. d(a, b) is the Euclidean distance between a and b, and C A is the total number of pixel points of image A.

[0045] In a preferred embodiment, in the high-frequency sub-band, high-frequency sub-band coefficients and coefficient correlation are used to replace the Harr wavelet response to improve the feature direction determination and feature point description method:

[0046] 1) Feature direction determination

[0047] After determining the feature point position, within a neighborhood with the position of the feature point as the center and a radius of 6s, where s is the scale value corresponding to the feature point, in the multi-scale high-frequency sub-band, calculate the sum of the scale correlation coefficients Corr(i,j) of all points within a 60° sector. The calculation of Corr(i,j) for each point is as follows:

[0048]

[0049] In the above formula, K is the number of decomposition scales, and L is the number of decomposition directions; then rotate the sector to traverse the entire circular area, and take the direction of the sector with the maximum sum of the scale correlation coefficients Corr(i,j) as the direction of the feature point;

[0050] 2) Feature descriptor generation

[0051] After rotating the coordinate axes to the main direction with the feature point as the center, select a square area with a side length of 20s and divide it into 16 sub-regions, and calculate the sum of the scale coefficients in 8 directions for each point: C 0 (i,j), C 1 (i,j), …, C 8 (i,j):

[0052]

[0053] Then, count the sum of the coefficients in 8 directions for each sub-region. The arrow direction represents the sum of the scale coefficients in 8 directions. Its feature descriptor uses a 4×4×8-dimensional coefficient description vector to obtain a 128-dimensional feature descriptor.

[0054] In a preferred embodiment, in Step 9, for the N pairs of matching points of I t and I t+1 , their positions in I t and I t+1 are respectively and Calculate the horizontal flow velocity of this point through Equation (16) Vertical flow velocity Calculate the average horizontal flow velocity v x and the average vertical flow velocity v y ;

[0055]

[0056]

[0057] In a preferred embodiment, in Step 10, the current average foam flow velocity and direction are calculated by Equation (18). Assume that the horizontal flow velocity detected at the previous moment is v x ', and the vertical flow velocity is v y '. The current horizontal flow acceleration a x and vertical flow acceleration a y are calculated by Equation (19), and the disorder degree of the flow velocity and direction is calculated by Equation (20), and direction are the average flow velocity and direction for the previous period of time;

[0058]

[0059]

[0060]

[0061] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a method for detecting the surface flow velocity of foam by infrared targets in the NSST domain and improved SURF matching. Multi-scale high-frequency sub-band denoising is performed in the NSST domain to reduce the influence of noise. Then, a graph cut energy function including boundary, brightness, and saliency constraint terms is constructed, which is beneficial to accurately estimating the target area and background of the foam infrared image, reducing the phenomena of over-segmentation and under-segmentation, and improving the segmentation accuracy; SURF feature point detection is performed on the segmented background area, the main direction of the feature points is determined by statistically calculating the scale correlation coefficient within the fan-shaped area, and a feature descriptor is constructed by using the multi-directional high-frequency coefficients in the neighborhood of the feature points, improving the anti-noise and scale transformation performance of the SURF algorithm and enhancing the matching accuracy. The present invention realizes the quantitative description of the flow characteristics on the foam surface. The flow velocity detection is less affected by bubble coalescence and fragmentation, and the detection accuracy and operation efficiency are greatly improved compared with the existing methods, laying a foundation for the subsequent optimization control of the drug addition amount. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the multi-scale transformation of the bubble image according to the preferred embodiment of the present invention;

[0063] Figure 2 It is a schematic diagram of the infrared thermal imaging of the flotation foam according to the preferred embodiment of the present invention;

[0064] Figure 3 It is a schematic diagram for determining the main direction of the features according to the preferred embodiment of the present invention;

[0065] Figure 4Schematic diagram of constructing feature descriptors for the preferred embodiment of the present invention;

[0066] Figure 5 Flowchart of foam flow velocity detection for the preferred embodiment of the present invention;

[0067] Figure 6 Schematic diagram of target segmentation results and comparison for the preferred embodiment of the present invention;

[0068] Figure 7 Schematic diagram of matching results and comparison of improved SURF for the preferred embodiment of the present invention;

[0069] Figure 8 Schematic diagram of flow velocity detection effect and comparison for the preferred embodiment of the present invention. Detailed implementation manners

[0070] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0071] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0072] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0073] A method for detecting the surface flow velocity of flotation foam by NSST-domain infrared target segmentation and improved SURF matching. First, decompose the adjacent two-frame foam infrared images by NSST, and construct the boundary, brightness, and saliency constraint terms of graph cut in the multi-scale domain to achieve the segmentation of merged and broken bubbles; then, detect and locate SURF feature points in the segmented background region, determine the main direction of the feature points by statistically calculating the scale correlation coefficient in the fan-shaped region, and construct feature descriptors using the multi-directional high-frequency coefficients in the neighborhood of the feature points; finally, match the feature points of the adjacent two-frame foam infrared images, and calculate the magnitude, direction, acceleration, and disorder degree of the foam flow velocity according to the matching results.

[0074] The detailed technical solutions are as follows:

[0075] NSST decomposition and denoising of foam infrared images

[0076] The non-subsampled Shearlet transform includes two parts: multi-scale decomposition and multi-directional decomposition. AsFigure 1 As shown, the synthetic and broken bubbles in the foam infrared image correspond to the high-temperature yellow areas in the figure. After the image is decomposed by the k-level non-subsampled pyramid (NSP) multi-scale decomposition, k + 1 sub-band images are obtained, including 1 low-frequency image and k high-frequency images with different scales. The high-frequency images are decomposed in the l-level multi-direction, and decomposed into 2l + 2 directional sub-band images. The foam low-frequency image removes noise, retains the bubble contour information, improves the saliency of the high-temperature area, and is conducive to locating the synthetic and broken bubbles. The high-frequency sub-band image contains bubble edges, texture features, gradient information, and noise coefficients, provides boundary reference information for infrared target segmentation, and can replace the Haar wavelet response to establish a multi-scale and multi-directional feature description for SURF.

[0077] To avoid the influence of noise on subsequent processing, noise is removed from the multi-scale high-frequency sub-bands. Assume represents the coefficient of the high-frequency sub-band in the k-th scale and l-th direction at the (i, j) point, represents the sub-band coefficient energy of the k-th scale and l-th direction, and defines the scale correlation coefficient of the pixel point (i, j) on the high-frequency sub-band in the k-th scale and l-th direction as:

[0078]

[0079] where, represents the product of the coefficients at the (i, j) position in different scales, represents the coefficient energy of the k-th scale and l-th direction sub-band, is the normalization process for facilitating coefficient comparison. After the foam image is decomposed by NSST, as the scale becomes finer and finer, the noise coefficient decays rapidly, and the edge coefficient is relatively stable, that is, the edge coefficient is strongly correlated, while the noise coefficient is weakly correlated. According to this characteristic, the noise coefficient will be eliminated:

[0080]

[0081] NSST Domain Foam Infrared Target Segmentation

[0082] To reduce the influence of merged and broken bubbles on the foam flow rate detection, the merged and broken bubbles are segmented before the SURF feature point detection. Infrared thermal imaging is performed on the foam on the surface of the flotation cell. As Figure 2 shown, it has a certain display effect on the broken and merged bubbles. After the bubbles are broken or merged, heat is released, and a bright yellow area appears after thermal imaging. Figure 2 (a) There are two bubbles collapsing, Figure 2 (b) Three small bubbles merge into one large bubble. Figure 2(c) In the visible light image of the foam, it is impossible to directly distinguish between broken and merged bubbles, while the corresponding infrared thermal imaging Figure 2 (d) directly shows the generated broken and merged bubbles. Therefore, the extraction of merged and broken bubbles can be achieved by target segmentation of the high-temperature regions in the infrared image.

[0083] After the NSST decomposition of the image, the merged and broken bubbles correspond to the significant regions in the low-frequency image, and the edge and gradient information contained in the high-frequency sub-band images provides boundary reference information for infrared target segmentation. Based on the graph cut algorithm, the present invention constructs boundary, brightness, and saliency constraint terms of the graph cut in the NSST multi-scale domain to achieve the segmentation of merged and broken bubbles. First, an energy function containing a region term and a boundary term is constructed to establish a graph cut model:

[0084]

[0085] In the above formula, A is the set of pixel points of the image, is the region term of the graph cut energy function, F I (f a ) is the brightness constraint term, F s (v i ′) is the saliency constraint term, α and β are the weight coefficients of the constraint terms, and α + β = 1, is the boundary constraint term. Finally, the minimum value of the energy function is solved using the maximum flow / minimum cut algorithm to obtain the segmentation result. The construction of the three constraint terms is as follows:

[0086] (1) Construction of the brightness constraint term for the low-frequency image

[0087] Assume that I is the brightness of the low-frequency image, and the brightness range is [I L , I H , and I a is the brightness value of pixel point a. The constructed brightness constraint term F I (f a ) is:

[0088]

[0089]

[0090]

[0091] In the above formula, f I (I) is the brightness function based on Gaussian fitting, and k is the contrast adjustment factor between the foreground target region and the background region.

[0092] (2) Low-frequency image saliency detection and construction of the constraint term

[0093] After the NSST decomposition of the image, the significant regions corresponding to the merged and broken bubbles in the low-frequency image are merged, and significant detection and constraint term construction are performed on the low-frequency image. For the significant detection of the low-frequency image, the fractional-order differential significant detection method is adopted, and the significant value calculation formula is:

[0094] S(x,y) = ||I u -I fd (x,y)|| (7)

[0095] In the above formula, I u is the mean value of the L, A, and B channels of the input image I, and I fd (x,y) is the mean value of the images of the L, A, and B channels after the fractional-order differential enhancement of the input image I. ||·|| is to take the Euclidean distance of the mean values of the three channels and the filtered image and sum them. According to the significant detection results, a significant constraint term is constructed, and the constraint term F s (v′ i ) is established as follows:

[0096]

[0097]

[0098]

[0099]

[0100] In the above formula: S′ i is the average significant value corresponding to each significant block, n is the number of significant region blocks, S′ is the overall average significant value, S′ Fi is the significant mean value of the foreground, and S′ Bi is the significant mean value of the background.

[0101] (3) Construction of the high-frequency subband boundary constraint term

[0102] The edge and gradient information contained in the high-frequency subband image provides boundary reference information for graph cut, and the boundary term B(f a , f b ) in the graph cut energy function is constructed as follows:

[0103]

[0104]

[0105] In the above formula, C a , C b respectively represent the average values of the coefficients of the K high-frequency scales at pixel point a and pixel point b, d(a, b) is the Euclidean distance between a and b, and C A is the total number of pixel points of image A.

[0106] Improvement of SURF Feature Direction and Description in NSST Domain

[0107] The traditional SURF algorithm mainly consists of five steps: scale-space extreme value extraction, feature point localization, determination of feature direction, feature point description, and feature point matching. The traditional SURF algorithm is a method for extracting feature points in grayscale images to find matching pairs, which only considers the luminance information of the image and ignores a large amount of edge information and texture details contained in the image. The extracted feature description information is incomplete and prone to false matching. The present invention improves the SURF algorithm and uses high-frequency subband coefficients and coefficient correlation in the high-frequency subband to replace the Harr wavelet response to improve the feature direction determination and feature point description methods:

[0108] (1) Feature direction determination

[0109] After determining the position of the feature point, in the multi-scale high-frequency subband, within a neighborhood with the position of the feature point as the center and a radius of 6s (s is the scale value corresponding to this feature point), as Figure 3 shown, calculate the sum of the scale correlation coefficients Corr(i,j) of all points within the 60° sector, where the Corr(i,j) of each point is calculated as:

[0110]

[0111] In the above formula, K is the number of scales of decomposition, and L is the number of decomposition directions. Then rotate the sector to traverse the entire circular area, and take the direction of the sector with the maximum sum of the scale correlation coefficients Corr(i,j) as the direction of this feature point.

[0112] (2) Feature descriptor generation

[0113] After rotating the coordinate axes to the main direction with the feature point as the center, as Figure 4 shown, select a square area with a side length of 20s and divide it into 16 sub-regions, and calculate the sum of the scale coefficients in 8 directions of each point: C 0 (i,j), C 1 (i,j), …, C 8 (i,j):

[0114]

[0115] Then count the sum of the coefficients in 8 directions of each sub-region. The arrow direction represents the sum of the scale coefficients in 8 directions, and its feature descriptor can be described by a 4×4×8-dimensional coefficient description vector to obtain a 128-dimensional feature descriptor.

[0116] Specific implementation process and steps

[0117] In summary, the process of the foam flow velocity detection method based on NSST-domain infrared target segmentation and SURF matching is as follows: Figure 5 as shown below, and the specific implementation steps are as follows:

[0118] Step1: Extract two consecutive foam infrared images \(I\) t and \(I\) t+1 with a time interval of \(\Delta t\). Perform NSST multi-scale decomposition on \(I\) t and \(I\) t+1 to obtain 1 low-pass sub-band image and \(k\) scale high-frequency sub-bands respectively. Each scale high-frequency sub-band is further decomposed into \(l\) direction sub-bands;

[0119] Step2: Construct a brightness constraint term for the low-frequency images of \(I\) t and \(I\) t+1 through equations (4)-(6). At the same time, perform fractional-order differential saliency detection on the low-frequency images, and then construct a saliency constraint term through equations (8)-(11);

[0120] Step3: For the \(k\) scale high-frequency sub-bands of \(I\) t and \(I\) t+1 , remove noise according to the scale correlation of the coefficients, and then construct a boundary constraint term through equations (12)-(13);

[0121] Step4: Construct a graph cut energy function containing the brightness constraint term, saliency constraint term, and boundary constraint term, and then use the maximum flow / minimum cut algorithm to solve the minimum value of the energy function to achieve the segmentation of merged and broken bubbles;

[0122] Step5: Perform SURF scale-space extreme value extraction and feature point detection on the segmented background regions of \(I\) t and \(I\) t+1 ;

[0123] Step6: With the position of the feature point as the center in the multi-scale high-frequency sub-band, calculate the sum of the scale correlation coefficients \(Corr(i,j)\) of all points within the fan through equation (14), and then rotate the fan to traverse the entire circular region. Take the direction of the fan with the maximum sum of \(Corr(i,j)\) as the main direction of the feature point;

[0124] Step7: After rotating the coordinate axis to the main direction with the feature point as the center, divide the surrounding neighborhood into 16 sub-regions, calculate the sum \(C\) l (i,j) of the scale coefficients in 8 directions of each point through equation (15), and then statistically calculate the sum of the coefficients in 8 directions of each sub-region. Take the sum of the coefficients in each direction as the feature vector to obtain a 128-dimensional feature descriptor.

[0125] Step8: For \(I\) t and \(I\)t+1 Feature point matching is performed on the segmented background region, and the RANSAC algorithm is used to eliminate the mismatched points.

[0126] Step9: For the N pairs of matching points of I t and I t+1 Their positions in I t and I t+1 are respectively and Calculate the horizontal flow velocity of this point through Equation (16) Vertical flow velocity Calculate the average horizontal flow velocity v x and the average vertical flow velocity v y .

[0127]

[0128]

[0129] Step10: Calculate the current average foam flow velocity and direction through Equation (18). Assume that the horizontal flow velocity detected at the previous moment is v x ', and the vertical flow velocity is v y '. Calculate the current horizontal flow acceleration a x and the vertical flow acceleration a y , calculate the disorder degree of the flow velocity and direction through Equation (20), and the direction are the average flow velocity and direction for the previous period of time.

[0130]

[0131]

[0132]

[0133] Specific embodiments and descriptions

[0134] Foam infrared target segmentation effect

[0135] To verify the segmentation effect of broken and merged bubbles, the segmentation results of the present invention are compared with the results of classical segmentation methods to verify its accuracy. For Figure 6 (a) The foam infrared image is subjected to NSST decomposition and target segmentation, Figure 6 (b)~ Figure 6 (c) are the decomposed low-frequency image and high-frequency subband images, Figure 6 (d) is the fractional-order differential saliency detection result of the low-frequency image. Figure 6(e) shows the bubble extraction effect of the region growing method, with problems of holes in the target region and blurred edges; Figure 6 (f) shows the bubble extraction effect of the edge segmentation algorithm, with problems of cluttered edges and under-segmentation; Figure 6 (g) shows the segmentation result of the K-clustering algorithm. The edge of the segmented target region is clear, but the accuracy is not high; Figure 6 (h) shows the infrared target segmentation result of the present invention. By constructing a brightness constraint term from the low-frequency image, a saliency constraint term is constructed according to the fractional-order differential saliency detection result of the low-frequency image, and a boundary constraint term is constructed by the coefficient correlation of the high-frequency subbands. Then, the minimum value of the energy function is solved using the maximum flow / minimum cut algorithm to achieve image segmentation, and the obtained result has a high degree of coincidence with the actual target.

[0136] Improve the matching effect of the SURF algorithm

[0137] To verify the matching effect of the improved SURF algorithm of the present invention, feature point detection and matching experiments are carried out on two adjacent frames of foam infrared images I t and I t+1 , and the matching results are compared with those of the improved SIFT algorithm and the improved SURF algorithm. The experimental process and results are as Figure 7 shown. Figure 7 (c) and (d) are respectively the feature point detection results of I t and I t+1 . Figure 7 (g) is the corresponding feature point matching result, with the largest number of matching point pairs and uniform distribution, and no false matching points appear; Figure 7 (e) is the feature point matching result of the improved SIFT, with many matching points and relatively uniform distribution; Figure 7 (f) is the feature point matching result of the improved SURF, with fewer matching point pairs and clustering, and non-uniform distribution of matching points. To further objectively verify the performance of the matching algorithm, matching experiments are respectively carried out on 100 pairs of Gaussian white noise images with a superimposed mean of 0 and different variances and 100 pairs of images with different scale ratios. The average matching accuracy and running time are shown in Table 1. When the noise variance is 10% and the scale ratio is 1:2: The improved SIFT has half less operation time than the original SIFT, has better anti-noise and scale transformation performance, and higher average matching accuracy, but the operation efficiency needs to be further improved; the anti-noise and scale transformation performance of the improved SURF algorithm is greatly improved, and it maintains a high operation efficiency, but the matching points are clustered and non-uniformly distributed; the anti-noise and scale transformation capabilities of the improved SURF algorithm of the present invention are greatly improved, with the best anti-noise performance, and the anti-scale transformation performance is equivalent to that of the improved SIFT, but the operation efficiency is higher than that of the improved SIFT algorithm. When the noise variance increases to 30% and the scale ratio increases to 1:8: The matching performance of all three algorithms drops significantly, but the present invention maintains a matching accuracy higher than 85%, with better anti-noise and scale transformation capabilities.

[0138] Table 1 Comparison of Matching Accuracy and Running Time

[0139]

[0140] Flow Velocity Detection Performance and Comparison

[0141] To verify the flow velocity detection effect of the present invention, feature point matching and flow velocity detection experiments were carried out on two adjacent frames of foam infrared images I t and I t+1 and comparative analysis was carried out with existing methods. The experimental results are as Figure 8 shown: The matching results and velocity vector diagrams obtained by using the adjacent domain matching block search method to extract and calculate the flow characteristics of the foam are as Figure 8 (e) shows. The arrowed line segments on the velocity vector diagram represent the velocity vectors of each feature point. This method has a fast matching speed, but is easily affected by external light factors, resulting in false matching and low flow velocity detection accuracy; The pixel tracking results and velocity vector diagrams of the pixel point tracking technology method are as Figure 8 (f) shows. The error rate of the matching results is relatively low, but the number of matching point pairs is small and the distribution of feature points is uneven; The matching results and velocity vector diagrams of the SIFT matching method are as Figure 8 (g) shows. The detection range is wide and evenly distributed, and there are many matching point pairs, but the SIFT operation efficiency is low; The matching results and velocity vector diagrams of the improved ORB matching method are as Figure 8 (h) shows. There are many matching points and they are evenly distributed, but affected by the fragmentation and merging of bubbles, there are certain deviations in the velocity magnitude and direction in the fragmentation and merging regions, affecting the overall flow velocity detection accuracy; The experimental results of the present invention are as Figure 8 (i) shows. The number of matching point pairs is large and the distribution is relatively even. Feature point detection and matching are carried out on the background region of the segmented infrared target, effectively removing the influence of bubble fragmentation and merging. The extracted flow velocity is the magnitude and direction of the overall velocity of the foam, and the detection accuracy is high.

[0142] To further objectively verify the flow velocity detection accuracy of the method of the present invention, two frames of 256×256 images I with an actual displacement of (15, 15) were selected t and I t+1 to carry out simulation experiments, and the magnitude V, direction θ v of the foam flow velocity, as well as the relative errors E v and E θ and the running time were calculated, and comparative analysis was carried out with existing methods. The statistical results of the data are shown in Table 2: The E v and E θIt is smaller and has higher detection accuracy. However, the SIFT matching method has low operation efficiency, while the improved ORB matching method is interfered by bubble fragmentation and merging in practical applications, affecting the overall detection accuracy. The present invention can effectively remove the influence of bubble fragmentation and merging and further improve the detection accuracy.

[0143] Table 2 Flow velocity detection results and comparison

[0144]

Claims

1. A method for detecting the flow rate of flotation foam by infrared target segmentation in the NSST domain and SURF matching, characterized in that, It includes the following steps: Step1: Extract two consecutive foam infrared images I with a time interval of Δt t and I t+1 , perform NSST multi-scale decomposition on I t and I t+1 , respectively obtain 1 low-pass sub-band image and k scale high-frequency sub-bands, and each scale high-frequency sub-band is further decomposed into l directional sub-bands; Step2: For I t and I t+1 construct a luminance constraint term for the low-frequency images, and at the same time perform fractional-order differential saliency detection on the low-frequency images, and then construct a saliency constraint term; Step3: For the k-scale high-frequency sub-bands of I t and I t+1 , noise removal is performed according to the scale correlation of the coefficients, and then a boundary constraint term is constructed; Step4: Construct a graph cut energy function that includes a brightness constraint term, a saliency constraint term, and a boundary constraint term, and then use the maximum flow / minimum cut algorithm to solve the minimum value of the energy function to achieve the segmentation of merged and broken bubbles; Step5: Perform SURF scale-space extreme value extraction and feature point detection on the background regions after segmenting I t and I t+1 ; Step6: With the position of the feature point as the center in the multi-scale high-frequency subbands, calculate the sum of the scale correlation coefficients Corr(i,j) of all points within the sector, and then rotate the sector to traverse the entire circular area. Take the direction of the sector with the maximum sum of Corr(i,j) as the main direction of the feature point; Step 7: After rotating the coordinate axes to the main direction with the feature point as the center, divide the surrounding area into 16 sub-regions, and calculate the sum C of the scale coefficients in 8 directions for each point (i, j). Then, count the sum of the coefficients in 8 directions for each sub-region, use the sum of the coefficients in each direction as the feature vector, and obtain a 128-dimensional feature descriptor. l (i,j), and then count the sum of the coefficients in 8 directions for each sub-region, use the sum of the coefficients in each direction as the feature vector, and obtain a 128-dimensional feature descriptor. Step8: Perform feature point matching on the background regions after splitting I t and I t+1 , and use the RANSAC algorithm to eliminate the mismatched points; Step9: For I t and I t+1 of N pairs of matching points, their positions in I t and I t+1 are respectively and By calculating the horizontal flow velocity of this point and the vertical flow velocity Calculate the average horizontal flow velocity v x and the average vertical flow velocity v y ; Step 10: Calculate the current average foam flow velocity and direction. Assume that the horizontal flow velocity detected at the previous moment is v x ', and the vertical flow velocity is v y '. Calculate the current horizontal flow acceleration a x and the vertical flow acceleration a y , and calculate the disorder degree of the flow velocity and direction.

2. The flotation foam flow velocity detection method for NSST domain infrared target segmentation and SURF matching according to claim 1, characterized in that After the image is multi-scale decomposed by the k-level non-subsampled pyramid NSP, k + 1 subband images are obtained, including 1 low-frequency image and k high-frequency images with different scales. The high-frequency images are decomposed in l levels in multiple directions into 2l + 2 directional subband images; the noise is removed from the foam low-frequency image, and the bubble contour information is retained; the high-frequency subband images contain bubble edges, texture features, gradient information, and noise coefficients, providing boundary reference information for infrared target segmentation, and replacing the Haar wavelet response to establish a multi-scale and multi-directional feature description for SURF.

3. The flotation foam flow velocity detection method for NSST domain infrared target segmentation and SURF matching according to claim 2, characterized in that, Hypothesis denotes the coefficient of the high-frequency sub-band in the \(l\)-th direction at the \(k\)-th scale at the point \((i, j)\). denotes the energy of the sub-band coefficients in the \(l\)-th direction at the \(k\)-th scale. Define the scale-dependent coefficient of the pixel point \((i, j)\) on the high-frequency sub-band in the \(l\)-th direction at the \(k\)-th scale as follows: Among them, represents the product of coefficients at the (i, j) position for different scales, represents the coefficient energy of the sub-band in the l-th direction at the k-th scale, is a normalization process for facilitating coefficient comparison; after the foam image is decomposed by NSST, as the scale becomes finer and finer, the noise coefficients decay rapidly, and the edge coefficients are relatively stable, that is, the edge coefficients are strongly correlated while the noise coefficients are weakly correlated. According to this feature, the noise coefficients will be removed:

4. The method for detecting the floatation foam flow rate by NSST domain infrared target segmentation and SURF matching according to claim 1, wherein Before detecting SURF feature points, first segment the merged and broken bubbles; perform infrared thermal imaging on the foam on the surface of the flotation cell; when bubbles break or merge, heat is released, and a bright yellow area appears after thermal imaging; After the image is decomposed by NSST, the merged and broken bubbles correspond to the salient regions in the low-frequency image, and the edge and gradient information contained in the high-frequency subband images provide boundary reference information for infrared target segmentation; first, construct an energy function that includes a region term and a boundary term to establish a graph cut model: In the above formula, A is the set of pixel points of the image, is the regional term of the graph cut energy function, F I (f a ) is the brightness constraint term, F s (v′ i ) is the saliency constraint term, α and β are the weight coefficients of the constraint terms, and α + β = 1, is the boundary constraint term. Finally, the minimum value of the energy function is solved using the maximum flow / minimum cut algorithm to obtain the segmentation result; the construction of the three constraint terms is as follows: (1) Construction of the brightness constraint term for the low-frequency image Assume that I is the luminance of the low-frequency image, and the luminance range is [I L , I H , where I a is the luminance value of pixel point a. Construct the luminance constraint term F I (f a ) as follows: In the above formula, f I (I) is the luminance function based on Gaussian fitting, and k is the contrast adjustment factor between the foreground target region and the background region; (2) Saliency detection and construction of the constraint term for the low-frequency image After the image is decomposed by NSST, the merged and broken bubbles correspond to the salient regions in the low-frequency image. Perform saliency detection and construction of the constraint term on the low-frequency image; for the saliency detection of the low-frequency image, use the fractional-order differential saliency detection method, and the formula for the saliency value is: S(x,y) = ||I u -I fd (x,y)|| (7) In the above formula, I u is the mean value of the L, A, and B channels of the input image I. I fd (x, y) is the mean value of the images in the L, A, and B channels after the input image I is enhanced by fractional-order differentiation. ‖‖ takes the Euclidean distance of the mean values of the three channels and the filtered image and sums them up. A saliency constraint term is constructed according to the saliency detection result, and the constraint term F s (v′ i ) is established: In the above formula: S' i is the average saliency value corresponding to each significant block, n is the number of significant region blocks, S' is the overall average saliency value, S' Fi is the saliency mean of the foreground, S' Bi is the saliency mean of the background; (3) Construction of the boundary constraint term for the high-frequency subbands The edge and gradient information contained in the high-frequency subband image provides boundary reference information for graph cuts, and constructs the boundary term B(f a , f b ) in the graph cut energy function: In the above formula, C a , C b respectively represent the average coefficients of the K high-frequency scales at pixel a and pixel b. d(a, b) is the Euclidean distance between a and b, and C A is the total number of pixels in image A.

5. The method for detecting the flotation foam flow rate by NSST domain infrared target segmentation and SURF matching according to claim 1, wherein In the high-frequency subbands, use the high-frequency subband coefficients and coefficient correlations to replace the Harr wavelet response to improve the feature direction determination and feature point description methods: 1) Feature direction determination After determining the position of the feature point, within a neighborhood with a radius of 6s (s is the scale value corresponding to this feature point) centered at the position of the feature point in the multi-scale high-frequency subbands, calculate the sum of the scale correlation coefficients Corr(i,j) of all points within a 60° sector, where the Corr(i,j) of each point is calculated as: In the above formula, K is the number of decomposition scales, and L is the number of decomposition directions; then rotate the sector to traverse the entire circular area, and take the direction of the sector with the maximum sum of the scale correlation coefficients Corr(i,j) as the direction of this feature point; 2) Generation of the feature descriptor After rotating the coordinate axes to the main direction with the feature point as the center, a square region with a side length of 20s is selected and divided into 16 sub-regions, and the sum of the scale coefficients in 8 directions of each point is calculated: C 0 C(i,j), C 1 C(i,j), …, C 8 C(i,j): Then, count the sum of the coefficients in 8 directions of each sub-region. The arrow direction represents the sum of the scale coefficients in 8 directions, and its feature descriptor is described by a 4×4×8-dimensional coefficient vector to obtain a 128-dimensional feature descriptor.

6. The method for detecting the floatation foam flow rate by NSST domain infrared target segmentation and SURF matching according to claim 1, wherein In Step9, for the N pairs of matching points of I t and I t+1 , their positions in I t and I t+1 are respectively and Calculate the horizontal flow velocity of this point through Equation (16) Vertical flow velocity Calculate the average horizontal flow velocity v x and the average vertical flow velocity v y ; 7. The flotation foam flow velocity detection method for NSST domain infrared target segmentation and SURF matching according to claim 1, characterized in that In Step 10, the current average foam flow velocity and direction are calculated by Equation (18). Assume that the horizontal flow velocity detected at the previous moment is v x ', and the vertical flow velocity is v y '. The current horizontal flow acceleration a x and vertical flow acceleration a y are calculated by Equation (19). The disorder degree of the flow velocity and direction is calculated by Equation (20), and the direction are the average flow velocity and direction over a previous period of time;

Images