Method, device and medium for on-line measurement of froth state in mineral processing based on three-dimensional vision

Abstract

The application discloses a beneficiation froth state online measurement method and device based on three-dimensional vision and a medium, relates to the technical field of image recognition in the mining industry, and comprises the following steps: removing non-froth point clouds in froth dense three-dimensional point clouds by using a froth region segmentation mask, restraining isolated noise points by adopting adjacent frame curvature change constraints, generating a froth three-dimensional surface by grid reconstruction, and splicing a multi-dimensional froth state feature matrix according to a unified time stamp; inputting the froth three-dimensional feature array into an online inference model, dynamically identifying the beneficiation froth state and jointly regressing key parameters, obtaining a confidence degree, simultaneously utilizing an online domain self-adaptive updating strategy driven by the confidence degree to correct network feature distribution in real time, and outputting a froth state result package. The application improves the stable characterization ability of image recognition on the real space state of froth.

Description

Technical Field

[0001] This invention relates to the field of mining image recognition technology, and in particular to a method, equipment and medium for online measurement of mineral processing foam state based on three-dimensional vision. Background Technology

[0002] With the continuous development of digital mines and intelligent mineral processing technologies, image recognition-based monitoring methods for mineral processing have gradually become an important technical means in gold ore flotation production. In gold ore beneficiation, the morphology, structure, and dynamic changes of the froth layer directly reflect the properties of the slurry, the effectiveness of reagents, and the stability of flotation conditions. Therefore, online sensing and identification of the froth state has significant engineering value. In existing technologies, image recognition methods have been widely applied in the field of mineral processing froth monitoring. Typically, industrial cameras are used to continuously acquire data on the froth surface, and algorithms such as image segmentation, texture feature extraction, and statistical analysis are combined to quantitatively analyze the froth size distribution, color characteristics, or surface roughness. Some studies further introduce machine learning or deep learning models to achieve froth state classification or indirect prediction of flotation indicators. These two-dimensional image recognition-based technologies have, to a certain extent, improved the informatization and automation level of the gold ore beneficiation process, providing data support for optimizing operating parameters and process control.

[0003] From an engineering application perspective, existing mineral processing foam monitoring technologies based on two-dimensional image recognition mainly focus on the planar information of the foam surface, and still have limitations in characterizing the spatial structural features of the foam. Under complex lighting conditions, high foam reflectivity, or partial occlusion, the stability and consistency of two-dimensional image recognition results are easily affected. Especially in scenarios that require continuous online identification of foam state and regression of key parameters, the lack of effective perception of the three-dimensional geometric morphology of the foam often restricts the ability of image recognition models to express the true physical state of the foam. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an online measurement method for the state of mineral processing foam based on three-dimensional vision to solve the problem of the difficulty in accurately perceiving the three-dimensional spatial state of foam.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides an online measurement method for the state of mineral processing foam based on three-dimensional vision. The method includes: synchronously acquiring multi-view images of the foam surface and performing online image preprocessing to output a synchronized foam image package; the synchronized foam image package includes synchronized images of the foam surface, laser stripe enhanced images, and a foam region segmentation mask; extracting the stripe centerline pixel set from the laser stripe enhanced images at different viewpoints, and performing three-dimensional reconstruction of the stripe centerline pixel set according to the parameters of the multi-view sensor array to obtain a dense three-dimensional point cloud of foam; using the foam region segmentation mask to remove non-foam point clouds from the dense three-dimensional point cloud of foam, and using adjacent frame curvature change constraints to suppress isolated noise points; generating a three-dimensional foam surface through mesh reconstruction, and stitching them together according to a unified timestamp to form a multi-dimensional foam state feature matrix; inputting the three-dimensional foam feature matrix into an online inference model to dynamically identify the state of mineral processing foam and perform joint regression of key parameters to obtain confidence levels; simultaneously using a confidence-driven online domain adaptive update strategy to perform real-time correction of the network feature distribution and output a foam state result package.

[0007] As a preferred embodiment of the online measurement method for mineral processing foam state based on three-dimensional vision described in this invention, the synchronous acquisition of multi-view foam surface images refers to the synchronous acquisition of RGB color images, grayscale images, and laser stripe images of the foam surface, and assigning them a unified timestamp.

[0008] As a preferred embodiment of the online measurement method for mineral processing foam state based on three-dimensional vision described in this invention, the steps of performing online image preprocessing and outputting a synchronized foam image package are as follows: Multi-view foam surface images are processed to eliminate camera lens distortion through distortion correction, and then brightness normalization and highlight suppression are performed to output synchronized foam surface images. The edge information of the laser stripe region in the synchronous image of the foam surface is enhanced by the Laplacian operator to form a laser stripe enhanced image. Based on the typical characteristics of foam, region labeling is performed on multi-view foam surface images to separate non-foam regions from foam regions, forming a foam region segmentation mask. The synchronized images of the foam surface, the laser stripe enhanced images, and the foam region segmentation mask are organized and packaged according to the viewpoint number to form a synchronized foam image package.

[0009] As a preferred embodiment of the online measurement method for mineral processing foam state based on three-dimensional vision described in this invention, the steps include: extracting the stripe centerline pixel set from laser stripe enhanced images at different viewpoints, and performing three-dimensional reconstruction of the stripe centerline pixel set according to the multi-view sensor array parameters to obtain a dense three-dimensional point cloud of foam. The specific steps are as follows: Automatic stripe detection and extraction are performed on laser stripe enhanced images from different perspectives, outputting a set of stripe centerline pixels. The optical flow method is used to track the changes of each laser stripe in consecutive frames, dynamically adjust the stripe centerline pixel extraction accuracy, and obtain a high-quality set of stripe centerline pixels. Based on the parameters of the multi-view sensor array, stereo matching is performed on the high-quality stripe centerline pixels under different viewpoints to calculate the three-dimensional coordinates and generate a preliminary three-dimensional point cloud. Error minimization and local smoothing are performed on the initial 3D point cloud, and the position and density of the point cloud are finely adjusted to obtain the initial foam-dense 3D point cloud. The initial foam-dense 3D point cloud is then post-processed using a surface reconstruction algorithm to output the foam-dense 3D point cloud.

[0010] As a preferred embodiment of the online measurement method for mineral processing foam state based on three-dimensional vision described in this invention, the steps of using a foam region segmentation mask to remove non-foam point clouds from a dense three-dimensional point cloud and employing curvature variation constraints between adjacent frames to suppress isolated noise points are as follows: By using the binarized value of the foam region segmentation mask, the foam surface is separated from the dense 3D point cloud of foam, the foam region is extracted, and the non-foam region is removed from the dense 3D point cloud of foam. The local curvature of the point cloud in the bubble region after removal is calculated to evaluate the surface curvature. The curvature difference between adjacent point clouds is used to identify outliers and isolated noise points. Isolated noise points in the point cloud in the bubble region after removal are suppressed based on curvature change constraints.

[0011] As a preferred embodiment of the online measurement method for mineral processing foam state based on three-dimensional vision described in this invention, the specific steps of generating a three-dimensional foam surface through gridded reconstruction and stitching it together according to a unified timestamp to form a multi-dimensional foam state feature matrix are as follows. Calculate the local neighborhood of each suppressed foam-dense 3D point cloud, and use the least squares method to calculate the normal direction of the points in the local neighborhood, and output the normal field; The suppressed dense 3D point cloud of foam is triangulated using the normal field to generate a 3D foam surface. The 3D foam surfaces with multiple timestamps are time-aligned and stitched together according to a unified timestamp to generate a multi-dimensional foam state feature matrix.

[0012] As a preferred embodiment of the online measurement method for mineral processing foam state based on three-dimensional vision described in this invention, the online inference model is formed by receiving a three-dimensional foam feature matrix through an input layer, inputting the three-dimensional foam feature matrix into the main branch of a CNN to extract spatial features and the auxiliary branch of an MLP to extract auxiliary features, performing temporal modeling through a lightweight temporal layer, and combining the main branch features and the auxiliary branch features through a fusion layer.

[0013] As a preferred embodiment of the online measurement method for mineral processing foam state based on three-dimensional vision described in this invention, the method involves: inputting a three-dimensional feature matrix of foam into an online inference model to dynamically identify the mineral processing foam state and perform joint regression of key parameters to obtain confidence levels; simultaneously, utilizing a confidence-driven online domain adaptive update strategy to perform real-time correction of the network feature distribution and output a foam state result package. The specific steps are as follows: The three-dimensional feature matrix of the foam is input into the online inference model. The spatial features of the three-dimensional feature matrix of the foam are extracted through the main branch of CNN to obtain spatial geometric features. The MLP auxiliary branch is used to identify the depth information of bubble size distribution and gloss, and output surface property features. Based on spatial geometric features and surface property features, the dynamic changes of foam state are captured, and the spatial geometric features and surface property features are spliced ​​and fused using a fusion layer to form foam joint features. The foam state is dynamically identified through a joint regression method, and the foam state results and confidence scores are output. Based on a confidence-driven online adaptive update strategy, the weights and learning rate of the network feature distribution are adjusted according to the confidence level, and an incremental learning method is used to correct the feature distribution in the network in real time, outputting a bubble state result package.

[0014] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the online measurement method for mineral processing foam state based on three-dimensional vision as described in the first aspect of the present invention.

[0015] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the online measurement method for mineral processing foam state based on three-dimensional vision as described in the first aspect of the present invention.

[0016] The beneficial effects of this invention are as follows: by combining multi-view image recognition with laser stripe 3D reconstruction, the three-dimensional geometric morphology of gold ore flotation foam can be acquired online. Under semantic constraints and temporal stabilization processing, multi-dimensional state features of foam are constructed. Furthermore, an online inference model with confidence-driven adaptive capability is used to complete the dynamic identification and parameter regression of foam state, thereby improving the image recognition's ability to stably represent the real spatial state of foam. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a method for online measurement of mineral processing foam state based on 3D vision.

[0019] Figure 2 This is a flowchart for laser stripe enhancement image extraction and 3D point cloud generation.

[0020] Figure 3 The flowchart for foam region segmentation and noise removal.

[0021] Figure 4 This is a flowchart for the reconstruction of the three-dimensional surface of foam and the generation of its feature matrix.

[0022] Figure 5 This is a graph showing the change in 3D reconstruction error of foam over time.

[0023] Figure 6 This is a comparison chart of confidence level and recognition accuracy. Detailed Implementation

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0025] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0026] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0027] Reference Figures 1-6 This is one embodiment of the present invention, which provides an online measurement method for the state of mineral processing foam based on three-dimensional vision, including the following steps: S1. Simultaneously acquire multi-view foam surface images, perform online image preprocessing, and output a foam synchronous image package.

[0028] S1.1. Synchronously acquire RGB color images, grayscale images, and laser stripe images of the foam surface according to the three-dimensional vision calibration package, and assign a unified timestamp to obtain multi-view foam surface images.

[0029] Furthermore, the specific composition of the multi-view sensor array is determined, including the number and number of each RGB camera, grayscale camera, depth camera, and laser. Fixed installation is completed according to the view coverage and baseline relationship. The initial spatial position and orientation of each sensor in the device coordinate system are clarified, and a unified hardware or clock synchronization acquisition method is configured. After the array is built, a calibration target with known geometric dimensions and feature distribution is used to synchronously acquire images of the calibration target from all views. The intrinsic parameters, distortion parameters, and extrinsic parameters of each imaging sensor relative to the unified reference coordinate system are calculated using a geometric calibration algorithm. These parameters are then uniformly packaged into a 3D visual calibration package. Based on the pre-determined sensor numbers, intrinsic distortion parameters, extrinsic spatial transformation relationships, and synchronization triggering parameters in the 3D vision calibration package, the multi-view camera and the laser / depth camera are synchronously triggered from the same source, so that each viewpoint is exposed or sampled simultaneously within the same acquisition cycle. Subsequently, RGB color images, grayscale images, and laser stripe images of the foam surface are acquired and saved in each acquisition cycle, and all image data in the same acquisition cycle are written to the same unified timestamp. The images are associated one-to-one and their integrity is verified according to the fixed index rule of "timestamp-viewpoint number-modal type". Data groups with missing frames or inconsistent timestamps are removed to obtain a time-aligned multi-view foam surface image dataset.

[0030] S1.2. The multi-view foam surface images are processed to eliminate camera lens distortion through distortion correction, and brightness normalization and highlight suppression are performed to output a synchronized image of the foam surface.

[0031] Furthermore, based on the camera intrinsic parameter matrix and distortion coefficients in the 3D vision calibration package, distortion correction is performed on each frame of the multi-view foam surface image from each viewpoint, mapping radial and tangential distortion back to the ideal imaging plane and outputting a distortion-corrected image. Subsequently, the distortion-corrected image undergoes brightness normalization processing, adjusting the overall brightness of different views and frames to the same intensity range through a unified brightness mapping rule. Specular enhancement suppression processing is then performed on the brightness-normalized image, detecting and locating high-brightness reflection areas and compressing or replacing the pixel intensity of these areas to ensure they do not affect subsequent stripe extraction and region segmentation. Finally, a synchronized image of the foam surface after distortion correction, brightness normalization, and specular enhancement suppression is output.

[0032] It should be noted that the brightness mapping rule is obtained by acquiring multi-view reference images under bubble-free or standard lighting conditions, statistically analyzing the grayscale distribution of each view image, and calculating the mapping function between the grayscale distribution and the target uniform brightness distribution.

[0033] S1.3. The edge information of the laser stripe region in the synchronous image of the foam surface is enhanced by the Laplacian operator to form a laser stripe enhanced image.

[0034] Furthermore, candidate regions for laser stripes are extracted frame by frame from the synchronized image of the foam surface according to the viewpoint. The Laplacian operator convolution operation is performed on the image within the candidate regions to calculate the second-order gradient response, thus obtaining the laser stripe edge response map. The laser stripe edge response map is then normalized and thresholded to suppress non-stripe textures and random noise. The thresholded edge response is then weighted and fused with the original synchronized image of the foam surface to highlight the edge and center structure of the laser stripes, outputting an enhanced laser stripe image.

[0035] S1.4. Based on the typical characteristics of foam, perform region labeling in multi-view foam surface images to separate non-foam regions from foam regions, forming a foam region segmentation mask.

[0036] Furthermore, based on the typical feature parameter set of foam, brightness, texture, and color features are calculated for each view of the multi-view foam surface image. Then, based on the typical feature parameter set of foam, joint similarity discrimination is performed on the calculated brightness, texture, and color features in the multi-view foam surface image. Using the feature statistical distribution described by the typical feature parameter set of foam as a reference, the similarity metric between the current pixel feature vector and the typical feature vector of foam is calculated pixel by pixel. Pixels whose similarity satisfies the consistency constraint (the current pixel is similar to the typical feature statistical distribution of foam in three dimensions: brightness, texture, and color) are marked as foam candidate pixels. Foam candidate regions are formed by the spatial aggregation of the foam candidate pixels. Connected component filtering and morphological processing are performed on the foam candidate regions to remove false regions and improve boundaries, outputting a foam region segmentation mask of the same size as the corresponding multi-view foam surface image.

[0037] It should be noted that typical foam features refer to a set of visual features that can stably distinguish between foam and non-foam areas. These features mainly include the brightness distribution features, texture roughness features, color space distribution features, and specular reflection morphology features of the foam surface. Typical foam features are formed by collecting a large number of foam and non-foam sample images under different working conditions, annotating the samples, statistically analyzing the distribution range of various features, and combining feature discrimination analysis to select feature parameters with high discriminative power and stability.

[0038] S1.5 Organize and encapsulate the foam surface synchronization image, laser stripe enhancement image, and foam region segmentation mask according to the viewpoint number to form a foam synchronization image package.

[0039] Furthermore, the synchronous image of the foam surface, the laser stripe enhanced image, and the foam region segmentation mask under the same time stamp are matched from view to view. Then, the three types of images are combined and packaged according to a fixed data structure using the view number as an index, and a unified timestamp and modal identification information are recorded during the packaging process to form a foam synchronous image package.

[0040] S2. Extract the set of pixels along the center line of the stripes from the laser stripe enhancement images from different perspectives, and perform three-dimensional reconstruction of the set of pixels along the center line of the stripes based on the parameters of the multi-view sensor array to obtain a dense three-dimensional point cloud of foam.

[0041] S2.1 Automatic stripe detection and extraction are performed on laser stripe enhanced images from different perspectives, outputting a set of stripe centerline pixels. Optical flow is used to track the changes of each laser stripe in consecutive frames, dynamically adjusting the extraction accuracy of stripe centerline pixels to obtain a high-quality set of stripe centerline pixels.

[0042] Furthermore, for laser stripe enhancement images from different viewpoints, the Canny edge detection algorithm is used for image preprocessing, identifying edge information in the image through gradient calculation. The optimal fitting curve for each laser stripe is located on the edge information using curve fitting methods (such as least squares fitting or polynomial fitting). The centerline pixel position of each laser stripe is determined by minimizing the computational error, thus obtaining accurate laser stripe positions. Optical flow is used to track pixel changes of each laser stripe in consecutive frames. Optical flow estimates pixel displacements between adjacent frames in the time series based on pixel motion in the image. The centerline pixels of each laser stripe are temporally correlated based on the laser stripe position and pixel displacement to ensure the consistency of the centerline of each stripe in consecutive frames. The extraction accuracy of the stripe centerline is dynamically adjusted using pixel displacement information in consecutive frames to ensure accurate tracking of stripes even under deformation or occlusion conditions. By merging high-quality stripe centerline pixels from multiple frames, noise and errors are eliminated to obtain a high-quality stripe centerline pixel set.

[0043] It should be noted that optical flow is a motion estimation method based on the assumptions of image brightness consistency and temporal continuity. By analyzing the changes in pixel grayscale distribution in adjacent time frames, it calculates the displacement vector of pixels on the image plane, thereby obtaining the motion information of the target in the time series. In this step, optical flow is used to estimate the displacement change of the laser stripe centerline pixels between consecutive frames. By constraining the movement direction and displacement amplitude of the stripe pixels, stable association and continuous tracking of the same laser stripe in different time frames are achieved. Based on the obtained pixel displacement information, the position estimation accuracy of the stripe centerline can be dynamically corrected, and the temporal consistency and robustness of the stripe centerline extraction results can be maintained even when the stripe is deformed, partially occluded, or experiences brightness fluctuations.

[0044] S2.2. Based on the parameters of the multi-view sensor array, perform stereo matching on the high-quality stripe centerline pixels under different viewpoints, calculate the three-dimensional coordinates, and generate a preliminary three-dimensional point cloud.

[0045] Furthermore, based on the intrinsic and extrinsic parameters of the multi-view sensor array, for each frame of the multi-view foam surface image, the centerline pixels of the laser stripes in the enhanced image under each viewpoint are extracted. Based on the intrinsic and extrinsic parameter information obtained from camera calibration, epipolar geometric relationships are used to apply epipolar constraints to the centerline pixels of the stripes under different viewpoints, limiting the matching search range to a local area near the corresponding epipolar line to reduce the search space and avoid interference from irrelevant areas. Based on the epipolar constraints, considering the continuous linear structure of the laser stripes in the image, a search window is constructed along the normal direction perpendicular to the local direction of the stripes, using the centerline pixels as a reference. Pixel matching is then performed along the stripe normal within the epipolar constraint range, thus avoiding matching ambiguities caused by excessively high grayscale similarity in the stripe direction. Within the limited search area, the SAD algorithm is used to measure the similarity of the gray-level distribution of the candidate pixel neighborhood. By calculating the sum of the absolute differences in gray levels of pixels within the window, the pixel pair with the smallest difference is selected as the initial matching result. At the same time, combined with the geometric consistency constraint of the laser stripe centerline in space, the matching results are further filtered. The geometric consistency constraint includes at least the consistency of the local slope of the stripes under different viewpoints, the continuity of curvature changes, or the consistency of the relative positional relationship between stripe segments, which is used to eliminate erroneous matching points that do not meet the geometric continuity characteristics of the stripes. After completing the matching or spatial intersection of the stripe centerline pixels under different viewpoints, the three-dimensional spatial coordinates corresponding to the pixels are uniformly calculated and the coordinate system is transformed using the relationship between the camera's intrinsic and extrinsic parameters and the principle of triangulation. The obtained three-dimensional coordinate points are summarized to generate a preliminary three-dimensional point cloud.

[0046] It's important to note that the SAD algorithm (Sum of Absolute Differences) is a common similarity measurement method used in image matching. In stereo matching, the SAD algorithm evaluates the matching degree by calculating the differences between pixels in the same region under two viewpoints. Specifically, it compares pixel values ​​in selected regions under viewpoints 1 and 2 pixel by pixel, calculates the absolute difference of each pixel, and sums these differences. The pixel pair with the smallest difference is considered the best match. The SAD algorithm is simple and efficient, and is widely used in stereo image matching, especially in relatively uniform or simple lighting conditions. Epipolar constraint is a geometric constraint in stereo vision determined by the relative positional relationship between two cameras. It means that the corresponding point of a pixel in one viewpoint must fall on its corresponding epipolar line in another viewpoint. Epipolar constraints are obtained by calculating the fundamental or essential matrix using intrinsic and extrinsic parameters obtained from camera calibration, and then using this matrix to map pixels in one viewpoint to epipolar lines in another viewpoint.

[0047] S2.3. Minimize errors and smooth local areas of the initial 3D point cloud, finely adjust the position and density of the point cloud, obtain the initial foam-dense 3D point cloud, and post-process the initial foam-dense 3D point cloud using a surface reconstruction algorithm to output the foam-dense 3D point cloud.

[0048] Furthermore, by accumulating multiple frames over time, projecting multiple laser stripes, or scanning stripes on the foam surface over time to cover different spatial locations, multiple sets of preliminary 3D point clouds obtained within a continuous acquisition period are fused and superimposed in a unified coordinate system, thus forming a continuous and densely distributed preliminary 3D point cloud set within the spatial range of the foam surface. An error minimization algorithm is used to process noise and systematic errors in the preliminary 3D point cloud set, reducing the impact of outliers introduced by stripe matching errors, camera calibration errors, or sensor noise. Simultaneously, local smoothing processing is performed on the point cloud to suppress irregular fluctuations and high-frequency noise, finely adjusting the point cloud while maintaining the macroscopic geometry of the foam. The initial dense 3D point cloud of foam is obtained by determining the positional continuity and spatial density distribution. When post-processing the initial dense 3D point cloud using a surface reconstruction algorithm, smoothing operations (such as Gaussian smoothing or bilateral filtering) are used to remove noise points and small-scale high-frequency fluctuations, while preserving large-scale structural information. Point cloud interpolation methods are used to fill missing regions in the point cloud, using the average value of surrounding points or a surface fitting-based approach to fill blank areas. Discontinuous surfaces are corrected using surface normal estimation and boundary repair methods to make surface connections smoother and eliminate cracks. A smooth and continuous 3D surface is generated using Delaunay triangulation or Poisson reconstruction algorithms, outputting the dense 3D point cloud of foam.

[0049] like Figure 5 As shown, the reconstruction error over time is illustrated when acquiring flotation foam state data for gold ore by combining multi-view image recognition and laser stripe 3D reconstruction technology. The blue curve in the figure represents the 3D reconstruction error, which fluctuates over time. This fluctuation is due to factors such as image distortion correction, gloss suppression, and laser stripe density, leading to changes in the accuracy of the 3D reconstruction. In practical applications, analyzing the error fluctuations can determine whether preprocessing steps (such as illumination adjustment and distortion correction) have adequately eliminated environmental and equipment interference, ensuring high-precision foam geometry data. This figure supports the verification of the effectiveness of image preprocessing and laser stripe adjustment in the invention and ensures the accuracy of 3D modeling of flotation foam.

[0050] S3. Use a foam region segmentation mask to remove non-foam point clouds in the dense foam 3D point cloud, and use the curvature change constraint of adjacent frames to suppress isolated noise points. Reconstruct the foam 3D surface through meshing, and stitch it together according to a unified timestamp to form a multi-dimensional foam state feature matrix.

[0051] S3.1. Using the binarized value of the foam region segmentation mask, the foam surface is separated from the dense 3D point cloud of foam, the foam region is extracted, and the non-foam region is removed from the dense 3D point cloud of foam.

[0052] Furthermore, the brightness, texture, color, and specular features of the multi-view foam surface images are thresholded. Based on the typical feature parameter set of foam, the foam region is distinguished from the non-foam region to obtain the binarized value in the foam region segmentation mask. According to the binarized value in the foam region segmentation mask, the pixels marked as foam in the foam region segmentation mask are mapped to the pixels of non-foam. Each 3D point in the dense 3D point cloud of foam is compared with the pixel value in the foam region segmentation mask to extract the 3D point of the corresponding foam region. The points corresponding to the non-foam region are removed to obtain the foam region point cloud after removal.

[0053] S3.2 Calculate the local curvature of consecutive frames of the point cloud in the bubble region after removal to evaluate the surface curvature. Use the curvature difference between adjacent frame point clouds to identify outliers and isolated noise points. Suppress isolated noise points in the point cloud in the bubble region after removal based on curvature change constraints.

[0054] Furthermore, by selecting the neighboring points of each point in the bubble region point cloud after removal and calculating the covariance matrix, eigenvalue decomposition is performed on the covariance matrix of the neighboring point cloud for each point, and the eigenvector corresponding to the smallest eigenvalue is selected as the normal direction of that point. By calculating the eigenvalues ​​of the neighboring point cloud, the local curvature is obtained, where the principal curvature is represented by the difference between the smallest and largest eigenvalues. The surface curvature of each point is evaluated using the local curvature, thereby obtaining the local curvature distribution of the entire bubble region point cloud. Based on the local curvature distribution, using the bubble region point clouds of adjacent frames, by comparing the curvature differences of each bubble region point cloud in adjacent frames, points with large curvature changes in different frames that may belong to outliers or isolated noise points are identified. Based on the curvature change constraint, these outliers and isolated noise points are suppressed and removed from the bubble region point cloud after removal, thereby obtaining a suppressed bubble-dense 3D point cloud.

[0055] By selecting the neighboring points of each point in the point cloud of the bubble region after removal and calculating the covariance matrix, the expression is: ; in: It is a point The covariance matrix reflects the geometric distribution and orientation information of the neighboring points of that point; Neighboring points 3D coordinates; It is an index of the neighboring points; It's a transpose operation, which converts a vector... Transpose into a column vector; By calculating the eigenvalues ​​of the neighborhood point cloud, the local curvature is obtained, expressed as: ; in: It is local curvature, representing a point. The degree of curvature of the surface; the larger the value, the stronger the curvature. It is the smallest eigenvalue of the covariance matrix, representing the degree of change in the direction of strongest curvature; It is the largest eigenvalue of the covariance matrix, representing the degree of change in the direction of weakest curvature; It should be noted that curvature variation constraint refers to identifying and limiting points with excessive curvature variation by calculating the local curvature difference of the point cloud in the same bubble region in adjacent frames, so as to avoid interference from abnormal or isolated noise points; curvature variation constraint is obtained by analyzing the statistical characteristics of curvature variation of point cloud in bubble region in consecutive frames.

[0056] S3.3 Calculate the local neighborhood of each suppressed foam-dense 3D point cloud, and use the least squares method to calculate the normal direction of the points in the local neighborhood, and output the normal field.

[0057] Furthermore, for each point in the suppressed, dense 3D point cloud of foam, its local neighbors are selected. This is typically done by setting a neighborhood radius or using the k-nearest neighbor method to select several surrounding points. The least squares method is then used to calculate the deviation vector between each neighboring point and the current point. By minimizing the sum of the squares of these deviation vectors, an optimal normal direction is obtained. The normal direction is determined by the fitted principal direction, minimizing the error of the point cloud being perpendicular to this direction. By performing the above calculations on all points, the normal field of the entire point cloud is output, i.e., the normal direction of each point, which describes the local normal information of the foam region surface.

[0058] S3.4. Triangulate the suppressed dense three-dimensional point cloud of foam using the normal field to generate a three-dimensional foam surface. Then, align the three-dimensional foam surfaces with multiple timestamps and stitch them together according to a unified timestamp to generate a multi-dimensional foam state feature matrix.

[0059] Furthermore, for each point in the suppressed dense 3D point cloud of foam, its neighboring points are selected and the covariance matrix of the neighboring point cloud is calculated. The normal direction is obtained through eigenvalue decomposition to characterize the local normal information of the point cloud surface. Using this local normal information, the adjacent points in the suppressed 3D point cloud are connected into a triangular mesh using the Delaunay triangulation algorithm to generate the 3D foam surface. The 3D foam surfaces at different time stamps are time-aligned. Based on a unified time stamp, spatial transformation methods (such as rigid transformation or affine transformation) are used to ensure the spatial consistency of the foam surfaces at different time stamps. All time-aligned 3D foam surfaces are stitched together in a unified time stamp order to form a continuous 3D foam surface dataset. Based on these stitched surfaces, a multidimensional foam state feature matrix is ​​generated to reflect the temporal evolution and dynamic changes of the foam state.

[0060] S4. Input the three-dimensional feature matrix of the foam into the online inference model to dynamically identify the foam state in mineral processing and perform joint regression of key parameters to obtain the confidence level. At the same time, use the confidence level-driven online domain adaptive update strategy to correct the network feature distribution in real time and output the foam state result package.

[0061] S4.1. The foam 3D feature matrix is ​​received through the input layer. The foam 3D feature matrix is ​​input into the main branch of CNN to extract spatial features and the auxiliary branch of MLP to extract auxiliary features. Temporal modeling is performed through a lightweight temporal layer. The main branch features and auxiliary branch features are combined by the fusion layer to form an online inference model.

[0062] Furthermore, the input layer receives data from the 3D feature matrix of the foam, which is typically a multi-dimensional tensor containing spatial geometric information. This input layer directly passes the data to the main branch of the CNN. In the main branch, the first layer is a convolutional layer, where convolutional operations extract spatial features through convolutional kernels. Then, pooling layers reduce the dimensionality of the feature maps, preserving the most salient spatial information to form a compressed spatial feature representation. These spatial features, after convolution and pooling, are then passed to the temporal layer for temporal modeling. The input layer also passes the data of the 3D feature matrix of the bubble to the MLP auxiliary branch. The MLP auxiliary branch, composed of fully connected layers, is responsible for extracting auxiliary features from the input feature matrix, such as bubble size and gloss. In this branch, the input data is first processed by multiple fully connected layers, with the output of each layer serving as the input to the next. The processed feature vectors ultimately form auxiliary feature representations, which are also passed to the temporal layer for temporal modeling.

[0063] After the CNN main branch and MLP auxiliary branch extract spatial and auxiliary features respectively, their outputs are passed to a lightweight temporal layer. This temporal layer processes time-series features, combining the temporal features output from the CNN main branch and MLP auxiliary branch to perform temporal modeling and capture temporal relationships. The temporal layer typically uses recurrent neural networks (such as LSTM) or temporal convolutional networks (such as TCN) to model the changes in features over time.

[0064] After the temporal modeling of the features is completed in the temporal layer, the output temporal features are passed to the fusion layer. In the fusion layer, the temporal features of the CNN main branch and the MLP auxiliary branch are merged. Common fusion methods include concatenation (connecting the two outputs into a large feature vector) or weighted averaging (weighted summation of the two features). The fused feature representation integrates spatial information, auxiliary features, and temporal variation information, and these features are used for the final bubble state recognition.

[0065] The merged features are passed to the network's output layer. Depending on the task, the output layer may be a regression layer (used to predict numerical features of the foam, such as foam thickness or bubble size) or a classification layer (used to identify different foam state categories). By calculating the loss function and performing backpropagation, the prediction results of the output layer are compared with the true labels, updating the network weights and forming an online inference model that integrates spatial information and surface features.

[0066] It should be noted that a large amount of labeled 3D bubble feature dataset is used, and the data is input into the online inference model for training. The input layer receives the 3D bubble feature matrix, and then extracts spatial and auxiliary features through the main branch of the CNN and the auxiliary branch of the MLP, respectively. The temporal layer models the spatial and auxiliary features over time to capture the temporal variation of the bubble state. The fusion layer integrates features from different sources to form a unified feature representation. Through backpropagation and optimization algorithms (such as the Adam optimizer), the model parameters are updated according to the loss function (usually regression loss or classification loss) to minimize the prediction error. During training, the weights of the convolutional layers, fully connected layers, temporal layers, and fusion layers of the online inference model are continuously adjusted until the online inference model can accurately output the bubble state prediction results.

[0067] S4.2 Input the three-dimensional feature matrix of the foam into the online inference model, extract spatial features from the three-dimensional feature matrix of the foam through the main branch of CNN to obtain spatial geometric features, and use the auxiliary branch of MLP to identify the depth information of bubble size distribution and gloss to output surface property features.

[0068] Furthermore, after inputting the 3D feature matrix of the foam into the online inference model, the CNN main branch extracts spatial geometric features through convolutional layers, processes the spatial information in the 3D feature matrix of the foam, and obtains geometric features reflecting the shape, structure, and spatial distribution of the foam surface. In this process, convolutional operations progressively extract local spatial features through convolutional kernels and reduce feature dimensionality through pooling layers, preserving key spatial information. The MLP auxiliary branch uses fully connected layers to perform deep information recognition on surface property features such as bubble size distribution and gloss. By learning and mapping the input features layer by layer, it ultimately outputs surface property features such as foam surface gloss and bubble size distribution, which are high-level feature parameters describing the physical and visual attributes of the foam surface. These parameters mainly reflect bubble size distribution, surface gloss, and their spatial variation characteristics, and are used to characterize the appearance and structural features of the foam state. This process mainly extracts deep surface information from the input data through weight training and feature mapping, ensuring that the output surface features accurately reflect the foam state.

[0069] S4.3. Based on spatial geometric features and surface property features, capture the dynamic changes of foam state, and use a fusion layer to splice and fuse spatial geometric features and surface property features to form foam joint features. Then, use a joint regression method to dynamically identify foam state and output foam state results and confidence level.

[0070] Furthermore, based on spatial geometric features and surface property features, the dynamic changes of foam state are captured. Time series information is used to model the changes in foam state to ensure that the spatial and surface features at each moment can reflect the evolution of foam state. In the fusion layer, the extracted spatial geometric features and surface property features are spliced ​​and fused to form a comprehensive foam joint feature that includes spatial geometric information and surface property information. Using a joint regression method, dynamic identification is performed through this joint feature to accurately predict the current state of foam. The prediction results and confidence levels of foam state are output to reflect the accuracy and reliability of the model in identifying foam state.

[0071] S4.4. Based on the confidence-driven online adaptive update strategy, the weights and learning rate of the network feature distribution are adjusted according to the confidence level, and the incremental learning method is used to correct the feature distribution in the network in real time, and output the bubble state result package.

[0072] Furthermore, based on a confidence-driven online adaptive update strategy, the weights and learning rate of the network feature distribution are adjusted according to the confidence level of each bubble state prediction. This ensures that the online inference model increases the update intensity when the confidence level is low and decreases the learning rate when the confidence level is high, thereby optimizing the stability and convergence of the online inference model. An incremental learning method is adopted to correct the feature distribution in the network in real time. By continuously introducing new data and updating the model parameters, the online inference model can adapt to the changing bubble states and output the updated bubble state result package.

[0073] like Figure 6 As shown, the changes in system confidence and corresponding fluctuations in recognition accuracy can be observed when the foam state changes. When the foam state changes significantly, the system confidence fluctuates, and these changes directly affect the recognition accuracy. The maximum difference point and feature peak in the figure mark the key points of confidence and accuracy changes during the recognition process, demonstrating how the confidence mechanism guides the model to adaptively update, ensuring the stability and accuracy of the recognition results. This verifies the role of the confidence-driven model update strategy in improving the stability of foam state recognition in dynamic environments, ensuring efficient recognition capabilities in real-time monitoring.

[0074] This embodiment also provides a computer device applicable to the online measurement method of mineral processing foam state based on three-dimensional vision, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the online measurement method of mineral processing foam state based on three-dimensional vision as proposed in the above embodiment.

[0075] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0076] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the online measurement method for mineral processing foam state based on three-dimensional vision as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0077] In summary, this invention achieves online acquisition of the three-dimensional geometric morphology of gold ore flotation foam by combining multi-view image recognition with laser stripe three-dimensional reconstruction. Under semantic constraints and temporal stabilization processing, multi-dimensional state features of the foam are constructed. Furthermore, an online inference model with confidence-driven adaptive capabilities is used to complete the dynamic identification and parameter regression of the foam state, thereby improving the image recognition's ability to stably represent the real spatial state of the foam.

[0078] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for online measurement of mineral processing foam state based on three-dimensional vision, characterized in that: include, Simultaneously acquire multi-view images of the foam surface and perform online image preprocessing to output a synchronized foam image package; the synchronized foam image package includes synchronized foam surface images, laser stripe enhanced images, and foam region segmentation masks; The pixel set of the stripe center line is extracted from the laser stripe enhanced image from different perspectives. The pixel set of the stripe center line is reconstructed in three dimensions according to the parameters of the multi-view sensor array to obtain a dense three-dimensional point cloud of foam. Non-foam point clouds in dense foam 3D point clouds are removed by using a foam region segmentation mask, and isolated noise points are suppressed by the curvature change constraint between adjacent frames. The foam 3D surface is generated by mesh reconstruction and stitched together according to a unified timestamp to form a multi-dimensional foam state feature matrix. The three-dimensional feature matrix of the foam is input into the online inference model to dynamically identify the foam state in mineral processing and perform joint regression of key parameters to obtain the confidence level. At the same time, the online domain adaptive update strategy driven by the confidence level is used to correct the network feature distribution in real time and output the foam state result package.

2. The online measurement method for mineral processing foam state based on three-dimensional vision as described in claim 1, characterized in that: The multi-view foam surface image is obtained by simultaneously acquiring RGB color images, grayscale images, and laser stripe images of the foam surface and assigning them a unified timestamp.

3. The online measurement method for mineral processing foam state based on three-dimensional vision as described in claim 2, characterized in that: The online image preprocessing is performed to output a bubble-synchronized image package. The specific steps are as follows: Multi-view foam surface images are processed to eliminate camera lens distortion through distortion correction, and then brightness normalization and highlight suppression are performed to output synchronized foam surface images. The edge information of the laser stripe region in the synchronous image of the foam surface is enhanced by the Laplacian operator to form a laser stripe enhanced image. Based on the typical characteristics of foam, region labeling is performed on multi-view foam surface images to separate non-foam regions from foam regions, forming a foam region segmentation mask. The synchronized images of the foam surface, the laser stripe enhanced images, and the foam region segmentation mask are organized and packaged according to the viewpoint number to form a synchronized foam image package.

4. The online measurement method for mineral processing foam state based on three-dimensional vision as described in claim 3, characterized in that: The process involves extracting the pixel set of the stripe centerline from laser stripe enhancement images at different viewpoints, and then performing 3D reconstruction of the pixel set of the stripe centerline based on the parameters of the multi-view sensor array to obtain a dense 3D point cloud of foam. The specific steps are as follows. Automatic stripe detection and extraction are performed on laser stripe enhanced images from different perspectives, outputting a set of stripe centerline pixels. The optical flow method is used to track the changes of each laser stripe in consecutive frames, dynamically adjust the stripe centerline pixel extraction accuracy, and obtain a high-quality set of stripe centerline pixels. Based on the parameters of the multi-view sensor array, stereo matching is performed on the high-quality stripe centerline pixels under different viewpoints to calculate the three-dimensional coordinates and generate a preliminary three-dimensional point cloud. Error minimization and local smoothing are performed on the initial 3D point cloud, and the position and density of the point cloud are finely adjusted to obtain the initial foam-dense 3D point cloud. The initial foam-dense 3D point cloud is then post-processed using a surface reconstruction algorithm to output the foam-dense 3D point cloud.

5. The online measurement method for mineral processing foam state based on three-dimensional vision as described in claim 4, characterized in that: The process of using a foam region segmentation mask to remove non-foam point clouds from a dense 3D point cloud and employing curvature variation constraints between adjacent frames to suppress isolated noise points involves the following specific steps. By using the binarized value of the foam region segmentation mask, the foam surface is separated from the dense 3D point cloud of foam, the foam region is extracted, and the non-foam region is removed from the dense 3D point cloud of foam. The local curvature of the point cloud in the bubble region after removal is calculated to evaluate the surface curvature. The curvature difference between adjacent point clouds is used to identify outliers and isolated noise points. Isolated noise points in the point cloud in the bubble region after removal are suppressed based on curvature change constraints.

6. The online measurement method for mineral processing foam state based on three-dimensional vision as described in claim 1, characterized in that: The process involves generating a three-dimensional foam surface through mesh reconstruction and then stitching the layers together according to a unified timestamp to form a multi-dimensional foam state feature matrix. The specific steps are as follows: Calculate the local neighborhood of each suppressed foam-dense 3D point cloud, and use the least squares method to calculate the normal direction of the points in the local neighborhood, and output the normal field; The suppressed dense 3D point cloud of foam is triangulated using the normal field to generate a 3D foam surface. The 3D foam surfaces with multiple timestamps are time-aligned and stitched together according to a unified timestamp to generate a multi-dimensional foam state feature matrix.

7. The online measurement method for mineral processing foam state based on three-dimensional vision as described in claim 1, characterized in that: The online inference model receives a foam 3D feature matrix through an input layer, inputs the foam 3D feature matrix into the main branch of a CNN to extract spatial features and the auxiliary branch of an MLP to extract auxiliary features, performs temporal modeling through a lightweight temporal layer, and combines the main branch features and auxiliary branch features through a fusion layer.

8. The online measurement method for mineral processing foam state based on three-dimensional vision as described in claim 7, characterized in that: The process involves inputting a three-dimensional feature matrix of foam into an online inference model to dynamically identify the foam state in mineral processing and perform joint regression of key parameters to obtain confidence levels. Simultaneously, a confidence-driven online domain adaptive update strategy is used to correct the network feature distribution in real time and output a foam state result package. The specific steps are as follows: The three-dimensional feature matrix of the foam is input into the online inference model. The spatial features of the three-dimensional feature matrix of the foam are extracted through the main branch of CNN to obtain spatial geometric features. The MLP auxiliary branch is used to identify the depth information of bubble size distribution and gloss, and output surface property features. Based on spatial geometric features and surface property features, the dynamic changes of foam state are captured, and the spatial geometric features and surface property features are spliced ​​and fused using a fusion layer to form foam joint features. The foam state is dynamically identified through a joint regression method, and the foam state results and confidence scores are output. Based on a confidence-driven online adaptive update strategy, the weights and learning rate of the network feature distribution are adjusted according to the confidence level, and an incremental learning method is used to correct the feature distribution in the network in real time, outputting a bubble state result package.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the online measurement method for mineral processing foam state based on three-dimensional vision as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the online measurement method for mineral processing foam state based on three-dimensional vision as described in any one of claims 1 to 8.

Images