Image recognition-based on-line diagnosis method and device for froth state of mineral separation and medium

Abstract

The application discloses an image recognition-based on-line diagnosis method and device for ore dressing foam state and a medium, and relates to the technical field of machine vision. The method comprises the following steps: collecting two foam images with different polarization angles and synchronously collecting process parameters to establish a sliding window; performing polarization difference processing on the foam images to obtain a polarization feature map and performing image quality self-checking to generate a foam working area; extracting bubble instances in the foam working area, counting breakage events and merger events to obtain a breakage rate and a merger rate, and simultaneously calculating a foam surface speed average based on adjacent frame displacement; inputting the polarization feature map time series data, the breakage rate, the merger rate, the foam surface speed average and process parameter time series data into a deep learning model, outputting foam state categories and soft measurement prediction data, and constructing consistency evaluation; recursively calibrating the soft measurement prediction data, and triggering a drift alarm when the consistency evaluation is continuously abnormal and the calibration residual is continuously out of limit, so that on-line prediction of grade and recovery rate is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, in particular to an image recognition-based online diagnosis method, device and medium for a beneficiation froth state. BACKGROUND

[0002] In recent years, the froth state monitoring of the flotation process has gradually evolved from manual experience identification to online observation based on industrial cameras, and further developed a froth state recognition method with machine vision as the core. Related research has expanded from early hand-crafted feature extraction such as gray level / texture / edge and traditional classifiers to end-to-end deep learning time series modeling of froth images, while combining process parameters to achieve process state evaluation and soft measurement prediction. In order to improve the observability of the physical properties such as froth film reflection and water content changes, polarization imaging, time series analysis and multi-source fusion have become important development directions, making online diagnosis gradually have the engineering basis of continuity, quantification and deployability.

[0003] There are still two key deficiencies in the existing related technologies: first, most methods rely on visible light appearance features or single-frame segmentation, and when facing light fluctuations, mirror reflections and local occlusions, the froth working area and bubble boundaries are easy to drift or break, resulting in unstable instance extraction and breakage / merger statistics, which is difficult to support continuous and reliable online diagnosis. Second, the existing methods directly use the soft measurement prediction results as the online output, lack of fusion consistency discrimination based on polarization features, breakage rate, merger rate, average froth surface speed and process parameter deviation, and cannot suppress the release of soft measurement prediction data under abnormal working conditions, so it is difficult to obtain long-term stable online soft measurement prediction effect. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides an image recognition-based online diagnosis method for a beneficiation froth state, which solves the problems of unstable extraction of the froth working area under light fluctuations, reflections and occlusions, and lack of consistency evaluation for the release of soft measurement prediction data in the prior art.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides an image recognition-based online diagnosis method for a beneficiation froth state, which includes collecting two different polarization angle froth images and synchronously collecting process parameters, and establishing a sliding window.

[0008] The polarization angle froth images in the sliding window are subjected to polarization difference processing to obtain a polarization feature map, and the polarization feature map is subjected to image quality self-checking to generate a froth working area.

[0009] In the foam working area, bubble instances are extracted from the polarization feature map, and bubble breakage events and merger events are counted in the sliding window to obtain breakage rate and merger rate, and the average foam surface speed is calculated based on the displacement of adjacent frames;

[0010] The time series data of the polarization feature map, the breakage rate, the merger rate, the average foam surface speed, and the time series data of the process parameters are input into a deep learning model to output foam state categories and soft measurement prediction data, and a consistency evaluation is constructed;

[0011] The soft measurement prediction data is recursively calibrated, and a drift alarm is triggered when the consistency evaluation is continuously abnormal and the calibration residual is continuously out of limit.

[0012] As a preferred scheme of the image recognition-based beneficiation foam state online diagnosis method, the two different polarization angle foam images are collected and the process parameters are collected synchronously, a sliding window is established, and the specific steps are as follows,

[0013] A hard trigger signal is sent to an industrial camera to obtain original polarization images composed of different polarization angle pixel subsets in the same field of view;

[0014] Pixel rearrangement and aggregation are performed on the original polarization images to obtain two different polarization angle images;

[0015] A process parameter vector is read, and a pair of polarization images and a process parameter vector with the shortest time stamp distance from the diagnosis moment are selected;

[0016] When the time difference between the collection time stamp and the diagnosis moment is not more than the time difference threshold, the pair of polarization images and the process parameter vector are written into the sliding window to form a sliding window data set.

[0017] As a preferred scheme of the image recognition-based beneficiation foam state online diagnosis method, the polarization angle foam images in the sliding window are polarization difference processed to obtain a polarization feature map, and the polarization feature map is subjected to image quality self-checking to generate a foam working area, and the specific steps are as follows,

[0018] Based on the sliding window data set, two different polarization angle foam images in the same field of view are calculated pixel by pixel to obtain a total intensity image and a polarization feature map;

[0019] The clarity, brightness deviation, and overexposure proportion are calculated for each diagnosis moment, and a quality score is constructed, and the proportion of qualified moments is counted for all diagnosis moments in the sliding window;

[0020] When the qualified moment proportion is greater than the proportion threshold value, the total intensity image and the polarization feature map are input into a lightweight U-Net semantic segmentation network as input to obtain a foam region probability map, the foam region probability map is binarized according to a fixed probability threshold value to form a foam region mask, and consistent fusion is performed on the mask in the window to obtain a foam working area.

[0021] As a preferred scheme of the image recognition-based beneficiation foam state online diagnosis method, wherein: in the foam working area, bubble instances are extracted based on the polarization feature map, bubble rupture events and merger events are counted in the sliding window, a rupture rate and a merger rate are obtained, and a foam surface speed average is calculated based on displacement of adjacent diagnosis time points, and the specific steps are as follows,

[0022] According to the foam working area, the total intensity image and the polarization feature map are cropped, the polarization feature map is smoothed and denoised, and a gradient amplitude map is calculated, and the gradient amplitude map is taken as a bubble boundary candidate map.

[0023] The total intensity image is subjected to contrast enhancement in the foam working area, and is binarized pixel by pixel with a local threshold value to obtain a bubble membrane boundary candidate region, and then the bubble membrane boundary candidate region is inverted to obtain a bubble interior region binary image and is subjected to Euclidean distance transformation to form a bubble center potential map.

[0024] After the initial marking points are obtained by searching the neighborhood maximum value on the bubble center potential map and are merged, the bubble interior region is divided into pixel-connected domains by watershed segmentation and is given a unique instance number to form a bubble instance set of the current frame, and the area and the centroid are calculated.

[0025] As a preferred scheme of the image recognition-based beneficiation foam state online diagnosis method, wherein: in the foam working area, bubble instances are extracted based on the polarization feature map, bubble rupture events and merger events are counted in the sliding window, a rupture rate and a merger rate are obtained, and a foam surface speed average is calculated based on displacement of adjacent diagnosis time points, and the specific steps are as follows,

[0026] The bubble instance sets of adjacent diagnosis time points are subjected to deterministic matching with the centroid distance as the main constraint and the area overlap ratio as the secondary constraint, and bubble rupture events and merger events are counted.

[0027] The rupture rate and the merger rate are calculated through the number of events in the sliding window.

[0028] At each pair of adjacent diagnosis time points, only the dense optical flow of the total intensity image is calculated in the foam working area to obtain a pixel displacement vector field, and the spatial average and the time average of the displacement module length are taken as the foam surface speed average.

[0029] As a preferred scheme of the image recognition-based online diagnosis method for the froth state of mineral separation provided in the present application, wherein: the time series data of the polarization feature map, the breakage rate, the coalescence rate, the average froth surface speed, and the time series data of the process parameters are input into a deep learning model to output the froth state category and the soft measurement prediction data, and the consistency evaluation data is constructed, and the specific steps are as follows,

[0030] The polarization feature map is cropped to the froth working area according to a fixed size, the process parameters in the sliding window are standardized according to the mean and standard deviation of each dimension, the standardized parameter sequence is obtained, the breakage rate, the coalescence rate, and the average froth surface speed are normalized according to a fixed scale coefficient, and the normalized dynamic quantity is obtained.

[0031] The polarization feature map, the standardized parameter sequence, and the normalized dynamic quantity are input into a deep learning model of a multi-input multi-task network using CNN+GRU to perform forward inference, and the froth state category probability vector and the soft measurement prediction data are obtained.

[0032] As a preferred scheme of the image recognition-based online diagnosis method for the froth state of mineral separation provided in the present application, wherein: the consistency evaluation is constructed, and the specific steps are as follows,

[0033] For each dimension of the process parameters, the parameter deviation is taken as an absolute value after scale reduction and accumulated to obtain the process parameter deviation degree.

[0034] Based on the process parameter deviation degree, the breakage rate, the coalescence rate, and the surface speed mean are normalized according to a fixed scale coefficient to construct the consistency evaluation.

[0035] When the consistency evaluation is greater than or equal to the consistency threshold, the froth state category and the soft measurement prediction data are released, and when the consistency evaluation is less than the consistency threshold, the froth state category is released but the soft measurement data is suppressed.

[0036] As a preferred scheme of the image recognition-based online diagnosis method for the froth state of mineral separation provided in the present application, wherein: the soft measurement prediction data is recursively calibrated, and the drift alarm is triggered when the consistency evaluation is continuously abnormal and the calibration residual is continuously out of limit, and the specific steps are as follows,

[0037] The grade and recovery rate in the soft measurement prediction data are respectively subjected to recursive calibration, and the difference between the assay value and the calibrated soft measurement value is taken as the calibration residual of the grade and the recovery rate.

[0038] A continuous counter is maintained, and when the consistency evaluation at the current time is lower than the consistency threshold and the grade residual exceeds the grade residual threshold or the recovery rate residual exceeds the recovery rate residual threshold, the continuous counter is incremented by one, otherwise the continuous counter is cleared.

[0039] When the continuous counter accumulates to the set number of times, it is determined that there is drift and a drift alarm flag is set.

[0040] In a second aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program, when executed by the processor, implements any step of the image recognition-based online diagnosis method for froth state in mineral separation according to the first aspect of the present application.

[0041] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step of the image recognition-based online diagnosis method for froth state in mineral separation according to the first aspect of the present application.

[0042] The present application has the following beneficial effects: the stable extraction of the froth working area is achieved by inputting the total intensity image and the polarization feature map into the lightweight U-Net semantic segmentation network and performing consistent fusion on the mask in the window to generate the froth working area; and the online soft measurement prediction of the grade and the recovery rate is achieved by inputting the polarization feature map time series data, the normalized dynamic quantity and the standardized parameter sequence into the multi-input multi-task network of CNN+GRU and outputting the froth state category and the soft measurement prediction data. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0044] Figure 1 The flowchart of the image recognition-based online diagnosis method for froth state in mineral separation.

[0045] Figure 2 The flowchart for generating the froth working area.

[0046] Figure 3 The flowchart for extracting the bubble instance.

[0047] Figure 4 The flowchart for drift alarm.

[0048] Figure 5 The comparative data graph of the mask of adjacent frames changing with disturbance intensity.

[0049] Figure 6 The comparative data graph of the grade assay value and the online soft measurement prediction. DETAILED DESCRIPTION

[0050] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0051] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0052] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0053] Reference Figures 1-4 This is one embodiment of the present invention, which provides an online diagnostic method for mineral processing foam state based on image recognition, including the following steps:

[0054] S1. Acquire two foam images with different polarization angles and simultaneously acquire process parameters to establish a sliding window;

[0055] An industrial camera is installed above the froth zone of the flotation cell. A polarization imaging component is fixedly installed at the incident optical path end of the industrial camera, so that two froth images with different polarization angles can be acquired under the same field of view and the same exposure conditions.

[0056] At each diagnostic moment Upon arrival, a hard trigger signal is sent to the industrial camera, causing the camera to acquire a raw polarization image within the same field of view. This raw polarization image consists of a subset of pixels with different polarization angles output by the polarization imaging component. A deterministic pixel rearrangement and aggregation is performed on the raw polarization image to acquire two images with different polarization angles, including the polarization angle... Image and polarization angle image.

[0057] At the same diagnostic time Upon arrival, data is read from the PLC (Programmable Logic Controller) via OPC UA (OPC Unified Architecture). The process parameter vector corresponding to each moment includes several items such as collector flow rate, frother flow rate, aeration rate, slurry concentration, liquid level, and slurry flow rate.

[0058] At each diagnostic moment Generate an alignment reference timestamp, and select the closest timestamp from the camera acquisition queue. a pair of polarized images, while the sampling time stamp of the PLC parameter is selected from the PLC parameter acquisition queue closest to the process parameter vector; when the time difference of the closest sample exceeds the time difference threshold, the diagnosis moment is marked as out of sync and the data is discarded; when the time difference does not exceed the time difference threshold, the pair of polarized images and the process parameter vector are uniformly bound to the same aligned sample and written into the sliding window to form a sliding window data set.

[0059] It should be noted that the time difference threshold is obtained by simultaneously recording the camera acquisition time stamp and the PLC parameter sampling time stamp, counting the time difference distribution of the two under normal communication and normal load conditions, and taking the upper limit (e.g., 99% quantile) of the time difference distribution as the time difference threshold, which is generally in the range of [0.1s, 0.3s].

[0060] S2, polarized differential processing is performed on the polarized angle foam images in the sliding window to obtain a polarized feature map, and the polarized feature map is subjected to image quality self-checking to generate a foam working area;

[0061] Based on the sliding window data set, the total intensity image and the polarized feature map are calculated pixel by pixel from two different polarized angle foam images in the same field of view, and the expression is:

[0062] ;

[0063] ;

[0064] wherein, represents the total intensity image at the diagnosis moment ; represents the pixel intensity value of the foam image with a polarized angle of , represents the pixel intensity value of the foam image with a polarized angle of , represents the polarized feature image pixel value at the diagnosis moment ; represents a small positive number, represents the time stamp of the diagnosis moment in the sliding window.

[0065] The clarity, brightness deviation and overexposure ratio are calculated for each diagnosis moment , and a quality score is constructed to determine whether the frame can be used for online diagnosis with a certainty threshold, specifically, for the diagnosis moment The total intensity image is denoised and gray value normalized, and then the definition index and brightness index are calculated in the full diagnosis time range. The definition index uses the average value of the gradient amplitude to represent whether the image details are sufficient, and the brightness index uses the full-frame average gray value to represent whether the exposure is in the normal interval. At the same time, the saturation pixel ratio, i.e. the proportion of pixels with a gray value reaching the upper limit of the sensor in the total intensity image, is calculated to identify the loss of effective information caused by overexposure or strong light reflection. The definition index, brightness index and saturation pixel ratio are compared with the pre-calibrated reference threshold value. When the definition is lower than the reference threshold value, it is determined to be blurred. When the brightness is lower than the reference threshold value, it is determined to be too dark. When the brightness is higher than the reference threshold value or the saturation pixel ratio exceeds the reference threshold value, it is determined to be overexposed.

[0066] It should be noted that the reference threshold value is obtained by continuously collecting a representative foam image sequence (covering the typical state of normal definition and normal exposure) under the condition of clean equipment, normal light compensation and stable working condition, respectively calculating the distribution of the definition index, the brightness index and the saturation pixel ratio in the representative foam image sequence, taking the median as the reference value, and determining the upper and lower limit thresholds with the quantile range (for example, 5% quantile and 95% quantile). At the same time, a small amount of control samples of “deliberate defocus / obstruction / overdark / overexposure” are collected to verify that the threshold value can stably distinguish between normal and abnormal. Finally, the reference threshold value is fixed.

[0067] The proportion of qualified time instants in the sliding window is calculated. When the proportion of qualified time instants is less than or equal to the proportion threshold, the non-diagnosable label is output. When the proportion of qualified time instants is greater than the proportion threshold, it is considered that the window image quality is qualified, and the working area is generated. Specifically, the total intensity image and the polarization feature map are input into the lightweight U-Net semantic segmentation network which has been trained offline and fixed to obtain the foam region probability map. The foam region mask is formed by binarization according to a fixed probability threshold, and the mask in the window is fused consistently. Specifically, the foam region binary mask obtained for each frame in the sliding window is accumulated at the same pixel position for each diagnosis time instant. The number of times a pixel is judged as a foam region in the window is counted. When the number of times reaches a fixed proportion of the window diagnosis time instants (for example, not less than two-thirds of the number of window diagnosis time instants), the pixel is retained as a foam region. Otherwise, it is judged as a non-foam region, and a stable foam working area is obtained.

[0068] It should be noted that the proportion threshold is obtained by taking the normal diagnosable window and the non-diagnosable window (for example, a window containing frames of blur, occlusion, overexposure, etc.) collected by field calibration as samples, counting the distribution of the proportion of qualified frames in each window, and selecting the proportion value that minimizes the sum of the number of "diagnosable windows misjudged as non-diagnosable" and the number of "non-diagnosable windows misjudged as diagnosable" as the proportion threshold; the fixed probability threshold is used to determine whether a pixel is a foam pixel when the foam probability of the pixel is not less than the probability threshold, otherwise it is determined to be a non-foam pixel, and the probability threshold is determined offline before operation by using the offline calibration data to verify the segmentation network by using the manually labeled foam region true value, and a plurality of candidate probability thresholds are traversed, and the threshold that makes the comprehensive segmentation index (for example, F1 that takes into account both missed detection and false detection) of the foam region optimal and has the least fluctuation on multiple batches of samples is selected as the probability threshold.

[0069] It should be noted that the encoding end of the lightweight U-Net semantic segmentation network extracts multi-scale features by multiple layers of convolution and downsampling, the decoding end restores boundary details by upsampling and jump connection with the corresponding scale of the encoding features, the network input is a two-channel image (total intensity image and polarization feature image spliced), and the output is a foam region probability map with the same size as the input; the training process is to collect sample images under different working conditions (normal foam, thin, viscous, strong reflection), manually or label the foam region mask as the true value for each sample, divide the samples into a training set, a validation set and a test set at a fixed ratio, perform uniform scaling and normalization on the input image, and use a weighted combination of cross-entropy and Dice loss as the optimization objective to suppress missed detection and false detection at the same time, use small batch gradient descent to iteratively update the network parameters, use F1 as the evaluation index on the validation set for early stopping and best model selection, solidify the model parameters with the optimal and stable validation index, and perform forward inference in the online stage to obtain the foam region probability map, and then binarize the foam region mask according to the pre-determined probability threshold.

[0070] In this embodiment, in order to verify the influence of the consistency fusion strategy on the stability of the foam working area extraction, foam image sequences are constructed under different disturbance intensities, and single-frame baseline, lightweight U-Net and lightweight U-Net plus consistency fusion are used for foam working area segmentation comparison, as shown in Figure 5As shown, the core evaluation metric is the adjacent frame mask, and the perturbation intensity is used to represent the fluctuation of the working condition. As the perturbation intensity increases, the cross-union ratio (CUR) of the single-frame baseline and the lightweight U-Net decreases more significantly, indicating that under strong perturbation, single-frame segmentation or relying solely on spatial semantic segmentation is prone to mask jitter and region drift. The lightweight U-Net plus consistency fusion curve is higher overall and decreases more slowly, indicating that by performing consistency fusion on the mask within the window, short-term noise and local false detections can be suppressed, and stable extraction of the foam working area can be achieved. The high perturbation interval is marked by a dashed rectangle and enlarged in the figure below. The CUR corresponding to the perturbation intensity is further marked at several feature points. At the same time, the point with the largest difference between the two curves is marked with dashed lines and arrows, and the maximum difference is given to quantify the robustness gain brought by consistency fusion. This proves that inputting the total intensity image and polarization feature map into the lightweight U-Net and combining it with consistency fusion can continuously output a stable foam working area mask under complex perturbation.

[0071] S3. Within the foam working area, extract bubble instances from the polarization feature map, and count bubble rupture and merger events within the sliding window to obtain rupture and merger rates. Calculate the average surface velocity of the foam based on the displacement of adjacent frames.

[0072] For each diagnostic moment First, the total intensity image and polarization feature map are cropped based on the foam working area, retaining only the pixels within the foam working area and setting the pixels outside the working area to zero. The polarization feature map is smoothed and denoised to suppress random noise, and the gradient magnitude map is calculated on the denoised polarization feature map, which is used as the candidate map for bubble boundaries. The total intensity image is contrast-enhanced within the foam working area to improve the difference between light and dark areas. The candidate map for bubble boundaries and the contrast-enhanced total intensity image are used together as input for bubble instance extraction.

[0073] For the enhanced total intensity image cropped to the foam working area at each diagnostic time, the working area is first divided into several grid blocks according to a fixed grid size. Within each grid block, the grayscale mean is calculated only for the effective pixels falling within the foam working area. with standard deviation and with local threshold (in (Indicates a fixed coefficient) Binarizes each grid block pixel by pixel, with grayscale values ​​not less than [value missing]. pixels are recorded as 1 and the rest are recorded as 0, to obtain a binary candidate image which is automatically adjusted according to the local brightness; a one-time opening operation and a one-time closing operation are performed and connected domains with an area less than an area threshold are deleted to obtain a continuous bubble membrane boundary candidate region; the bubble membrane boundary candidate region is inverted and limited in the foam working area to obtain a bubble interior region binary image; the boundary pixels in the bubble boundary candidate image are taken as the zero-distance boundary set of distance transformation, and Euclidean distance transformation is performed on the bubble interior region binary image to calculate the shortest Euclidean distance from each bubble interior pixel to the nearest boundary pixel in the zero-distance boundary set, and the greater the distance value, the closer to the bubble center, thereby forming a bubble center potential image; a fixed radius neighborhood maximum value search is used on the bubble center potential image to extract local maximum points as initial marking points, and points with a distance less than a distance threshold between marking points are merged to avoid repeated marking; the bubble boundary candidate image corresponding to the bubble membrane boundary candidate region is taken as a segmentation barrier line, the negative value of the bubble center potential image is taken as a watershed terrain surface, and the marking points are taken as initial seeds of the watershed to perform watershed segmentation, so that the bubble interior region is divided into a plurality of mutually non-overlapping pixel connected domains; each pixel connected domain is assigned a unique instance number to form a bubble instance set of the current frame, and the area and centroid of each instance are calculated, wherein the area is the product of the number of pixels in the connected domain and a pixel area conversion coefficient, and the centroid is the arithmetic average of the coordinates of all pixels in the connected domain.

[0074] It should be noted that the fixed coefficient is selected by traversing a plurality of candidate values on a sample sequence with qualified image quality and stable working conditions, and selecting a value that makes the bubble membrane boundary candidate region and the manually verified boundary optimal in terms of false rejection rate and missed detection rate and has the smallest cross-batch fluctuation; the area threshold is obtained by statistically analyzing the bubble instance area distribution in the same type of stable working condition samples, recording the area of each connected domain, and manually verifying and labeling "real bubble / non-bubble" for these connected domains to obtain two sets of area sample sets, and taking the area threshold as the dividing point between the minimum value of the "real bubble area sample set" and the maximum value of the "non-bubble area sample set"; the distance threshold is used to merge adjacent local maximum marking points, and by comparing the proportion of over-segmentation and under-segmentation caused by repeated marking on the labeled samples, the threshold value that makes the over-segmentation rate and under-segmentation rate comprehensive minimum is selected as the distance threshold; the pixel area conversion coefficient is obtained by camera calibration, and after the installation position is fixed, the pixel size is calibrated by a calibration board or a known size reference to obtain the actual area corresponding to a single pixel; the fixed radius is obtained by dividing the bubble instance set on the same type of stable working condition samples before commissioning, statistically analyzing the bubble instance area and converting it to obtain the equivalent diameter, taking the median of the equivalent diameter as the pixel radius of the typical bubble, and taking half of the typical bubble radius as the fixed radius of the neighborhood maximum value search.

[0075] The bubble instance set of adjacent diagnosis time is performed with deterministic matching, taking the centroid distance as the main constraint and the area overlap ratio as the secondary constraint, to establish one-to-one or many-to-one correspondence between the last diagnosis time instance and the current diagnosis time instance. Specifically, for each pair of instances in the last diagnosis time and the current diagnosis time, the Euclidean distance between the centroids is calculated, the overlap area ratio of the last diagnosis time instance and the current diagnosis time instance is calculated (i.e. the ratio of the intersection to the union), and if the centroid distance is less than the centroid distance threshold and the overlap ratio is greater than the overlap threshold, it is considered that the two instances are the same bubble.

[0076] It should be noted that the distance threshold is obtained by manually calibrating and verifying in the training sample, counting the centroid change range of the same bubble between different frames, and selecting a threshold that can better distinguish adjacent bubbles and the same bubble. Generally, the value range is between several pixels and several tens of pixels (for example, assuming that the image resolution is 1920*1080, the bubble diameter is 10 pixels, and the bubble diameter is 100 pixels, the distance threshold is 10 pixels). If the diameter of the bubble is several millimeters to several centimeters, the centroid distance threshold may be between 5 pixels and 30 pixels); the overlap threshold is obtained by analyzing the overlap degree of different bubble instances on the labeled data, calculating the area overlap ratio of each "corresponding instance pair of the same bubble in adjacent diagnosis time", and calculating the area overlap ratio of each "instance pair of different bubbles in adjacent diagnosis time". Get "same bubble overlap ratio set" and "different bubble overlap ratio set", then traverse the candidate threshold, select the overlap threshold that has the maximum discrimination between "the proportion of the overlap ratio of the same bubble instance pair not less than the overlap threshold" and "the proportion of the overlap ratio of the different bubble instance pair higher than the overlap threshold", and generally the value range is [0, 1].

[0077] If there is an instance in the last diagnosis time and no matching in the current diagnosis time, and the area of the instance is not less than the minimum area threshold, the instance is counted as a breakage event; if at least two different instances in the last diagnosis time are matched as the same instance in the current frame, and the difference between the area of the current diagnosis time instance and the sum of the areas of the participating merging instances in the last diagnosis time is not more than the tolerance threshold, it is counted as a merging event. Accumulate the number of events for each adjacent frame pair in the window to obtain the breakage event sequence and the merging event sequence.

[0078] It should be noted that the minimum area threshold is obtained by first manually labeling the area of the bubble instance and calculating the area distribution of the bubble instance in the calibration process, and selecting an area lower limit that can effectively eliminate noise instances (such as splashes and reflective fragments) smaller than the real bubble as the minimum area threshold; the tolerance threshold is obtained by analyzing the morphological changes of different bubbles between adjacent frames, and setting a tolerance threshold to avoid misjudging the instance when the bubble merges or breaks. Generally, the maximum tolerance range of the bubble size change is taken as the tolerance threshold.

[0079] The breakage rate and the merging rate are calculated by the number of events in the sliding window, specifically, the breakage rate is calculated by averaging the number of breakage events in the window, and the merging rate is calculated by averaging the number of merging events in the window; at each pair of adjacent diagnosis moments With In the above, only the dense optical flow is calculated on the total intensity image in the foam working area to obtain the pixel displacement vector field, and the spatial mean and the time mean of the displacement module length are taken as the window surface velocity mean, and the expression is:

[0080] ;

[0081] Among them, represents the foam surface velocity mean at the diagnosis moment , represents the number of diagnosis moments of the sliding window, represents the velocity vector calculated by the optical flow at the pixel at the adjacent diagnosis moment, represents the foam working area, represents the number of pixels of the foam working area, represents the time index corresponding to the current diagnosis moment .

[0082] S4, input the time sequence data of the polarization feature map, the breakage rate, the merging rate, the foam surface velocity mean and the time sequence data of the process parameters into the deep learning model, output the foam state category and the soft measurement prediction data, and construct a consistency evaluation;

[0083] The polarization feature map at each diagnosis moment in the sliding window is cropped to the foam working area with a fixed size, and the polarization feature map at each diagnosis moment is linearly normalized to limit the value range to be consistent; the process parameters in the sliding window are standardized according to the mean and standard deviation of each dimension obtained by offline calibration to obtain a standardized parameter sequence; the breakage rate, the merging rate and the foam surface velocity mean are normalized according to the fixed scale coefficient of offline calibration to obtain normalized dynamic quantities; the polarization feature map, the standardized parameter sequence and the normalized dynamic quantities are input into a deep learning model of a multi-input multi-task network using CNN+GRU, forward inference is performed, and a foam state category probability vector and soft measurement prediction data are obtained.

[0084] ​It should be noted that the fixed scale coefficient is obtained by respectively counting the numerical distribution of the breakage rate, the merging rate and the surface speed mean value on the historical samples with stable working conditions and qualified image quality, taking the median and quartile range of the numerical distribution as the normalization reference, so that the normalized quantity falls within the stable range in most normal samples, and the fixed scale coefficient is obtained; the deep learning model of the multi-input multi-task network of CNN+GRU is a two-channel image sequence composed of the total intensity image and the polarization feature map as the first input branch, the spatial features are extracted frame by frame by the lightweight convolutional network first, then the frame features are input into the gated recurrent unit GRU in time sequence to extract the time sequence dynamic features, the normalized process parameter sequence is taken as the second input branch, the parameter time sequence features are extracted by one or more GRUs, and the normalized breakage rate, merging rate and surface speed mean value are taken as the third input branch, and the features are mapped by a fully connected layer; the three features are spliced in the fusion layer and output two types of results by the fully connected layer, one is the probability vector of the foam state category, and the other is the soft measurement prediction value of the grade and recovery rate; the training process is to collect window samples covering different working conditions, each sample contains two-channel image sequence, process parameter sequence and dynamic quantity, and the manually labeled foam state is taken as the classification label and the assay grade and recovery rate are taken as the regression label, the image is unified and normalized in size, the parameters are standardized, the classification cross-entropy loss and regression mean square error loss are calculated respectively during training, and both are minimized at each iteration to update the network parameters, and the classification accuracy and regression error are simultaneously investigated on the validation set to select the optimal model and early stop, and finally the optimal parameters are fixed and deployed.

[0085] For each process parameter dimension, take the reference value of the dimension in the stable working condition calibration section and the scale constant (both offline fixed), take the absolute value of the parameter deviation at the current time after scaling by the scale, and accumulate to obtain the process parameter deviation degree, the greater the process parameter deviation degree, the more the current working condition deviates from the stable reference state.

[0086] After normalizing the breakage rate, merging rate and surface speed mean value according to the fixed scale coefficient, a consistency evaluation is constructed, the expression is:

[0087] ;

[0088] Among them, represents the consistency evaluation, represents the process parameter deviation degree, represents the breakage rate, represents the merging rate, represents the time the average of the polarization features in the foam working area, represents the consistency integral window length constant.

[0089] When When greater than or equal to the consistency threshold, the bubble state category and the soft measurement prediction data are published; when When less than the consistency threshold, the bubble state category is published but the soft measurement prediction data is suppressed.

[0090] It should be noted that the consistency threshold is calculated by selecting a batch of completed image and process parameter time-aligned historical data windows, calculating the consistency evaluation for each window, and calculating the soft measurement error (e.g., the absolute error of the calibrated grade or recovery rate and the assay value) corresponding to each window. All windows are sorted by error size from small to large, and the consistency evaluation that makes the boundary between the windows with errors within the acceptable range and the windows with errors outside the acceptable range most clear is taken as the consistency threshold. The assay value represents the true measurement result obtained after detecting the sampled material.

[0091] S5, recursively calibrating the soft measurement prediction data and triggering a drift alarm when the consistency evaluation is continuously abnormal and the calibration residual is continuously out of limit;

[0092] The grade and recovery rate in the soft measurement prediction data are respectively subjected to recursive calibration. Specifically, the difference between the prediction value and the assay value at the current time is recorded as a bias sample and stored in a fixed-length bias queue. The arithmetic mean of the biases in the bias queue is calculated as the current bias correction after each update of the bias queue. Then, the bias correction is added to the subsequent soft measurement prediction data to output the calibration result. When the bias queue is not full, the average of the existing bias samples is used. If the consistency evaluation at the time corresponding to the current assay sample is low consistency, the bias sample does not enter the queue to avoid pollution of the calibration by abnormal data.

[0093] For each completed calibration sample, the difference between the assay value and the calibrated soft measurement value is taken as the calibration residual of the grade and recovery rate. A continuous counter is maintained to determine continuous abnormality. When the consistency evaluation at the current time is lower than the consistency threshold and the grade residual exceeds the grade residual threshold or the recovery rate residual exceeds the recovery rate residual threshold, the continuous counter is incremented by one. Otherwise, the continuous counter is cleared. When the continuous counter accumulates to a set number of times, it is determined that there is a drift and the drift alarm flag is set to 1.

[0094] It should be noted that the grade residual threshold and the recovery rate residual threshold are obtained by selecting a batch of historical windows with stable working conditions and qualified image quality, calculating the calibration residual sequence according to the same calibration process, respectively counting the distribution of the grade residual and the recovery rate residual, and taking the high quantile (for example, the 95% quantile) of the absolute residual as the grade residual threshold and the recovery rate residual threshold; the set number is obtained by selecting a batch of historical alignment data before operation, calculating the consistency evaluation, the grade calibration residual and the recovery rate calibration residual at each time, marking the drift occurrence section by manual or production record, and playing back the candidate integer (for example, starting from 1 and increasing) one by one to record the time when the drift alarm is triggered for the first time in the marked drift section and the number of times it is triggered in the non-drift section, and selecting the integer that minimizes the comprehensive minimum of the alarm delay and the number of false alarms as the set number.

[0095] In order to verify the influence of the three-way full-input multi-input multi-task network (CNN+GRU) based on polarization feature map time series data, normalized dynamic quantity and standardized parameter sequence on the prediction accuracy of online grade soft measurement, corresponding time series samples are collected under disturbance working conditions, and the online prediction results are aligned and compared with the assay grade in the window number dimension, as shown in Figure 6 The upper graph shows the soft measurement prediction curve and the assay grade in the same time series, which are aligned in the window number dimension, and can intuitively reflect the tracking ability of online prediction on discrete assay results; the enlarged window marked by the dashed rectangle is used to highlight a fluctuation interval, which facilitates observation of the response and stability of the prediction curve when the disturbance or foam state changes significantly; the lower graph is a local enlarged view, which reflects the retention ability of the prediction curve on the key fluctuation structure; the vertical dashed line and the bidirectional arrow mark the window with the largest difference between the assay grade and the soft measurement prediction, and give the maximum difference, which facilitates the quantification of the upper bound of the deviation under the most unfavorable working condition; the continuous output in the time dimension and the alignment with the assay point can not only verify the online linearity with the overall trend of the curve, but also verify the error control ability of the key window with local amplification and maximum difference marking.

[0096] When the drift alarm flag is equal to 1, the sliding window sample corresponding to the trigger time is taken out from the cache according to a fixed length and solidified as a retraining sample record, and the retraining sample record includes the polarization feature map sequence in the sliding window, the breakage rate, the merger rate, the average foam surface speed at the end of the window, the process parameter sequence and the consistency evaluation; the retraining sample record is used to update the deep learning model.

[0097] The embodiment also provides a computer device suitable for the image recognition-based on-line diagnosis method of mineral processing froth state, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the image recognition-based on-line diagnosis method of mineral processing froth state proposed in the above embodiment.

[0098] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0099] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to realize the image recognition-based on-line diagnosis method of mineral processing froth state proposed in the above embodiment. The storage medium can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.

[0100] To sum up, the present application realizes the stable extraction of the foam working area by inputting the total intensity image and the polarization feature map into the lightweight U-Net semantic segmentation network and performing consistent fusion on the mask in the window to generate the foam working area; and realizes the online soft measurement prediction of the grade and the recovery rate by inputting the polarization feature map time series data, the normalized dynamic quantity and the standardized parameter sequence into the multi-input multi-task network of CNN+GRU and outputting the foam state category and the soft measurement prediction data.

[0101] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. An online diagnostic method for mineral processing foam status based on image recognition, characterized in that: include, Two foam images with different polarization angles were acquired, and process parameters were acquired simultaneously to establish a sliding window; Polarization difference processing is performed on the polarization angle foam image within the sliding window to obtain a polarization feature map. The image quality of the polarization feature map is then self-checked to generate the foam working area. Within the foam working area, bubble instances are extracted from the polarization feature map, and bubble rupture and merger events are counted within the sliding window to obtain the rupture rate and merger rate. The average velocity of the foam surface is calculated based on the displacement of adjacent frames. The time-series data of polarization feature maps, rupture rate, coalescence rate, average foam surface velocity, and time-series data of process parameters are input into a deep learning model to output foam state category and soft measurement prediction data, and construct a consistency evaluation. Perform recursive calibration on soft measurement prediction data and trigger a drift alarm when the consistency evaluation is continuously abnormal and the calibration residuals continue to exceed the limit. The steps involve performing polarization difference processing on the polarization angle foam image within the sliding window to obtain a polarization feature map, performing image quality self-checks on the polarization feature map, and generating the foam working area. Based on the sliding window dataset, two foam images with different polarization angles in the same field of view are calculated pixel by pixel to obtain the total intensity image and polarization feature map; For each diagnostic moment, the sharpness, brightness deviation, and overexposure ratio are calculated, and a quality score is constructed. The percentage of qualified moments is calculated for all diagnostic moments within the sliding window. When the proportion of qualified moments is greater than the proportion threshold, the total intensity image and polarization feature map are fed into the lightweight U-Net semantic segmentation network as input to obtain the foam region probability map, which is then binarized according to a fixed probability threshold to form a foam region mask. The masks within the window are then uniformly fused to obtain the foam working area. The total intensity image and polarization feature map are represented as follows: ； ； in, Indicates at the time of diagnosis Total intensity image; The polarization angle of the sampled data is... The pixel intensity values ​​of the foam image. The polarization angle of the sampled data is... The pixel intensity values ​​of the foam image. Indicates at the time of diagnosis The polarization feature image pixel values, Represents a small positive number. Indicates the first [number]th ... A timestamp at the moment of diagnosis.

2. The online diagnostic method for mineral processing foam state based on image recognition as described in claim 1, characterized in that: The specific steps for acquiring two foam images with different polarization angles and simultaneously acquiring process parameters, and establishing a sliding window, are as follows: Send a hard trigger signal to an industrial camera to acquire a raw polarization image composed of a subset of pixels with different polarization angles in the same field of view; Perform pixel rearrangement and aggregation on the original polarization image to obtain two images with different polarization angles; Read the process parameter vector and select the pair of polarization images and process parameter vectors with the shortest acquisition timestamps from the diagnostic time. When the time difference between the acquisition timestamp and the diagnosis time does not exceed the time difference threshold, a pair of polarization images and process parameter vectors are written into a sliding window to form a sliding window dataset.

3. The online diagnostic method for mineral processing foam state based on image recognition as described in claim 2, characterized in that: Within the foam working area, bubble instances are extracted based on polarization feature maps, and bubble rupture and merger events are statistically analyzed within a sliding window to obtain rupture and merger rates. The average surface velocity of the foam is calculated based on the displacement at adjacent diagnostic times. The specific steps are as follows: The total intensity image and polarization feature map are cropped based on the foam working area. The polarization feature map is smoothed and denoised, and the gradient magnitude map is calculated. The gradient magnitude map is used as a candidate map for bubble boundary. Contrast enhancement is performed on the total intensity image within the foam working area. Local thresholding is used to binarize pixel by pixel and opening and closing operations are combined to obtain candidate regions of the bubble boundary. The candidate regions of the bubble boundary are then inverted to obtain a binary map of the bubble interior region and Euclidean distance transformation is performed to form the potential energy map of the bubble center. After obtaining initial marker points on the potential energy map at the center of the bubble by searching for neighborhood maxima and merging them, the internal region of the bubble is divided into pixel connected domains by watershed segmentation and assigned a unique instance number to form the bubble instance set of the current frame, and the area and centroid are calculated.

4. The online diagnostic method for mineral processing foam state based on image recognition as described in claim 3, characterized in that: The process involves statistically analyzing bubble rupture and merger events within a sliding window, obtaining rupture and merger rates, and calculating the average surface velocity of the foam based on displacements at adjacent diagnostic times. The specific steps are as follows: For the set of bubble instances at adjacent diagnostic times, deterministic matching is performed with centroid distance as the primary constraint and region overlap ratio as the secondary constraint, and bubble bursting events and merger events are counted. The breakup rate and merger rate are calculated by the number of events within the sliding window; At each pair of adjacent diagnostic moments, dense optical flow is calculated only on the total intensity image within the foam working area to obtain the pixel displacement vector field, and the spatial mean and temporal mean of the displacement modulus are taken as the average velocity of the foam surface.

5. The online diagnostic method for mineral processing foam state based on image recognition as described in claim 4, characterized in that: The process involves inputting time-series data of polarization feature maps, bursting rate, coalescence rate, average foam surface velocity, and time-series data of process parameters into a deep learning model. The model outputs foam state categories and soft-sensor prediction data, and constructs polarization features and consistency evaluation data. The specific steps are as follows: The polarization feature map is cropped to the foam working area at a fixed size. The process parameters in the sliding window are standardized according to the mean and standard deviation of each dimension to obtain the standardized parameter sequence. The bursting rate, coalescence rate and foam surface velocity mean are normalized according to a fixed scale coefficient to obtain the normalized dynamic quantity. The polarization feature map, standardized parameter sequence, and normalized dynamic input are input into a deep learning model with a CNN+GRU multi-input multi-task network. Forward inference is then performed to obtain the probability vector of the foam state category and the soft measurement prediction data.

6. The online diagnostic method for mineral processing foam state based on image recognition as described in claim 5, characterized in that: The consistency evaluation is constructed, and the specific steps are as follows: For each process parameter dimension, the absolute value of the parameter deviation is normalized according to the scale and then accumulated to obtain the process parameter deviation degree. Based on the deviation of process parameters, the mean values ​​of fracture rate, agglomeration rate and surface velocity are normalized according to a fixed scaling factor to construct a consistency evaluation. When the consistency evaluation is greater than or equal to the consistency threshold, the bubble state category and soft measurement prediction data are released; when the consistency evaluation is less than the consistency threshold, the bubble state category is released but the soft measurement data is suppressed.

7. The online diagnostic method for mineral processing foam state based on image recognition as described in claim 6, characterized in that: The specific steps for recursively calibrating the soft measurement prediction data and triggering a drift alarm when the consistency evaluation remains abnormal and the calibration residuals continue to exceed limits are as follows: Recursive calibration was performed on grade and recovery rate in the soft measurement prediction data, and the difference between the test value and the calibrated soft measurement value was used as the calibration residual for grade and recovery rate. Maintain a continuous counter. When the current consistency evaluation is lower than the consistency threshold and the grade residual exceeds the grade residual threshold or the recovery rate residual exceeds the recovery rate residual threshold, increment the continuous counter by one; otherwise, clear the continuous counter to zero. When the continuous counter accumulates to a set number of times, it is determined to be a drift and a drift alarm flag is set.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the online diagnostic method for mineral processing foam state based on image recognition as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the online diagnostic method for mineral processing foam state based on image recognition as described in any one of claims 1 to 7.

Images