An online concentration monitoring method based on filling slurry image features

Abstract

The present invention provides an online concentration monitoring method based on the image features of filling slurry, which belongs to the field of mine filling technology. The method adopts a dual-resolution slurry detection sensor to clearly image the mesoscale range and collect static images and video stream data during the slurry stirring process. A semiconductor refrigeration and high-pressure air dual cooling device is arranged at the lens end of the slurry detection sensor. A comprehensive analysis is made of the relationship between the shallow common features of natural images, the multi-scale features of mesoscale images and the video optical flow features and concentration, and a filling slurry concentration monitoring model based on image features and video optical flow features is established. In production, the slurry concentration in the stirring tank is measured in real time by real-time images. The method combines machine vision, deep learning and artificial intelligence technologies to quantitatively characterize the qualitative features of the mesoscale images and video data of the slurry, providing a good reference for the intelligent development of filling mining.

Description

Technical Field

[0001] The present invention relates to the technical field of mine filling, and in particular to an online concentration monitoring method based on filling slurry image features. Background Art

[0002] Cemented backfill technology in metal mines has evolved through several stages, from water-sand backfill to high-concentration backfill. Slurry concentration has been a core parameter throughout the development of cemented backfill mining technology. Precise control of slurry concentration is crucial during backfill slurry preparation, as the slurry's moisture content plays a decisive role in its flow properties and strength.

[0003] The filling concentration has a significant impact on the flow properties and conveying performance of the filling slurry. During on-site filling, slump is often used to characterize the flow properties of the slurry. The larger the slump, the better the slurry flow. Related studies have shown that fluidity decreases with increasing slurry mass concentration, and there is a critical concentration point that causes the rate of decrease in fluidity to be faster when the critical concentration is exceeded. Yield stress and apparent viscosity are the core parameters for calculating the pipeline resistance of filling slurry. The greater the yield stress and apparent viscosity, the greater the pipeline transportation resistance and the more difficult the slurry is to be transported. The yield stress and apparent viscosity increase with increasing concentration. Similarly, there is a critical concentration point that causes the rheological parameters to increase more. Therefore, during the slurry preparation process, its concentration must be accurately measured and controlled online in real time, otherwise it will seriously threaten the safe operation of the filling system. Secondly, the filling concentration has a significant impact on the strength and durability of the filling body formed by the slurry. The strength of the hardened slurry determines its support for the surrounding rock in the void. The weaker the filling strength, the less support it provides for the pillars and surrounding rock, which is detrimental to mine support and can easily lead to geological hazards such as impact depressions and surface collapse. High filling concentrations also reduce the fluidity of the slurry, making pipeline transportation difficult. Therefore, precise measurement and control of filling concentrations are necessary to keep fluctuations within a reasonable range.

[0004] Accurate measurement of filling concentration is the prerequisite for achieving precise concentration control. Existing online concentration measurement methods such as nuclear density meters, ultrasonic density meters, microwave density meters and other equipment have great limitations and have high requirements for the measured medium (such as conductivity, strong polarity, etc.). This has led to a lack of online concentration measurement equipment for filling slurry, and it is impossible to quickly and accurately obtain the real-time concentration value of the slurry. With the rapid development of machine learning, image feature algorithms have become an important monitoring method and have gradually been widely used in various fields (such as artificial intelligence, smoke monitoring, etc.). Based on this, a non-contact filling slurry concentration monitoring method based on filling slurry image features is proposed. This method has wide adaptability and can monitor the slurry concentration in real time. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an online concentration monitoring method based on the image characteristics of filling slurry. Through machine learning, the visual differences in the stirring fluidity of slurries of different concentrations are identified, a slurry video concentration analysis mode is designed, and the video optical flow characteristics of slurry transportation are extracted, thereby achieving more accurate concentration analysis and prediction, and ultimately providing a reference for the intelligent development of filling mining.

[0006] The method comprises the following steps:

[0007] S1: Data acquisition: During the preparation and stirring of the filling slurry, a visual mesoscale slurry detection sensor is used to capture the slurry surface at the front, middle, and rear positions of the stirring shaft in the stirring tank. Natural images, mesoscale images, and video stream data of the slurry stirring process under different feeding conditions are collected to form visual perception features related to slurry concentration (particle size distribution, texture, etc.) and motion features (flow rate and flow direction, etc.);

[0008] S2: Feature extraction: Convolutional neural networks are used to extract features from the image and video stream data collected in S1, respectively, to obtain shallow common features of natural images, multi-scale features of mesoscale images, and optical flow features of videos;

[0009] S3: Concentration calibration: extract the slurry in the stirring tank under different feeding conditions, measure the concentration by drying method, and obtain the concentration reference value of the slurry under different feeding conditions;

[0010] S4: The shallow common features of natural images and the multi-scale features of mesoscale images are integrated through the FPN (feature pyramid networks) network structure, and the transfer learning of natural image features is realized through the pre-training model. At the same time, the relationship between the optical flow features of images and videos and the slurry concentration is comprehensively analyzed, and the slurry concentration analysis model pre-training technology based on small sample image transfer learning is used to establish a filling slurry concentration monitoring model based on image features;

[0011] S5: Filler slurry concentration monitoring: Under industrial production conditions, the filler slurry concentration is monitored through the real-time image obtained in S1 according to the filler slurry concentration monitoring model established in S4.

[0012] The core of this monitoring method lies in interpreting video optical flow characteristics as particle velocity and image characteristics as particle distribution. Changes in concentration affect particle velocity and distribution during mixing. A neural network is trained using optical flow and image characteristics as independent variables and concentration as the dependent variable to generate a predictive model. This predictive model determines the concentration of the slurry by identifying both optical flow and image characteristics.

[0013] The main body of the slurry detection sensor in S1 is an industrial camera and an anti-shake lens to form a dual-resolution imaging system, which specifically includes a camera body, a dustproof cover, a power interface, a light source array, polarized optical glass and a high-pressure air curtain outlet. A dustproof cover is provided on the outside of the camera body, and a power interface is provided on one side of the camera body. A camera power supply, a semiconductor radiator, an industrial high-definition camera, an industrial anti-shake medium-focus lens, and an industrial anti-shake telephoto lens are provided inside the camera body. A light source array composed of light sources is provided above the industrial anti-shake medium-focus lens and the industrial anti-shake telephoto lens. Polarized optical glass is provided directly in front of the industrial anti-shake medium-focus lens and the industrial anti-shake telephoto lens. A high-pressure air curtain outlet is provided in front of the polarized optical glass, and the high-pressure air curtain outlet is connected to the high-pressure air input switch.

[0014] When photographing the slurry surface in S1, daylight and fill light were used to maintain brightness, and the shooting distance was 20 cm to 30 cm from the liquid surface.

[0015] The convolutional neural network in S2 contains five convolutional layers and one pooling layer, where:

[0016] Convolutional layer: A three-channel convolutional layer is used to extract image features. The convolution kernels of the five convolutional layers are all 3×3 pixels, the sliding step size is 1, and the padding type is SAME.

[0017] Pooling layer: The image features after convolutional layer processing are high in dimension, so the pooling layer performs dimensionality reduction. The image is divided into disjoint blocks and the block maximum value is calculated, i.e., max pooling. The kernel size is 3×3 pixels, the sliding step is 4, and the padding type is VALID.

[0018] Activation function: In order to improve the nonlinear interpretation ability of the convolution layer and the pooling layer and improve the convergence of the model, the activation function is calculated after the operation of the convolution layer and the pooling layer. The ReLu function (Max(0, x)) is used as the activation function to improve the fault tolerance of the convolutional network model.

[0019] The optical flow feature calculation of the video in S2 adopts the LiteFlowNet-en (based on PIV deep optical flow neural network) model which is improved based on LiteFlowNet (optical flow neural network) and FlowNetS.

[0020] Transfer learning in S4 is based on a pre-trained model. The pre-trained model selected is a Keras model with pre-trained weights for prediction, feature extraction, and fine-tuning.

[0021] The small-sample learning method used in S4 is a twin network, which employs a two-way network structure to construct different pairs of samples through combination and input them into the network for training. The top-level network determines the sample category and outputs a probability distribution. During operation, the twin network processes each set of test samples and support samples, ultimately outputting the category with the highest probability in the support set as the result.

[0022] The slurry detection sensor can clearly image the mesoscale range of um to cm. It can not only obtain static parameters such as particle size distribution and uniformity, but also dynamically analyze the flow velocity and direction of the paste liquid surface.

[0023] The LiteFlowNet-en model consists of two parts: NetC and NetE. NetC is used to convert each frame of the video into high-dimensional features using shared convolution kernels and network weights. Its main structure is a pyramid. NetE is used to infer the resolution of high-dimensional features and ultimately form the motion between pixels. Its main structure is a cascaded inference structure. Its calculation formula is as follows:

[0024] X 2 =f(F3(I2),F3(I1),X 3 )

[0025] Where: f represents the operation function at the NetC pyramid layer; the superscripts 2 and 3 represent the 2nd and 3rd layer networks respectively; X represents the speed; I i Represents the optical flow features of a frame image in the video, where i = 1, 2…, i represents the number of optical flow features; F k Represents the level of the pyramid, where k = 1, 2, 3, ..., the smaller k is, the lower the level of the pyramid.

[0026] When using the LiteFlowNet-en model to calculate optical flow features, the captured video file needs to be processed to form an optical flow file consisting of t frames and t+1 frames of images, namely (optical flow file).FLO.

[0027] The beneficial effects of the above technical solution of the present invention are as follows:

[0028] In the above scheme, by identifying the apparent characteristics of filling slurries of different concentrations and combining advanced technologies such as machine vision, deep learning and artificial intelligence, the qualitative characteristics of the mesoscale images and video data of the filling slurry are quantitatively characterized, thereby converting the image and video information into concentration data. It has outstanding advantages such as high visualization, strong operability and strong timeliness, and will provide a good reference for the intelligent development of filling mining. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1This is a flow chart of the online concentration monitoring method based on filling slurry image features of the present invention;

[0030] Figure 2 Schematic diagram of the structure of the slurry detection sensor in the online concentration monitoring method based on the filling slurry image feature of the present invention;

[0031] Figure 3 Schematic diagram of the convolutional neural network structure for extracting image features in the online concentration monitoring method based on filling slurry image features of the present invention;

[0032] Figure 4 This is a flow chart of a small sample learning method (Twin Network) in the online concentration monitoring method based on filling slurry image features of the present invention;

[0033] Figure 5 Schematic diagram of the optical flow feature extraction process in the online concentration monitoring method based on filling slurry image features of the present invention.

[0034] Among them: 1-camera body; 2-dust cover; 3-power interface; 4-camera power supply; 5-semiconductor heat sink; 6-high-voltage air input switch; 7-industrial high-definition camera; 8-industrial anti-shake medium-focus lens; 9-industrial anti-shake telephoto lens; 10-light source array; 11-light source; 12-polarized optical glass; 13-high-pressure air curtain outlet. DETAILED DESCRIPTION

[0035] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0036] The present invention provides an online concentration monitoring method based on filling slurry image characteristics.

[0037] like Figure 1 As shown, the method includes the following steps:

[0038] S1: Data acquisition: During the preparation and stirring of the filling slurry, a visual mesoscale slurry detection sensor is used to capture the slurry surface at the front, middle, and rear positions of the stirring shaft in the stirring tank. Natural images, mesoscale images, and video stream data of the slurry stirring process under different feeding conditions are collected to form visual perception features related to slurry concentration (particle size distribution, texture, etc.) and motion features (flow rate and flow direction, etc.);

[0039] S2: Feature extraction: Convolutional neural networks are used to extract features from the image and video stream data collected in S1, respectively, to obtain shallow common features of natural images, multi-scale features of mesoscale images, and optical flow features of videos;

[0040] S3: Concentration calibration: extract the slurry in the stirring tank under different feeding conditions, measure the concentration by drying method, and obtain the concentration reference value of the slurry under different feeding conditions;

[0041] S4: The shallow common features of natural images and the multi-scale features of mesoscale images are integrated through the FPN (feature pyramid networks) network structure, and the transfer learning of natural image features is realized through the pre-training model. At the same time, the relationship between the optical flow features of images and videos and the slurry concentration is comprehensively analyzed, and the slurry concentration analysis model pre-training technology based on small sample image transfer learning is used to establish a filling slurry concentration monitoring model based on image features;

[0042] S5: Filler slurry concentration monitoring: Under industrial production conditions, the filler slurry concentration is monitored through the real-time image obtained in S1 according to the filler slurry concentration monitoring model established in S4.

[0043] like Figure 2 As shown, the main body of the slurry detection sensor in S1 is an industrial camera and an anti-shake lens to form a dual-resolution imaging system, specifically including a camera body 1, a dust cover 2, a power interface 3, a light source array 10, polarized optical glass 12 and a high-pressure air curtain outlet 13. The dust cover 2 is provided on the outside of the camera body 1, and the power interface 3 is provided on one side of the camera body 1. The camera power supply 4, a semiconductor radiator 5, an industrial high-definition camera 7, an industrial anti-shake medium-focus lens 8, and an industrial anti-shake telephoto lens 9 are provided inside the camera body 1. A light source array 10 composed of a light source 11 is provided above the industrial anti-shake medium-focus lens 8 and the industrial anti-shake telephoto lens 9. Polarized optical glass 12 is provided directly in front of the industrial anti-shake medium-focus lens 8 and the industrial anti-shake telephoto lens 9. A high-pressure air curtain outlet 13 is provided in front of the polarized optical glass 12. The high-pressure air curtain outlet 13 is connected to the high-pressure air input switch 6.

[0044] When photographing the slurry surface in S1, daylight and fill light were used to maintain brightness, and the shooting distance was 20 cm to 30 cm from the liquid surface.

[0045] like Figure 3 , the convolutional neural network in S2 contains five convolutional layers and one pooling layer, where:

[0046] Convolutional layer: A three-channel convolutional layer is used to extract image features. The convolution kernels of the five convolutional layers are all 3×3 pixels, the sliding step size is 1, and the padding type is SAME.

[0047] Pooling layer: The image features after convolutional layer processing are high in dimension, so the pooling layer performs dimensionality reduction. The image is divided into disjoint blocks and the block maximum value is calculated, i.e., max pooling. The kernel size is 3×3 pixels, the sliding step is 4, and the padding type is VALID.

[0048] Activation function: In order to improve the nonlinear interpretation ability of the convolution layer and the pooling layer and improve the convergence of the model, the activation function is calculated after the operation of the convolution layer and the pooling layer. The ReLu function (Max(0, x)) is used as the activation function to improve the fault tolerance of the convolutional network model.

[0049] The optical flow feature calculation of the video in S2 adopts the LiteFlowNet-en (based on PIV deep optical flow neural network) model which is improved based on LiteFlowNet (optical flow neural network) and FlowNetS.

[0050] Transfer learning in S4 is based on a pre-trained model. The pre-trained model selected is a Keras model with pre-trained weights for prediction, feature extraction, and fine-tuning.

[0051] The small sample learning method used in S4 is the twin network (such as Figure 4 ), employing a dual-path network structure, constructs different pairs of samples through combination and inputs them into the network for training. The top-level network determines the sample category and outputs a probability distribution. During operation, the twin network processes each set of test samples and support samples, ultimately outputting the category with the highest probability in the support set as the result.

[0052] The main body of the slurry detection sensor is an industrial camera 7, which is combined with an industrial anti-shake medium-focus lens 8 and an industrial anti-shake telephoto lens 9 to form a dual-resolution imaging system. It can clearly image the mesoscale range of um to cm, and can not only obtain static parameters such as particle size distribution and uniformity, but also dynamically analyze the flow velocity and direction of the paste liquid surface.

[0053] The LiteFlowNet-en model consists of two parts: NetC and NetE. NetC is used to convert each frame of the video into high-dimensional features using shared convolution kernels and network weights. Its main structure is a pyramid. NetE is used to infer the resolution of high-dimensional features and ultimately form the motion between pixels. Its main structure is a cascaded inference structure. Its calculation formula is as follows:

[0054] X 2 =f(F3(I2),F3(I1),X 3 )

[0055] Where: f represents the operation function at the NetC pyramid layer; the superscripts 2 and 3 represent the 2nd and 3rd layer networks respectively; X represents the speed; I i Represents the optical flow features of a frame image in the video, where i = 1, 2…, i represents the number of optical flow features; F kRepresents the level of the pyramid, where k = 1, 2, 3, ..., the smaller k is, the lower the level of the pyramid.

[0056] like Figure 5 When using the LiteFlowNet-en model to calculate the optical flow features, the captured video file needs to be processed to form an optical flow file consisting of t frames and t+1 frames of images, namely (optical flow file).FLO.

[0057] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. An online concentration monitoring method based on filling slurry image features, characterized in that: The steps are as follows: S1: Data acquisition: During the preparation and stirring of the filling slurry, a visual mesoscale slurry detection sensor is used to capture the slurry surface at the front, middle, and rear positions of the stirring shaft in the stirring tank. Natural images, mesoscale images, and video stream data of the slurry stirring process under different feeding conditions are collected to form visual perception features and motion feature representations related to the slurry concentration; S2: Feature extraction: Convolutional neural networks are used to extract features from the image and video stream data collected in S1, respectively, to obtain shallow common features of natural images, multi-scale features of mesoscale images, and optical flow features of videos; S3: Concentration calibration: extract the slurry in the stirring tank under different feeding conditions, measure the concentration by drying method, and obtain the concentration reference value of the slurry under different feeding conditions; S4: The shallow common features of natural images and the multi-scale features of mesoscale images are integrated through the FPN network structure, and transfer learning of natural image features is achieved through a pre-trained model. At the same time, the relationship between the optical flow features of images and videos and slurry concentration is comprehensively analyzed. The slurry concentration analysis model pre-training technology based on small sample image transfer learning is used to establish a filling slurry concentration monitoring model based on image features and video optical flow features. S5: Filler slurry concentration monitoring: Under industrial production conditions, the filler slurry concentration is monitored through the real-time image obtained in S1 according to the filler slurry concentration monitoring model established in S4.

2. The online concentration monitoring method based on filling slurry image features according to claim 1 is characterized in that: The main body of the slurry detection sensor in S1 is an industrial camera and an anti-shake lens to form a dual-resolution imaging system, specifically including a camera body, a dustproof cover, a power interface, a light source array, polarized optical glass and a high-pressure air curtain outlet. A dustproof cover is provided on the outside of the camera body, a power interface is provided on one side of the camera body, a camera power supply, a semiconductor radiator, an industrial high-definition camera, an industrial anti-shake medium-focus lens, and an industrial anti-shake telephoto lens are provided inside the camera body, a light source array composed of light sources is provided above the industrial anti-shake medium-focus lens and the industrial anti-shake telephoto lens, polarized optical glass is provided directly in front of the industrial anti-shake medium-focus lens and the industrial anti-shake telephoto lens, a high-pressure air curtain outlet is provided in front of the polarized optical glass, and the high-pressure air curtain outlet is connected to the high-pressure air input switch.

3. The online concentration monitoring method based on filling slurry image features according to claim 1 is characterized in that: When photographing the slurry surface in S1, daylight and fill light are used to maintain brightness, and the photographing distance is 20 cm to 30 cm from the liquid surface.

4. The online concentration monitoring method based on filling slurry image features according to claim 1 is characterized in that: The convolutional neural network in S2 contains five convolutional layers and one pooling layer, where: Convolutional layer: A three-channel convolutional layer is used to extract image features. The convolution kernels of the five convolutional layers are all 3×3 pixels, the sliding step size is 1, and the padding type is SAME. Pooling layer: The image features after convolutional layer processing are high in dimension, so the pooling layer performs dimensionality reduction. The image is divided into disjoint blocks and the block maximum value is calculated, i.e., max pooling. The kernel size is 3×3 pixels, the sliding step is 4, and the padding type is VALID. Activation function: In order to improve the nonlinear interpretation ability of the convolution layer and the pooling layer and improve the convergence of the model, the activation function is calculated after the operation of the convolution layer and the pooling layer. The ReLu function (Max(0, x)) is used as the activation function to improve the fault tolerance of the convolutional network model.

5. The online concentration monitoring method based on filling slurry image features according to claim 1 is characterized in that: The optical flow feature calculation of the video in S2 adopts the LiteFlowNet-en model improved based on LiteFlowNet and FlowNetS.

6. The online concentration monitoring method based on filling slurry image features according to claim 1 is characterized in that: The transfer learning in S4 is performed based on a pre-trained model. The pre-trained model selected is a Keras model with pre-trained weights for prediction, feature extraction, and fine-tuning.

7. The online concentration monitoring method based on filling slurry image features according to claim 1 is characterized in that: The small sample learning method used in S4 is a twin network, that is, a two-way network structure is adopted to construct different paired samples by combination, and input them into the network for training; wherein, the network at the top layer judges the sample category and outputs the probability distribution; during the operation, the twin network will process each group of test samples and support samples, and finally the twin network will output the category with the highest probability on the support set as the result.

8. The online concentration monitoring method based on filling slurry image features according to claim 2, characterized in that: The slurry detection sensor can clearly image the mesoscale range of um to cm, can obtain particle size distribution and uniformity parameters, and can dynamically analyze the flow velocity and flow direction of the paste liquid surface.

9. The online concentration monitoring method based on filling slurry image features according to claim 5, characterized in that: The LiteFlowNet-en model consists of two parts: NetC and NetE. NetC is used to convert each frame of the video into high-dimensional features using shared convolution kernels and network weights. Its main structure is a pyramid. NetE is used to infer the resolution of high-dimensional features and ultimately form the motion between pixels. Its main structure is a cascaded inference structure. Its calculation formula is as follows: X 2 =f(F3(I2),F3(I1),X 3 ) Where: f represents the operation function at the NetC pyramid layer; the superscripts 2 and 3 represent the 2nd and 3rd layer networks respectively; X represents the speed; I i Represents the optical flow features of a frame image in the video, where i = 1, 2…, i represents the number of optical flow features; F k Represents the level of the pyramid, where k = 1, 2, 3, ..., the smaller k is, the lower the level of the pyramid.

10. The online concentration monitoring method based on filling slurry image features according to claim 9, characterized in that: When using the LiteFlowNet-en model to calculate optical flow features, the captured video file needs to be processed to form an optical flow file consisting of t frames and t+1 frames of images.

Images