A visual language model guided method and system for monitoring the concentration of a paste online

Abstract

The application provides a visual language model guided paste concentration online monitoring method and system, and belongs to the technical field of industrial process monitoring and intelligent soft measurement. First, the video stream of the paste surface in the stirring tank is collected and preprocessed to retain natural disturbance to improve the robustness of the model. Then, the video segments are cut and sampled across the recording segments to construct concentration distinguishable sample pairs. Second, the visual language model is used to determine the concentration of the sample pairs and generate soft sequence probability labels. Third, the lightweight concentration sequence score prediction student model is trained through pair-wise ranking loss learning and distillation. Finally, the output space of the student model is mapped to the absolute concentration value through sparse monotonic calibration, and the model can be deployed on the edge for real-time monitoring. The application can realize non-contact online monitoring of paste concentration without large-scale accurate labeled data set, and solves the problems of difficult edge deployment of large models and dependence on manual labeling in traditional monitoring.

Description

Technical Field

[0001] This invention relates to the field of industrial process monitoring and intelligent soft measurement technology, and in particular to a method and system for online monitoring of paste concentration guided by a visual language model. Background Technology

[0002] Cement paste backfilling (CPB) is a commonly used tailings disposal and goaf support technology in green mining. The paste is prepared at a surface station and continuously mixed with tailings, binder, and water using a double-helix horizontal mixer before being transported to the underground goaf. Paste concentration, referring to the solid content, is a key indicator determining the fluidity and strength of the backfill. A low concentration leads to insufficient backfill strength, while a high concentration reduces fluidity and can cause pipeline blockages and unplanned downtime. Therefore, online concentration monitoring of the paste during CPB backfilling is crucial. Currently, commonly used online concentration monitoring methods in industrial settings include ultrasonic instruments and nuclear density or concentration meters. However, the accuracy of ultrasonic measurement tends to decrease under conditions such as high concentration, air bubble entrainment, or non-uniform flow. Nuclear density meters use principles such as gamma ray attenuation to estimate density or water content, and are usually more stable under high concentration conditions. However, the testing equipment is expensive and involves requirements for radiation source safety, transportation, shielding, and regulation. In addition, the delivery pipeline needs to be kept full, and voids or air bubbles will introduce significant deviations, limiting its applicability in paste delivery scenarios.

[0003] With the widespread use of industrial cameras and the development of computer vision technology, video-based non-contact soft measurement has become a promising alternative. Existing research shows a significant correlation between surface texture statistics of pastes and mixing uniformity and rheological properties, which can serve as visual proxy variables for the filling process state. To characterize more complex morphological patterns, deep models such as semantic segmentation are used to identify non-uniform regions such as agglomeration and bubbles, and to quantify mixing quality. However, high-capacity models often rely on a large number of high-precision annotations. While sampling and drying methods are considered benchmark methods for solid content measurement, they are costly and time-consuming, making them unsuitable for providing dense labels for large-scale videos. Therefore, directly training concentration regression models using supervised learning is difficult to implement in engineering. In recent years, semi-supervised, weakly supervised, and unsupervised learning paradigms have been used to reduce the dependence of industrial vision models on manual annotation. Simultaneously, using visual language models as automatic annotators to generate synthetic supervision signals and distilling them into lightweight networks has become an important path to solve the problem of scarce annotations. Therefore, guiding distillation from visual language models to lightweight network models has become an important means to solve the problem of online visual monitoring of paste concentration. Summary of the Invention

[0004] To address the problems in the prior art, this invention provides a method and system for online monitoring of paste concentration guided by a visual language model. This invention first utilizes a visual language model to perform a binary comparison of unlabeled video clip pairs, generating a soft-order supervisory signal with confidence. Then, the comparative knowledge is distilled into a lightweight video model that can be deployed at the edge. Finally, combined with a small number of calibrated samples, real-time quantitative output of paste concentration, i.e., solid phase mass fraction, is achieved. To achieve the above objective, the technical solution is as follows:

[0005] On one hand, the present invention provides a method for online monitoring of ointment concentration guided by a visual language model, the method comprising:

[0006] S1. An industrial camera is fixed above the paste mixing tank, and supplementary lighting is provided by lighting equipment to obtain the original video stream dataset of the paste surface;

[0007] S2. Based on the original video stream dataset of the ointment surface, extract video segments to obtain a video segment dataset without concentration labels.

[0008] S3. Based on the dataset of video clips without concentration labels, a student model for predicting ordinal scores is obtained through distillation training of a visual language model.

[0009] S4. Based on the ordinal score, predict the student model and obtain a student model that can predict the true concentration through sparse monotonic calibration.

[0010] S5. Based on the student model that can predict the true concentration, a lightweight real-time inference model is obtained by deploying it on an edge device.

[0011] Optionally, in S3, based on the unlabeled video clip dataset, a visual language model is trained via distillation to obtain a student model for predicting ordinal scores, including:

[0012] S31. Based on the video clip dataset without concentration labels, extract the clip pairs with concentration differences to obtain a dataset of clip pairs with concentration differences.

[0013] S32. Based on this fragment, the dataset is compared using a visual language model to obtain a soft-order probability label dataset that represents the confidence level of the concentration comparison results.

[0014] S33. Based on the fragment dataset and the soft-order probability label dataset, a student model for predicting order scores is obtained through pairwise sorting loss learning and distillation training.

[0015] Optionally, in step S31, based on the dataset of video segments without concentration labels, a dataset of segments with concentration differences is obtained by extracting segments with different concentrations, including:

[0016] S311. Based on the video clip dataset without concentration labels, group the recorded segments by differences in working conditions to obtain a two-dimensional feature segment dataset.

[0017] S312. Based on the two-dimensional feature fragment dataset, feature fragments are randomly extracted from different recording segments under the same working conditions to obtain a fragment pair dataset with concentration differences.

[0018] Optionally, in step S32, based on the fragment, the dataset is compared using a visual language model to obtain a soft-order probability label dataset representing the confidence level of the concentration comparison results, including:

[0019] S321. Based on this fragment, the dataset is standardized by input format to obtain fixed sequence fragment pairs;

[0020] S322. Based on the fixed sequence fragment pair, obtain fragment identifiers with higher concentration through cue word constraints of the visual language model;

[0021] S323. Based on the segment identifier with higher concentration, obtain the log probability of the segment identifier with higher concentration by reading the log probability of the visual language model;

[0022] S324. Based on the logarithmic probability, a soft-order probability label dataset representing the confidence level of the concentration comparison results is obtained through normalization.

[0023] Optionally, in S33, based on the fragment dataset and the soft-order probability label dataset, a student model for predicting ordinal scores is obtained through pairwise ranking loss learning and distillation training, including:

[0024] S331. Based on this fragment, output the ordinal score dataset using a lightweight spatiotemporal backbone model.

[0025] S332. Based on the ordinal score dataset and the soft-order probability label dataset, a student model for predicting ordinal scores is obtained through pairwise ranking loss learning.

[0026] Optionally, in S4, the student model predicting the student based on the ordinal score is sparsely and monotonically calibrated to obtain a student model that can predict the true concentration, including:

[0027] S41. Based on the ordinal score, predict the student model by inputting a video clip with the actual concentration and obtaining the ordinal score corresponding to the actual concentration.

[0028] S42. Based on the ordinal fraction corresponding to the true concentration, a first-stage true concentration percentage student model that can output the true concentration is obtained by monotonically mapping and calibrating it with the true concentration of the video segment.

[0029] S43. Based on the first-stage student model of true concentration percentage, and constrained by the concentration output range limited by actual working conditions, a student model that can predict the true concentration is obtained.

[0030] On the other hand, the present invention provides a visual language model-guided online monitoring system for ointment concentration, which is applied to a visual language model-guided online monitoring method for ointment concentration. The system includes:

[0031] The image acquisition module is used to fix an industrial camera above the paste mixing tank and obtain the original video stream dataset of the paste surface with the assistance of lighting equipment;

[0032] The video clip acquisition module is used to extract video clips without concentration labels from the original video stream dataset of the ointment surface.

[0033] The ordinal score prediction student model training module is used to train the ordinal score prediction student model through visual language model distillation based on the dataset of video clips without concentration labels.

[0034] The true concentration calibration module is used to predict the student model based on the ordinal score. Through sparse monotonic calibration, a student model that can predict the true concentration is obtained.

[0035] The lightweight real-time inference model acquisition module is used to obtain a lightweight real-time inference model by deploying it on an edge device based on the student model that can predict the true concentration.

[0036] Compared with the prior art, the technical solution of the present invention has at least the following beneficial effects:

[0037] The above-mentioned solution achieves several advantages. First, it avoids the accuracy degradation of contact sensors and the radioactive risks of nuclear instruments at high concentrations through non-contact measurement. Second, it reduces the reliance on large-scale manually labeled concentration data through automatic annotation using a visual language model, realizing the feasibility and economy of training a vision-based concentration detection model. Third, it reduces the complexity of the concentration detection inference model and improves the real-time performance of the model inference by using a lightweight network model trained by distillation, thus enabling the lightweight deployment of real-time inference for industrial vision models. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1This is a flowchart of an embodiment of the online ointment concentration monitoring method guided by the visual language model of the present invention;

[0040] Figure 2 This is a flowchart of the training process for the ordinal score prediction student model in an embodiment of the visual language model-guided online monitoring method for ointment concentration of the present invention.

[0041] Figure 3 This is a flowchart illustrating the acquisition of a dataset of fragment pairs with concentration differences, based on an embodiment of the visual language model-guided online monitoring method for ointment concentration according to the present invention.

[0042] Figure 4 This is a flowchart illustrating the acquisition of a soft-order probability label dataset in an embodiment of the visual language model-guided online monitoring method for ointment concentration according to the present invention.

[0043] Figure 5 This is a flowchart of the pairwise sorting loss distillation process in an embodiment of the visual language model-guided online monitoring method for paste concentration of the present invention.

[0044] Figure 6 This is a flowchart of the sparse monotonic calibration of an embodiment of the visual language model-guided online monitoring method for ointment concentration of the present invention;

[0045] Figure 7 This is a system block diagram of an embodiment of the online monitoring system for ointment concentration guided by the visual language model of the present invention. Detailed Implementation

[0046] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0047] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0048] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0049] like Figure 1 The flowchart shown is an embodiment of the visual language model-guided online monitoring method for ointment concentration of the present invention. The present invention provides a visual language model-guided online monitoring method for ointment concentration, which is implemented by a visual language model-guided online monitoring system for ointment concentration. The method includes:

[0050] S1. An industrial camera is fixed above the paste mixing tank, and supplementary lighting is provided by lighting equipment to obtain the original video stream dataset of the paste surface;

[0051] Specifically, the stirring motor agitates the paste in the paste mixing tank.

[0052] S2. Based on the original video stream dataset of the ointment surface, extract video segments to obtain a video segment dataset without concentration labels.

[0053] Specifically, based on the original video stream dataset of the ointment surface, a sliding window dataset is obtained by segmenting it using a sliding window; then, based on the sliding window dataset, a video segment dataset without concentration labels is obtained by sampling frames at fixed intervals.

[0054] Furthermore, the segment sampling parameters can be set according to the on-site frame rate, with 8 frames sampled for each segment being preferred to balance timing information and real-time performance.

[0055] S3. Based on the dataset of video clips without concentration labels, a student model for predicting ordinal scores is obtained through distillation training of a visual language model.

[0056] Specifically, such as Figure 2 The flowchart shown is a student model training process for the ordinal score prediction method of the online ointment concentration guided by the visual language model of the present invention. In step S3, the ordinal score prediction student model is obtained by distillation training of the visual language model based on the video clip dataset without concentration labels, including:

[0057] S31. Based on the video clip dataset without concentration labels, extract the clip pairs with concentration differences to obtain a dataset of clip pairs with concentration differences.

[0058] Furthermore, such as Figure 3 The flowchart shown in this embodiment of the visual language model-guided online monitoring method for ointment concentration of the present invention illustrates the acquisition of a dataset of fragment pairs with concentration differences. In step S31, based on the video fragment dataset without concentration labels, a dataset of fragment pairs with concentration differences is obtained through differential concentration fragment pair extraction, including:

[0059] S311. Based on the video clip dataset without concentration labels, group the recorded segments by differences in working conditions to obtain a two-dimensional feature segment dataset.

[0060] S312. Based on the two-dimensional feature fragment dataset, feature fragments are randomly extracted from different recording segments under the same working conditions to obtain a fragment pair dataset with concentration differences.

[0061] Furthermore, the dataset of segments with concentration differences can be represented by video A and video B;

[0062] S32. Based on this fragment, the dataset is compared using a visual language model to obtain a soft-order probability label dataset that represents the confidence level of the concentration comparison results.

[0063] Furthermore, such as Figure 4 The flowchart shown in this embodiment of the visual language model-guided online monitoring method for ointment concentration of the present invention illustrates the acquisition of a soft-order probability label dataset. In step S32, based on this segment, the dataset is compared using a visual language model to obtain a soft-order probability label dataset representing the confidence level of the concentration comparison results, including:

[0064] S321. Based on this fragment, the dataset is standardized by input format to obtain fixed sequence fragment pairs;

[0065] Furthermore, this segment can be divided into video A and video B, with the first 8 frames fixed.

[0066] S322. Based on the fixed sequence fragment pair, obtain fragment identifiers with higher concentration through cue word constraints of the visual language model;

[0067] Furthermore, you can interact with the visual language model using the following prompts: Compare the visual cues related to "paste concentration" in the two sequences, including: viscosity / thickness, fluidity and surface ripple attenuation, splash frequency and amplitude, surface texture and gloss, etc.; and try to ignore interfering factors such as foam, stirrer phase difference, and light flicker; please determine which video corresponds to a higher paste concentration; your answer must be "A" or "B", do not output other content.

[0068] Furthermore, the visual language model can be selected as a model with the ability to output visual understanding and decision confidence information, such as GPT 4.1, GPT 4.1 Mini, GPT 4O Mini, etc.

[0069] S323. Based on the segment identifier with higher concentration, obtain the log probability of the segment identifier with higher concentration by reading the log probability of the visual language model;

[0070] S324. Based on the logarithmic probability, a soft-order probability label dataset representing the confidence level of the concentration comparison results is obtained through normalization.

[0071] Furthermore, the normalization process is as shown in equation (1):

[0072] (1)

[0073] In the formula, These are soft-order probability labels, representing the probability that video A has a higher concentration than video B. , These represent the logarithmic probability values ​​of video A and video B, respectively. This is the sigmoid function.

[0074] S33. Based on the fragment dataset and the soft-order probability label dataset, a student model for predicting order scores is obtained through pairwise sorting loss learning and distillation training.

[0075] Furthermore, such as Figure 5 The flowchart of the pairwise ranking loss distillation of the visual language model-guided online monitoring method for ointment concentration of the present invention shown in S33, in which the student model for ranking score prediction is obtained by learning the pairwise ranking loss and distillation based on the dataset and the soft-order probability label dataset, including:

[0076] S331. Based on this fragment, output the ordinal score dataset using a lightweight spatiotemporal backbone model.

[0077] Furthermore, spatiotemporal models such as VideoSwin-T and VideoMAE-B can be selected to achieve real-time inference;

[0078] S332. Based on the ordinal score dataset and the soft-order probability label dataset, a student model for predicting ordinal scores is obtained through pairwise ranking loss learning.

[0079] Furthermore, pairwise ranking loss learning can adopt RankNet-style pairwise ranking loss, which trains the model by penalizing the difference between the ordinal score difference probability of the student's video model and the soft-order probability label of the visual language model, thus obtaining the ordinal score prediction student model.

[0080] S4. Based on the ordinal score, predict the student model and obtain a student model that can predict the true concentration through sparse monotonic calibration.

[0081] Specifically, such as Figure 6 The flowchart shown is a sparse monotonic calibration flowchart of an embodiment of the visual language model-guided online monitoring method for ointment concentration of the present invention. In step S4, a student model that can predict the true concentration is obtained through sparse monotonic calibration based on the ordinal score prediction student model, including:

[0082] S41. Based on the ordinal score, predict the student model by inputting a video clip with the actual concentration and obtaining the ordinal score corresponding to the actual concentration.

[0083] S42. Based on the ordinal fraction corresponding to the true concentration, a first-stage true concentration percentage student model that can output the true concentration is obtained by monotonically mapping and calibrating it with the true concentration of the video segment.

[0084] S43. Based on the first-stage student model of true concentration percentage, and constrained by the concentration output range limited by actual working conditions, a student model that can predict the true concentration is obtained.

[0085] S5. Based on the student model that can predict the true concentration, a lightweight real-time inference model is obtained by deploying it on an edge device.

[0086] like Figure 7 The diagram shown is a system block diagram of an embodiment of the visual language model-guided online monitoring system for ointment concentration of the present invention. The present invention provides a visual language model-guided online monitoring system for ointment concentration, which is applied to a visual language model-guided online monitoring method for ointment concentration. The system includes: an image acquisition module, a video segment acquisition module, an ordinal score prediction student model training module, a true concentration calibration module, and a lightweight real-time inference model acquisition module. Specifically, it includes:

[0087] The image acquisition module is used to fix an industrial camera above the paste mixing tank and obtain the original video stream dataset of the paste surface with the assistance of lighting equipment;

[0088] The video clip acquisition module is used to extract video clips without concentration labels from the original video stream dataset of the ointment surface.

[0089] The ordinal score prediction student model training module is used to train the ordinal score prediction student model through visual language model distillation based on the dataset of video clips without concentration labels.

[0090] The true concentration calibration module is used to predict the student model based on the ordinal score. Through sparse monotonic calibration, a student model that can predict the true concentration is obtained.

[0091] The lightweight real-time inference model acquisition module is used to obtain a lightweight real-time inference model by deploying it on an edge device based on the student model that can predict the true concentration.

[0092] This invention provides a method and system for online monitoring of paste concentration guided by a visual language model. First, the invention acquires a video stream of the paste surface within a stirring tank and preprocesses it, preserving natural perturbations to improve model robustness. Then, it segments the video and samples in pairs across recording segments to construct concentration-distinguishable sample pairs. Next, it uses a visual language model to determine the concentration level of the sample pairs, generating soft-order probability labels. Third, it trains a lightweight concentration order score prediction student model through pairwise ordering loss learning and distillation. Finally, through sparse monotonic calibration, the output space of the student model is mapped to absolute concentration values, allowing the model to be deployed at edge environments for real-time monitoring. This invention achieves non-contact online monitoring of paste concentration without requiring large-scale, precisely labeled datasets, solving the problems of large models being difficult to deploy at edges and traditional monitoring relying on manual annotation.

[0093] It is understood that the present invention has been described through the above embodiments and should not be construed as limiting the implementation and scope of the present invention. Those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for online monitoring of ointment concentration guided by a visual language model, characterized in that, The method includes: S1. An industrial camera is fixed above the paste mixing tank, and supplementary lighting is provided by lighting equipment to obtain the original video stream dataset of the paste surface; S2. Based on the original video stream dataset of the ointment surface, extract video segments to obtain a video segment dataset without concentration labels; S3. Based on the dataset of video clips without concentration labels, a student model for predicting ordinal scores is obtained through distillation training of a visual language model. S4. Based on the ordered fraction prediction student model, a student model that can predict the true concentration is obtained through sparse monotonic calibration. S5. Based on the student model that can predict the true concentration, a lightweight real-time inference model is obtained by deploying it on an edge device.

2. The method for online monitoring of ointment concentration guided by a visual language model according to claim 1, characterized in that, In step S3, based on the dataset of video clips without concentration labels, a visual language model is trained through distillation to obtain a student model for predicting ordinal scores, including: S31. Based on the video segment dataset without concentration labels, extract segment pairs with concentration differences to obtain a dataset of segment pairs with concentration differences. S32. Based on the fragments, the dataset is compared using a visual language model to obtain a soft-order probability label dataset that represents the confidence level of the concentration comparison results. S33. Based on the fragment pair dataset and the soft-order probability label dataset, a student model for predicting order scores is obtained through pairwise sorting loss learning and distillation training.

3. The method for online monitoring of ointment concentration guided by a visual language model according to claim 2, characterized in that, In step S31, based on the video segment dataset without concentration labels, a dataset of segment pairs with concentration differences is obtained by extracting segments with different concentrations, including: S311. Based on the video segment dataset without concentration labels, group the recorded segments by differences in working conditions to obtain a two-dimensional feature segment dataset; S312. Based on the two-dimensional feature fragment dataset, feature fragments are randomly extracted from different recording segments under the same working conditions to obtain a fragment pair dataset with concentration differences.

4. The method for online monitoring of ointment concentration guided by a visual language model according to claim 2, characterized in that, In step S32, based on the fragments in the dataset, a concentration comparison is performed using a visual language model to obtain a soft-order probability label dataset representing the confidence level of the concentration comparison results, including: S321. Based on the fragment pair dataset, obtain fixed sequence fragment pairs by standardizing the input format; S322. Based on the fixed sequence fragment pairs, and through the cue word constraints of the visual language model, obtain fragment identifiers with higher concentrations; S323. Based on the higher concentration fragment identifier, obtain the log probability of the higher concentration fragment identifier by reading the log probability of the visual language model; S324. Based on the logarithmic probability, a soft-order probability label dataset representing the confidence level of the concentration comparison results is obtained through normalization.

5. The method for online monitoring of ointment concentration guided by a visual language model according to claim 2, characterized in that, In step S33, based on the fragment pair dataset and the soft-order probability label dataset, a student model for predicting ordinal scores is obtained through pairwise ranking loss learning and distillation training, including: S331. Based on the fragment dataset, output the ordinal score dataset through a lightweight spatiotemporal backbone model; S332. Based on the ordinal score dataset and the soft-order probability label dataset, a student model for predicting ordinal scores is obtained through pairwise ranking loss learning.

6. The method for online monitoring of ointment concentration guided by a visual language model according to claim 1, characterized in that, In step S4, the student model predicted based on the ordinal score is used to obtain a student model that can predict the true concentration through sparse monotonic calibration, including: S41. Based on the ordinal score prediction student model, the ordinal score corresponding to the real concentration is obtained by inputting a video clip with the real concentration. S42. Based on the ordinal score corresponding to the true concentration, a first-stage true concentration percentage student model that can output the true concentration is obtained by monotonically mapping and calibrating it with the true concentration of the video segment. S43. Based on the student model of the true concentration percentage in the first stage, and constrained by the concentration output range limited by the actual working conditions, a student model that can predict the true concentration is obtained.

7. A visual language model-guided online monitoring system for ointment concentration, used to implement the visual language model-guided online monitoring method for ointment concentration as described in any one of claims 1-6, characterized in that, The system includes: The image acquisition module is used to fix an industrial camera above the paste mixing tank and obtain the original video stream dataset of the paste surface with the assistance of lighting equipment; The video clip acquisition module is used to extract video clips without concentration labels from the original video stream dataset of the ointment surface. The ordinal score prediction student model training module is used to train the ordinal score prediction student model through visual language model distillation based on the video clip dataset without concentration labels. The true concentration calibration module is used to predict the student model based on the ordinal score, and obtain a student model that can predict the true concentration through sparse monotonic calibration. The lightweight real-time inference model acquisition module is used to obtain a lightweight real-time inference model by deploying it on an edge device based on the student model with predictable real concentration.

Images