Special glass melting process monitoring control system based on computational vision

Abstract

The invention belongs to the technical field of industrial vision monitoring, and discloses a special glass melting process monitoring control system based on computational vision, which is characterized in that a processor acquires a melt surface video frame, corrects a transient highlight area by jointly judging pixel time domain transient and spatial saturation characteristics, and generates a purified video frame; based on the purified video frame, calculating a two-dimensional optical flow velocity field and synchronously generating an optical flow confidence map representing the reliability of the two-dimensional optical flow velocity field; and decomposing the two-dimensional optical flow velocity field into a curl scalar field and a divergence scalar field, and carrying out weighting processing on the curl scalar field and the divergence scalar field by using an optical flow confidence map so as to identify and quantitatively represent topological characteristics of a flow field structure.

Description

TECHNICAL FIELD

[0001] The present application relates to a special glass melting process monitoring control system based on computer vision, belonging to the technical field of industrial vision monitoring. BACKGROUND

[0002] At present, online monitoring based on computer vision is a widely accepted and universally adopted technical approach, and its core value lies in that through image acquisition devices and analysis algorithms, the morphological characteristics of products on the production line are measured and evaluated non-contactly, such as detecting the surface defects of solid parts, measuring the accuracy of size profile, or identifying specific marks. This technology based on static or quasi-static morphological analysis plays an important role in ensuring the stability and consistency of product quality.

[0003] However, when this mature technical approach is applied to the high-temperature melting process of special optical glass, a long-standing monitoring dilemma emerges, because for a glass melt that is being clarified and homogenized, the core quality indicator of the final product, i.e. optical uniformity, is not determined by the static characteristics of its surface, but by the overall fluid dynamics process inside it. A seemingly smooth and clean melt surface without any visible bubbles or stripes may be brewing a macroscopic flow field distortion that can lead to the scrap of the final product due to small heat or composition imbalance. The existing visual monitoring technology, due to its design principle of identifying visible morphological defects that have already formed, is completely unable to perceive such invisible dynamic abnormalities that are in the embryonic stage, resulting in a high-risk silent state, i.e. the monitoring system shows everything is normal, while the seeds of failure are growing silently.

[0004] To solve this problem, one of the most direct ideas is to continuously improve the resolution and sensitivity of the image acquisition system in order to capture earlier and weaker static defect precursors, but this path still belongs to the category of identifying after the physical defect is formed, it still waits for the physical defect to form after identifying, and cannot touch the root cause of the problem, that is, the health status of the flow field itself; Specifically, the prior art mainly has the following deficiencies: 1, the wrong position of the monitoring dimension, the industry is used to finding answers in the static, morphological dimension, while the real quality source is hidden in the dynamic, fluid mechanics dimension; 2, the lag of information, all monitoring based on defect identification, the alarm must be issued later than the occurrence of the defect cause, missing the best opportunity for active intervention; 3, the ambiguity of attribution, even if the defect is observed, it is difficult to directly and quickly trace back to the dynamic root cause of its generation, and the guidance brought by the process adjustment is limited. Therefore, how to establish a kind of dynamic information within the technical framework of industrial vision, which can directly extract and quantify the dynamic information representing the macro flow state of the melt inside from the continuous video image stream, and then realize the real-time evaluation of the uniformity of the melting process, becomes the technical problem to be solved by the present application. SUMMARY

[0005] The present application provides a kind of based on the special glass melting process monitoring control system of computing vision, its main purpose is to solve the problem that the existing visual monitoring method cannot directly evaluate the macro flow state of melt, can only passively, laggingly identify the static defect formed.

[0006] To achieve the above purpose, the present application provides a kind of based on the special glass melting process monitoring control system of computing vision, the system includes image acquisition device and processor, and the processor is configured to: acquire the continuous video frame of melt surface collected by image acquisition device; perform image preprocessing operation, which identifies and corrects the transient highlight area in the video frame generated by specular reflection by jointly determining the luminance instantaneous change of pixel in time domain and the luminance saturation in spatial domain, to generate a video frame after purification processing; based on the video frame after purification processing, calculate the two-dimensional optical flow velocity field covering the monitoring area, and based on the image gradient information of the video frame after purification processing, synchronously generate an optical flow confidence map representing the reliability of two-dimensional optical flow velocity field at each pixel point; based on two-dimensional optical flow velocity field, differential operation is carried out, and it is decomposed into a vorticity scalar field and a divergence scalar field; The optical flow confidence map is used as a weight mask to perform pixel-level multiplication operation on the vorticity scalar field and the divergence scalar field, and to identify and quantify the topological features representing the vortex cores and the source-sink centers in the operation results, so as to determine the dynamic state of the melting process.

[0007] Preferably, before performing the image preprocessing operation, the processor is further configured to perform image registration operation on the continuous video frames by using the fixed furnace wall edge in the continuous video frames as an anchor point to compensate for the unintended motion of the image acquisition device, so as to establish a stable observation coordinate system for calculating the two-dimensional optical flow velocity field; and the processor suppresses the interference of low-confidence data points caused by local weak texture or mirror surface state of the melt surface during the identification and quantification of the topological features by pixel-level multiplication operation.

[0008] Preferably, when generating the optical flow confidence map, the processor is configured to: for each pixel neighborhood in the purified video frame, calculate the spatial gradient and of the structure tensor matrix , wherein, is the gray scale change of the pixel point in the horizontal direction, is the gray scale change of the pixel point in the vertical direction; calculate the minimum eigenvalue of the structure tensor matrix ; and take the normalized value of the minimum eigenvalue as the confidence value of the corresponding pixel point in the optical flow confidence map, and the smaller the value of the minimum eigenvalue , the lower the reliability of the two-dimensional optical flow velocity field at the point.

[0009] Preferably, when identifying and quantifying the topological features, the processor is configured to: adaptively determine the segmentation threshold for identifying the topological features based on a statistical baseline established by reference to the video frames collected under the steady-state operating condition; perform segmentation operation on the results of the pixel-level multiplication operation by using the segmentation threshold to locate the vortex cores and the source-sink centers; and count the number of the vortex cores and the source-sink centers, calculate the intensity peak value of each vortex core and source-sink center, and evaluate the spatial distribution symmetry of all the vortex cores and source-sink centers in the monitoring area, so as to form a set of topological state parameters quantitatively describing the dynamic state.

[0010] Preferably, the processor is further configured to: based on the purified video frames, obtain the time series of the brightness values of each pixel point on the melt surface within a determined time window; calculate the Shannon information entropy of each pixel point based on the time series of the brightness values of the pixel point, so as to generate a time-domain texture entropy map representing the texture disorder degree of the melt surface; and based on the time-domain texture entropy map, identify the local high-entropy regions with no significant response in the vorticity scalar field and the divergence scalar field, so as to supplement the monitoring of the dynamic state.

[0011] Preferably, the system further comprises a plurality of physical heating units in communication with the processor, the processor being further configured to: define, within the monitoring region, a plurality of virtual sub-zones corresponding to the spatial locations of the plurality of physical heating units respectively; calculate, for each virtual sub-zone, the square mean of the values of the scalar field of the curl within it, to obtain a dynamic performance indicator representing the actual rotational stirring contribution of the physical heating unit corresponding to the sub-zone to the melt; and diagnose the working state of the physical heating unit based on the real-time value and historical trend of the dynamic performance indicator.

[0012] Preferably, the processor is further configured to: obtain the time series of the input power signal of each physical heating unit in real time; identify the transfer function between the time series of the input power signal and the time series of the dynamic performance indicator through system identification analysis; and determine a relative viscosity index representing the degree of response delay of the melt within the corresponding virtual sub-zone to the heat input, based on the phase delay characteristics of the transfer function, to distinguish whether the change in the dynamic state is caused by insufficient driving force of the physical heating unit or by the increase in the viscosity of the melt.

[0013] Preferably, the processor is further configured to: calculate the global gray histogram entropy of the continuous video frames in real time at the determined frequency as an indicator representing the amount of image information; compare the value of the global gray histogram entropy calculated in real time with a target entropy value representing the contrast of the image; and automatically adjust the exposure parameters of the image acquisition device through closed-loop control logic according to the difference between the two, so that the amount of information of the continuous video frames remains stable in different temperature intervals of the smelting period.

[0014] Preferably, the processor is further configured to: compare the topological state parameters generated in real time with a set of stored reference topological state parameters representing ideal dynamic states; when the difference between any one parameter or a combination of multiple parameters in the topological state parameters and the reference topological state parameters exceeds a determined fluctuation range, generate and output a warning signal indicating that the melting process deviates from the steady state, and the generation of the warning signal is independent of the identification of any visible physical defects that have been formed.

[0015] Preferably, when performing the image preprocessing operation to correct the transient highlight area, the processor is configured to: for any pixel identified as being within the transient highlight area, replace the saturated brightness value of the pixel in the current video frame with the brightness value of the corresponding position in the immediately preceding cleaned video frame; this replacement operation is completed before calculating the two-dimensional optical flow velocity field, to ensure that the calculation of the two-dimensional optical flow velocity field is based on an image sequence that eliminates false brightness changes caused by specular reflection, thereby suppressing the generation of false motion vectors.

[0016] Compared with the prior art, the present application has the following advantages: 1. This invention provides an industrial vision analysis method. First, the pixel motion vector field covering the monitoring area is calculated by acquiring the video stream of the melt surface. Then, instead of directly analyzing the vector itself, the vector field is differentiated and decomposed into a curl scalar field and a divergence scalar field. In this way, a global flow state that was originally implicit in continuous image frames and was independent of specific shapes is translated into two static grayscale images with clear physical meaning. The regions with different grayscale values ​​in the images directly correspond to the local rotation and source-sink distribution inside the fluid. This transforms the assessment of the dynamic state of the melt homogenization process from relying on indirect, delayed observation of physical defects to direct and real-time observation of the flow field structure itself.

[0017] 2. Based on the observation of the flow field topology, the information entropy of the brightness value of each pixel in the video stream over time is calculated in parallel to generate a temporal texture entropy map. In this way, the system has two different sources and orthogonal physical meanings for judgment: the overall flow field structure reflected by macroscopic pixel displacement and the local texture disorder reflected by microscopic pixel pulsation. When a certain area simultaneously shows structural distortion on the flow field topology map and high-entropy bright spots on the texture entropy map, the system's confidence in the existence of a real anomaly caused by minor subsurface disturbances in that area is confirmed, avoiding misjudgments that may be caused by accidental optical effects on the surface due to a single observation dimension.

[0018] 3. By statistically calculating the squared curl values ​​within the corresponding region of each heating unit, a dynamic efficiency index characterizing the rotational kinetic energy of the fluid in that region is obtained. Simultaneously, the system acquires the input power signals of each heating unit. By systematically identifying and analyzing the time series of the input power (cause) and the dynamic efficiency index (effect), it can distinguish whether the change in the current flow state is caused by insufficient driving force or by changes in the melt's own resistance characteristics. This provides a clear attribution direction for process adjustments. Furthermore, before calculating the optical flow velocity field, the video stream image is preprocessed. This preprocessing involves detecting pixels... By analyzing instantaneous brightness changes in the time domain and brightness saturation characteristics in the spatial domain, transient highlight regions caused by specular reflection are identified and corrected. Furthermore, during the calculation of optical flow, an optical flow confidence map is generated based on image gradient information to characterize the traceability of visual features in different regions. This sequential processing approach ensures the quality of image data entering the core analysis stage, while also providing pixel-level quantitative evaluation of the reliability of the analysis results. As a result, the entire monitoring system avoids silent and erroneous judgments when the input signal degrades, and can proactively identify and mark the boundaries of its perception capabilities. Attached Figure Description

[0019] Fig. 1 This is a flowchart of the computational vision-based method for evaluating the kinetic state of a melting process according to the present invention. Fig. 2 This is a quantitative relationship diagram between the optical flow confidence level and the surface texture characteristics of the melt in this invention; Fig. 3 This is a schematic diagram of the system hardware architecture and core data flow of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0021] The present invention discloses a special glass melting process monitoring and control system based on computer vision, including image sequence acquisition and registration, image sequence preprocessing, parallel calculation of optical flow velocity field and its confidence level, differential decomposition and weighting of velocity field, and quantification and state assessment of flow field topological features. The system captures continuous video frames of the melt surface through image acquisition equipment. First, image registration is performed to establish a stable observation coordinate system. Then, a purification process is used to identify and correct transient highlight areas caused by specular reflection. Subsequently, a two-dimensional optical flow velocity field covering the monitoring area is calculated on the purified image sequence, and an optical flow confidence map characterizing the reliability of each point of the velocity field is generated simultaneously. Finally, the system decomposes the two-dimensional optical flow velocity field into two scalar fields with clear physical meanings: curl and divergence, and uses the optical flow confidence map to weight them to accurately identify and quantify the topological features characterizing the dynamic state of the melt.

[0022] In a specific application scenario, the system is deployed for online monitoring of a special optical glass melting furnace. Its image acquisition device is an industrial camera with programmable exposure parameters, continuously acquiring a video stream of the melt surface at a resolution of 1920x1080 pixels at a frame rate of 30 frames per second. A common technical challenge during system operation is that slight mechanical vibrations of the image acquisition device can introduce unrealistic global pixel displacements. If this displacement is not compensated for, it will severely interfere with subsequent calculations of the actual melt flow. To address this challenge, before performing any analysis, the system is configured to use a fixed furnace wall edge, always visible in the continuous video frames, as an anchor point to perform image registration. This operation first occurs in the initial frame... Within the edge region of the furnace wall, at least 20 stable high-gradient feature points were identified using the Harris corner detection algorithm. In each subsequent frame, the Kanade-Lucas-Tomasi feature tracking algorithm was used to continuously track these feature points. Based on the coordinate displacement of these feature points in the new and old frames, an affine transformation matrix that describes the unexpected motion of the camera was calculated. The system used this matrix to perform an inverse transformation on each newly acquired video frame, thereby aligning all images to a stable observation coordinate system based on the initial frame. Through this procedure, pseudo-motions introduced by external vibrations were filtered out, laying the foundation for subsequent accurate calculation of the true flow velocity field on the melt surface.

[0023] After establishing a stable observation coordinate system, another key technical challenge arises from the simultaneous presence of diffuse reflection textures and specular reflection regions on the melt surface at high temperatures. The transient specular highlights generated by specular reflection exhibit drastic brightness changes and are unrelated to the actual physical flow of the melt, making them a major source of noise in optical flow calculations. Therefore, before calculating the two-dimensional optical flow velocity field, the system employs an image preprocessing operation to identify and correct transient specular highlight regions in video frames. This operation locates contamination points by jointly determining the instantaneous brightness changes of pixels in the temporal domain and their brightness saturation in the spatial domain. Specifically, for any pixel in the image, the system first determines whether its brightness value exceeds a spatial saturation threshold. For example, for an 8-bit grayscale image, this threshold... The value can be set to 250. If this condition is met, it is further determined whether the brightness difference between the pixel and the pixel at the same position in the previous frame is greater than a transient threshold in the time domain, such as 120. Only when both conditions are met is the pixel finally identified as a transient specular pollution point. For the identified specular pollution point, the system uses the brightness value of the pixel at the corresponding position in the previous frame to replace the saturation brightness value of the pixel in the current frame. This replacement operation is completed before calculating the two-dimensional optical flow velocity field, thereby generating a cleaned video frame. This is intended to ensure that the subsequent calculation of the two-dimensional optical flow velocity field is based on an image sequence that eliminates false brightness changes caused by specular reflection, thereby suppressing the generation of pseudo motion vectors.

[0024] Based on the cleaned video frames, the system enters the core analysis phase, which involves calculating the two-dimensional optical flow velocity field covering the monitored area. However, the texture richness varies greatly across different regions of the melt surface. In some highly homogeneous areas, the surface appears almost mirror-like, lacking traceable visual features, which makes the calculated optical flow vectors in these areas unreliable. To address this issue, the system is configured to simultaneously generate an optical flow confidence map characterizing the reliability of this velocity field at each pixel while calculating the two-dimensional optical flow velocity field. The generation procedure for this confidence map is as follows: For each pixel neighborhood in the cleaned video frame, such as a 5x5 pixel window, its spatial gradient is calculated. and Structure tensor matrix ,in, This represents the grayscale variation of a pixel in the horizontal direction. This represents the grayscale variation of a pixel in the vertical direction; subsequently, the system calculates the structural tensor matrix. The smallest eigenvalue It should be noted that this minimum eigenvalue The numerical value directly reflects the intensity and anisotropy of the texture within that neighborhood. The smaller the value, the more flat the region tends to be or the edge has only one direction, and the worse the traceability of its visual features; finally, the system calculates the corresponding values ​​for each pixel. The numerical values ​​are normalized to map them to the range of 0 to 1, and this normalized value is used as the confidence value of the corresponding pixel in the optical flow confidence map. In this way, an optical flow confidence map with the same pixel size as the two-dimensional optical flow velocity field is generated, which provides an objective basis for the reliability of the data for subsequent analysis.

[0025] Obtaining the two-dimensional optical flow velocity field Subsequently, directly analyzing dense vector diagrams is neither intuitive nor easy to quantify, as dynamic information is hidden within the structure of the flow field. Therefore, the system performs differential operations based on the two-dimensional optical flow velocity field, decomposing it into a curl scalar field. and a divergence scalar field Among them, the curl scalar field It characterizes the local rotational intensity and direction of the fluid at various points, while the divergence scalar field This characterizes the local convergence or divergence intensity of the fluid at various points. This decomposition operation translates a flow state independent of specific morphology into two topological maps with clear physical meaning. Furthermore, to incorporate the signal reliability information obtained in the preceding steps into the analysis, the system uses the optical flow confidence map as a weight mask to perform pixel-level multiplication operations on the curl scalar field and the divergence scalar field. For example, for any pixel in the curl field... its weighted value ,in, The confidence value of the corresponding point in the confidence graph is used for weighting. This weighting process can automatically suppress the interference of low-confidence data points caused by local weak textures or mirror-like conditions on the melt surface, thereby improving the accuracy of subsequent topological feature recognition.

[0026] After obtaining the weighted curl and divergence scalar fields, the system performs topological feature identification and quantization to ultimately determine the kinetic state of the melting process. A key aspect of this process is determining the segmentation threshold; a fixed threshold cannot adapt to the changing operating conditions at different stages of the melting cycle. To achieve adaptive processing, the system is configured to dynamically determine the segmentation threshold for identifying topological features based on a statistical baseline established from video frames acquired under reference steady-state operating conditions. The calibration procedure is as follows: during a known stable homogeneous melt flow stage, the system continuously acquires no fewer than 1000 video frames and calculates the corresponding weighted curl and divergence field sequences. Then, the standard deviations of the values ​​of all pixels in these two fields throughout the entire sequence are calculated. and Therefore, in real-time monitoring, for example... and As a segmentation threshold, the system uses this adaptive threshold to segment the real-time computation results to locate significant vortex cores and source-sink centers in the image. It further counts the number of these topological features, calculates the peak intensity of each feature, and assesses the spatial distribution symmetry of all features within the monitoring area. This forms a set of topological parameters that quantitatively describe the current dynamic state, constituting a direct and immediate quantitative assessment of the melt flow state. In addition, to supplement the monitoring of the dynamic state, the system also analyzes the temporal brightness changes of the pixels themselves in parallel to identify regions that are unresponsive in the flow field but may indicate subsurface anomalies. The technical solution involves the system acquiring the brightness value time series of each pixel on the melt surface within a defined time window, for example, 15 consecutive frames, based on purified video frames. Subsequently, the Shannon information entropy of the brightness value time series of each pixel is calculated. ,in, To display the brightness value of the pixel within this time window. The probability is used to generate a temporal texture entropy map characterizing the disorder of the melt surface texture; a local high entropy region, if it does not respond significantly in the curl and divergence scalar fields, may indicate a local anomaly caused by a small perturbation in the subsurface, providing another orthogonal judgment dimension for the system.

[0027] To correlate visual monitoring results with physical execution units, the system also includes multiple physical heating units that communicate with the processor. The system is further configured to predefine multiple virtual partitions within the monitoring area, corresponding to the spatial locations of each physical heating unit, based on the furnace design drawings. For each virtual partition, the system calculates the squared mean of the curl scalar field values ​​within it to obtain a kinetic efficacy index characterizing the actual rotational stirring contribution of the corresponding physical heating unit to the melt. The real-time value and historical trend of this index can be used to diagnose the working state of the physical heating units. Furthermore, to distinguish whether changes in the kinetic state are caused by insufficient driving force or increased melt viscosity, the system acquires the real-time input power signal time series of each physical heating unit and performs system identification and analysis on the time series of the aforementioned kinetic efficacy index, online... By identifying the transfer function between the two, and based on the phase delay characteristics of this transfer function, the system can determine a relative viscosity index that characterizes the degree of hysteresis in the response of the melt to thermal input within the corresponding virtual partition. Finally, to ensure the stability of the information content of the input image across different temperature ranges throughout the melting cycle, the system is also configured to execute a closed-loop control logic to automatically adjust the exposure parameters of the image acquisition device. This logic determines a frequency, such as once per second, to calculate the global grayscale histogram entropy of continuous video frames in real time, and uses it as an indicator of the image information content. The system compares the value of the real-time calculated global grayscale histogram entropy with a target entropy value that characterizes the ideal image contrast, and based on the difference between the two, automatically adjusts the camera's exposure time or gain through a proportional controller, thereby ensuring that the continuous video frames maintain stable information content and clear texture details across different temperature ranges throughout the melting cycle.

[0028] Example 1: In a continuously operating special rare-earth-doped optical glass melting furnace, the melt temperature is maintained near the upper limit of the clarification and homogenization stage. At this time, the surface tension of the melt is high, and most areas exhibit a low-texture mirror-like state. This condition presents two problems for vision-based monitoring: first, minute subsurface thermal disturbances are difficult to form traceable textures on the surface; second, high-temperature mirror surfaces are prone to transient high-gloss pseudo-signals unrelated to actual flow due to thermal radiation fluctuations. Under this condition, a tiny spalling point of refractory material on the inner wall of the furnace causes a localized, continuous heat dissipation anomaly of less than 0.5%. This anomaly cannot be detected by the temperature sensor, but a slow, asymmetric process has formed inside the melt above it. To mitigate convective distortion, the system continuously acquires consecutive video frames of the melt surface and first initiates image preprocessing. This process identifies and corrects randomly generated transient highlight regions by jointly determining the instantaneous brightness changes of pixels in the temporal domain and the brightness saturation in the spatial domain, generating cleaned video frames. Simultaneously, the system generates an optical flow confidence map based on the cleaned video frames. In this confidence map, the confidence values ​​of most low-texture areas on the melt surface are below 0.2, while the confidence values ​​of the mirror-like areas formed above the heat dissipation anomaly points due to flow stagnation are close to 0. This process allows the system to quantify the traceability of visual information in each region of the field of view before calculating motion vectors.

[0029] After calculating the two-dimensional optical flow velocity field covering the monitoring area, the system uses the optical flow confidence map as a weight mask to perform pixel-level multiplication operations on the decomposed curl and divergence scalar fields. Image preprocessing ensures that the image sequence used for subsequent calculations does not contain transient specular interference. Furthermore, the weighting operation using the optical flow confidence map suppresses unreliable calculation results in low-texture regions. This combination of steps allows the identification of a weak flow field topology driven by real convection distortion, which is masked by noise and low-texture conditions in the original video stream. Specifically, in the medium-confidence region surrounding the heat dissipation anomaly point, the system identifies a persistent curl peak as the baseline standard deviation. The vortex core is located in the central mirror region where the confidence level is close to 0, and its curl value is suppressed. The system continuously statistically analyzes the topological parameters and determines that the number, intensity and spatial position of the vortex core remain stable within 300 consecutive frames. Its state exceeds the random fluctuation range under steady-state operating conditions. The system then generates and outputs a warning signal, which indicates that the melting process has deviated from the steady state. Its generation is independent of the identification of any visible physical defects that have been formed. Based on the spatial position information provided by the warning signal, the process engineers make compensatory fine adjustments to the corresponding physical heating unit parameters. The convection distortion disappears, thus preventing an entire batch of high-value glass from being scrapped due to optical uniformity defects.

[0030] In a continuously operating special rare-earth-doped optical glass melting furnace, the melt temperature is maintained near the upper limit of the clarification and homogenization stage. At this point, the melt surface tension is high, and most areas exhibit a low-texture mirror-like state. This condition presents two challenges for vision-based monitoring: firstly, minute subsurface thermal disturbances are difficult to form traceable textures on the surface; secondly, high-temperature mirror surfaces are prone to transient specular highlights unrelated to actual flow due to thermal radiation fluctuations. Under these conditions, a tiny spalling point of refractory material on the furnace inner wall causes a localized, persistent heat dissipation anomaly of less than 0.5%. This anomaly cannot be detected by the temperature sensor, but a slow, asymmetric convection has formed within the melt above it. To address distortion, the system continuously acquires consecutive video frames of the melt surface and first initiates image preprocessing. This operation identifies and corrects randomly generated transient highlight areas by jointly determining the instantaneous brightness changes of pixels in the temporal domain and the brightness saturation in the spatial domain, generating cleaned video frames. Simultaneously, the system generates an optical flow confidence map based on the cleaned video frames. In this confidence map, the confidence values ​​of most low-texture areas on the melt surface are below 0.2, while the confidence values ​​of the mirror-like areas formed due to flow stagnation above the heat dissipation anomaly points are close to 0. This process allows the system to quantify the traceability of visual information in each region of the field of view before calculating motion vectors.

[0031] After calculating the two-dimensional optical flow velocity field covering the monitoring area, the system uses the optical flow confidence map as a weight mask to perform pixel-level multiplication operations on the decomposed curl and divergence scalar fields. Image preprocessing ensures that the image sequence used for subsequent calculations does not contain transient specular interference. The weighting operation using the optical flow confidence map suppresses unreliable calculation results in low-texture regions. This combination of steps allows the identification of a weak flow field topology driven by real convection distortion, which is masked by noise and low-texture conditions in the original video stream. Specifically, in the medium-confidence region surrounding the heat dissipation anomaly point, the system identifies a persistent flow field with a curl peak as the baseline standard deviation. The vortex core is located in the center mirror region where the confidence level is close to 0, and its curl value is suppressed. The system continuously counts the topological parameters and determines that the number, intensity and spatial position of the vortex core remain stable within 300 consecutive frames. Its state exceeds the random fluctuation range under steady-state operating conditions. The system then generates and outputs a warning signal, which indicates that the melting process has deviated from the steady state. Its generation is independent of the identification of any visible physical defects that have been formed. Based on the spatial position information provided by the warning signal, the process engineers make compensatory fine adjustments to the corresponding physical heating unit parameters. The convection distortion disappears, thus preventing an entire batch of high-value glass from being scrapped due to optical uniformity defects.

[0032] Example 2: To objectively verify the effectiveness of the technical solution of the present invention in identifying early flow field distortions, a controllable laboratory simulation test platform was constructed. This experiment aimed to quantitatively evaluate the performance of the method of the present invention in detecting weak dynamic anomalies under low signal-to-noise ratio conditions. The test platform consisted of a transparent glass container measuring 500mm x 500mm x 200mm, filled with a glycerol aqueous solution to simulate a high-viscosity fluid. A uniformly arranged heating film at the bottom was used to generate stable background thermal convection, with a temperature control accuracy of ±0.1. A miniature Peltier cooling unit with linearly adjustable power from 0W to 5.0W was placed at the center of the bottom of the container to simulate a controlled local cold spot disturbance. The experimental data acquisition system included an industrial camera mounted perpendicular to the liquid surface, with a monochrome CMOS sensor featuring a 1920x1080 pixel resolution and a 30fps acquisition frame rate. A small number of neutral buoyancy tracer particles were added to the fluid to provide traceable visual texture. Simultaneously, a laser Doppler velocimeter was configured to non-contactly measure the vortex intensity of the flow field in the region directly above the cooling unit, serving as the baseline for this experiment. Data processing was performed by two parallel software channels: the experimental group using the technical solution of this invention and the control group performing only standard optical flow calculations without confidence weighting. During the experiment, the power of the Peltier cooling unit started from 0W and increased in increments of 0.5W every 300 seconds until it reached 5.0W. This time interval was set to ensure that the flow field had sufficient time to reach a quasi-steady state after each power step.

[0033] During the experiment, when the cooling power was below 1.5W, the resulting local convection disturbances were not visible in the original video. The two-dimensional optical flow velocity field output by the control group appeared as disordered noise, and the calculated curl scalar field value fluctuated randomly around zero. When the cooling power was 1.0W, the true value measured by the laser Doppler velocimeter was 0.25. At this point, the peak curl intensity output by the control group was 0.48. There were no irregular fluctuations in the vicinity, while the peak curl intensity of the experimental group was consistently 0.26. The deviation from the true value was less than 4%. With increasing cooling power, the peak curl intensity output by the test group showed a positive correlation with the power level. When the cooling power reached 5.0W, the true value of the tachometer was 1.28. The output value of the experimental group was 1.31. The output value of the control group was 2.05. If the peak curl intensity output by the experimental group is plotted as a function of cooling power, the curve shows a linear growth trend that is highly consistent with the true value curve of the laser Doppler velocimeter. In contrast, the data points of the control group show a scattered distribution without trend throughout the power range, and their values ​​are much higher than the background noise floor. The mechanism of weighted processing based on optical flow confidence map used in the experimental group is the reason why it can accurately extract weak dynamic features from high noise background. The experimental results confirm that the technical solution of the present invention can effectively identify and quantify early flow field topological distortions below the detection limit of traditional visual monitoring methods. Its monitoring dimension has changed from morphological observation of physical defects to direct measurement of the flow field structure itself.

[0034] Example 3: This example combines Figs. 1 to 3 This document describes a monitoring and control system for the melting process of special glass based on computer vision. Fig. 1 As shown, using continuous video frames from the melt surface as the original video input, the process first involves image sequence acquisition and registration. By compensating for unexpected camera motion, a stable coordinate system is established. The video frames are then input into the image sequence preprocessing module. This module corrects transient highlight areas to generate cleaned video frames. These cleaned video frames are then fed into two parallel processing branches. The first is a temporal texture entropy map generation branch, which calculates the information entropy of the brightness value time series to generate a temporal texture entropy map to supplement the monitoring of local anomalies. The second is the main analysis process, namely the parallel calculation of the optical flow field and its confidence level. This step generates a two-dimensional optical flow velocity field and an optical flow confidence level. The confidence map is then used to weight the decomposed curl / divergence scalar fields in the differential decomposition and weighting steps of the velocity field, generating a weighted curl / divergence field. This weighted field is used for two purposes: firstly, to calculate the dynamic efficiency index, which is related to the input power of the physical heating unit to diagnose the working state and attribution; and secondly, to quantify the topological characteristics of the flow field to identify the vortex core and source-sink center and generate topological state parameters. Finally, in the comprehensive evaluation of the dynamic state, the real-time topological state parameters are compared with the baseline parameters to determine whether the melting process deviates from the steady state. When the difference exceeds the fluctuation range, an early warning signal is generated to indicate that the melting process deviates from the steady state.

[0035] like Fig. 2 As shown, the horizontal axis represents the spatial location of pixels, and the vertical axis represents the optical flow confidence. The three curves in the figure correspond to the typical confidence distribution of high-texture regions, medium-texture regions, and mirror regions, respectively. The curves show that the high-texture region has the highest optical flow confidence due to its rich traceable visual features, with values ​​mainly distributed in the range of 0.8 to 1.0. The medium-texture region has the second highest confidence, while the mirror region, which lacks visual features, has its optical flow confidence effectively suppressed to a low level close to 0.1.

[0036] like Fig. 3 As shown, the system mainly includes an industrial camera and multiple physical heating units as field devices, as well as an edge computing server and an operator workstation. The industrial camera acquires high-definition video streams and transmits them to the edge computing server. The physical heating units exchange power signals and status data bidirectionally with the server. The edge computing server, as the core of the system, integrates a core analysis engine responsible for optical flow calculation, topology decomposition, and entropy analysis; a parameter database storing benchmark topological parameters and thermodynamic response models; and a status assessment and decision-making module that performs comparison with benchmark parameters and generates early warning signals. The server finally outputs the analyzed and generated early warning and diagnostic information to the operator workstation, which, through its human-machine interface (HMI), intuitively presents the real-time dynamic state of the melting process to the process personnel.

[0037] Example 4: This example supplements the explanation of the determination method of key parameters and thresholds involved in the aforementioned technical solution. In a specific engineering practice, when the monitoring and control system of this invention is first deployed on a specific glass melting furnace, a standardized offline calibration and initialization procedure needs to be executed to match its internal parameters with the optical environment thermal characteristics and melt material of the current furnace. This procedure begins when the melt has reached a stable typical production temperature and all physical heating units are in steady-state operation. At this time, the system first enters the calibration stage of adaptive exposure parameters. The system is configured to automatically drive the image acquisition device, performing an exposure time scan from 1ms to 100ms in 1ms increments. Under each exposure time setting, the system continuously acquires 10 frames of images and calculates the average value of the global grayscale histogram entropy of these 10 frames. The system selects the energy production... The exposure time that generates the maximum average entropy value is used as the initial operating point, and the maximum average entropy value of 5.6 bits measured in this calibration is stored as the target entropy value used by the subsequent closed-loop control logic. Subsequently, the system enters the calibration stage of transient highlight area correction parameters. In this stage, several transient highlight events are generated on the surface of the melt by briefly turning on an auxiliary light source. The system synchronously records a video sequence of 300 frames containing these events. Then, the system analyzes the sequence, extracts the time series of all pixels with brightness values ​​greater than 240, and calculates their first-order differences. The system determines the 5% quantile 251 of the brightness peak in all highlight events as the spatial saturation threshold, and determines the 5% quantile 135 of the brightness first-order difference peak as the temporal transient threshold. The use of statistical quantiles rather than extreme values ​​aims to establish a parameter benchmark that can respond to weak highlight signals.

[0038] After setting the above parameters, the system enters the baseline establishment phase for dynamic state assessment. Once the furnace is confirmed to be in a stable operating state (i.e., the readings of all physical sensors fluctuate by less than 0.2% over the past 30 minutes), the system automatically acquires a reference video stream of at least 3000 frames. The system then applies the calibrated parameters to process this video stream until a weighted curl scalar field is generated. With divergence scalar field The sequence was used to calculate the standard deviation characterizing the steady-state background fluctuations. and Regarding the determination of the multiplier in the segmentation threshold, the system further analyzes the kurtosis of the probability density distribution of the two scalar field values. When the kurtosis value is between 2.5 and 3.5, it indicates that the background fluctuation is close to a Gaussian distribution, and the system uses 3 as the standard multiplier. If the kurtosis value is greater than 3.5, the system will use a function... To determine the multiplier, in this calibration, the measured kurtosis value was 4.1. Based on this function, the multiplier was calculated to be 3.3. Therefore, the segmentation thresholds were set as follows: and To quantify the spatial distribution symmetry of the topological parameters, this embodiment specifies the calculation method for these parameters. The system first calculates the geometric center coordinates of the monitored area image. In real-time monitoring, after identifying N topological feature points, the system will record the location coordinates of each feature point. Its corresponding peak intensity By performing a weighted average, the intensity centroids of these topological features are calculated. Ultimately, spatial distribution symmetry is defined as the Euclidean distance between these two center points. The smaller the distance value, the better the symmetry of the flow field structure.

[0039] Example 5: During the production process, when a new batch of batch material with known minor differences in composition compared to the previous batch, and this difference is expected to affect the melt viscosity, is fed into the furnace, the system initiates an online transfer function identification and model parameter update process. This process updates the benchmark used by the system to distinguish the attribution of changes in flow state. Specifically, within a preset time window, the system synchronously acquires the time series of the input power signal of each physical heating unit. and the time series of dynamic performance indicators of the corresponding virtual partitions .

[0040] The system is based on a pre-defined second-order autoregressive external factor input model and uses recursive least squares method to process the collected data. and The data is used for online parameter estimation to identify the transfer function characterizing the thermodynamic response of the heating zone. The system then extracts the phase delay at the dominant heating control frequency from this transfer function. and based on Calculate the relative viscosity index of the current batch, where, The updated relative viscosity index is used as the reference phase delay determined during the initial calibration. This updated relative viscosity index is used as the new attribution analysis benchmark. In subsequent monitoring, when the system identifies a slowdown in flow through topological parameters, the real-time kinetic performance index can be compared with this new benchmark to distinguish whether the change in flow state is caused by insufficient driving force of the physical heating unit or by an increase in the intrinsic melt viscosity of the current batch of raw materials.

[0041] Example 6: To avoid false alarms caused by brief, non-critical process fluctuations, this example provides a supplementary explanation of the generation logic for the warning signal. When the real-time value of any topology parameter deviates from its reference value, the system does not immediately trigger an alarm. Instead, it starts a time integrator to continuously accumulate the degree of deviation of the parameter within a preset sliding time window. Only when the accumulated deviation value exceeds an integration threshold associated with the length of the window is the parameter set to a state awaiting warning.

[0042] The generation of a final warning signal requires at least two independent topological parameters within the monitored area: the number of vortex cores and the symmetry of their spatial distribution. The corresponding time integrators must be triggered and in a state of readiness for warning. This judgment mechanism filters out transient disturbances of single parameters by requiring the persistence of abnormal states over time and their co-occurrence across parameter dimensions. Only when this multi-parameter consensus condition is met does the system formally generate and output a warning signal. To address potential physical characteristic drift in the furnace due to long-term operation, the system also includes a periodic verification and update procedure for the baseline topological parameters. This procedure is set to automatically update the physical sensor readings at a preset time point at a fixed point in each production cycle. During steady-state operation when fluctuations within the window are less than a predetermined threshold, data is collected and a set of temporary topological parameter statistics are generated. The system calculates the Mahalanobis distance between these temporary statistics and the stored baseline topological parameters. If the distance is less than a preset drift threshold, the baseline remains unchanged. If the distance exceeds the threshold, the system alerts the operator that the baseline may have become invalid. After the operator confirms that the operating conditions during this period can be used as a new stable baseline, the system is authorized to use the newly generated set of temporary statistics to replace the original baseline topological parameters. Furthermore, during the later stages of clarification and homogenization in a production batch, the overall melt flow has become smoother, and the system's curl scalar field and divergence scalar field have not exceeded the baseline standard deviation during continuous monitoring. The topological features were observed. However, during this period, a local high-entropy bright spot with a diameter of approximately 50 pixels continuously appeared at a specific location in the monitored area on the temporal texture entropy map generated in parallel by the system. The average Shannon information entropy of the pixels inside the bright spot reached 4.5 bits, which was higher than the background average of 2.1 bits in the surrounding area. After retrieving the process records, it was confirmed that a small amount of refractory zirconium particles were mixed in during a feeding process below this location. The particle size itself was insufficient to cause macroscopic flow that could be identified by the two-dimensional optical flow velocity field. However, its continuous and weak melting and dethermal process caused irregular high-frequency local refractive index fluctuations on the surface of the melt. These fluctuations were effectively captured by the temporal texture entropy map, thus providing an early warning for a potential quality hazard caused by minor component inhomogeneities in the subsurface layer when the main monitoring channel did not respond.

[0043] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A special glass melting process monitoring and control system based on computer vision, characterized in that, The system includes an image acquisition device and a processor, the processor being configured as follows: Acquire continuous video frames of the melt surface captured by an image acquisition device; An image preprocessing operation is performed. This operation identifies and corrects transient highlight areas caused by specular reflection in video frames by jointly determining the instantaneous brightness change of pixels in the time domain and the brightness saturation in the spatial domain, thereby generating cleaned video frames. Based on the purified video frames, the two-dimensional optical flow velocity field covering the monitoring area is calculated, and based on the image gradient information of the purified video frames, an optical flow confidence map characterizing the reliability of the two-dimensional optical flow velocity field at each pixel is generated simultaneously. Differential operations are performed on the two-dimensional optical flow velocity field to decompose it into a curl scalar field and a divergence scalar field; Using the optical flow confidence map as a weight mask, pixel-level multiplication operations are performed on the curl scalar field and the divergence scalar field. The topological features of the vortex core and source-sink center are identified and quantified in the operation results to determine the dynamic state of the melting process.

2. The special glass melting process monitoring and control system based on computer vision according to claim 1, characterized in that, Before performing image preprocessing operations, the processor is also configured to use the fixed furnace wall edge in the continuous video frames as anchor points to perform image registration operations on the continuous video frames to compensate for the unexpected motion of the image acquisition device, thereby establishing a stable observation coordinate system for calculating the two-dimensional optical flow velocity field. Furthermore, the processor uses pixel-level multiplication operations to suppress interference from low-confidence data points caused by local weak textures or mirror-like states on the melt surface during the identification and quantization of topological features.

3. The special glass melting process monitoring and control system based on computer vision according to claim 1, characterized in that, When synchronously generating the optical flow confidence map, the processor is configured to: calculate the spatial gradient for each pixel neighborhood in the cleaned video frame. and Structure tensor matrix ,in, This represents the grayscale variation of a pixel in the horizontal direction. The grayscale variation of a pixel in the vertical direction; calculate the structure tensor matrix. The smallest eigenvalue ; and the smallest eigenvalue The normalized value is used as the confidence value of the corresponding pixel in the optical flow confidence map, and the minimum eigenvalue is used as the minimum eigenvalue. The smaller the value, the lower the reliability of characterizing the two-dimensional optical flow velocity field at that point.

4. The special glass melting process monitoring and control system based on computer vision according to claim 1, characterized in that, When identifying and quantifying topological features, the processor is configured to: adaptively determine the segmentation threshold for identifying topological features based on a statistical baseline established by video frames acquired under reference steady-state operating conditions; perform segmentation operations on the results of pixel-level multiplication operations to locate vortex cores and source-sink centers; and count the number of vortex cores and source-sink centers, calculate the intensity peak of each vortex core and source-sink center, and evaluate the spatial distribution symmetry of all vortex cores and source-sink centers within the monitoring area to form a set of topological state parameters that quantitatively describe the dynamic state.

5. The special glass melting process monitoring and control system based on computer vision according to claim 1, characterized in that, The processor is also configured to: acquire the time series of brightness values ​​of each pixel on the melt surface within a defined time window based on the purified video frames; calculate the Shannon information entropy of the brightness value time series of each pixel to generate a temporal texture entropy map characterizing the disorder of the melt surface texture; and, based on the temporal texture entropy map, identify local high-entropy regions that have no significant response in the curl scalar field and divergence scalar field to supplement the monitoring of the dynamic state.

6. The special glass melting process monitoring and control system based on computer vision according to claim 1, characterized in that, The system also includes multiple physical heating units that communicate with the processor. The processor is further configured to: define multiple virtual partitions within the monitoring area that correspond to the spatial locations of the multiple physical heating units; for each virtual partition, calculate the square mean of the curl scalar field values ​​within it to obtain a kinetic efficacy index characterizing the actual rotational stirring contribution of the physical heating unit corresponding to that partition to the melt; and diagnose the working status of the physical heating units based on the real-time value and historical trend of the kinetic efficacy index.

7. The special glass melting process monitoring and control system based on computer vision according to claim 6, characterized in that, The processor is also configured to: acquire the time series of the input power signal of each physical heating unit in real time; and identify the transfer function between the time series of the input power signal and the time series of the kinetic efficiency index online by performing system identification and analysis on the two. Furthermore, based on the phase delay characteristics of the transfer function, a relative viscosity index is determined to characterize the degree of hysteresis in the response of the melt to thermal input within the corresponding virtual partition.

8. The special glass melting process monitoring and control system based on computer vision according to claim 1, characterized in that, The processor is also configured to: calculate the global grayscale histogram entropy of consecutive video frames in real time at a determined frequency as an indicator of image information content; compare the value of the real-time calculated global grayscale histogram entropy with a target entropy value that characterizes image contrast; and automatically adjust the exposure parameters of the image acquisition device through closed-loop control logic based on the difference between the two.

9. A special glass melting process monitoring and control system based on computer vision according to claim 4, characterized in that, The processor is also configured to: compare the topological state parameters generated in real time with a set of stored reference topological state parameters that characterize the ideal dynamic state; and generate and output an early warning signal when the difference between any one or more parameters in the topological state parameters and the reference topological state parameters exceeds a defined fluctuation range.

10. A special glass melting process monitoring and control system based on computer vision according to claim 1, characterized in that, When performing image preprocessing operations to correct transient highlight areas, the processor is configured to replace the saturation brightness value of the pixel in the current video frame with the brightness value of the corresponding position of the pixel in the immediately preceding cleaned video frame for any pixel identified as a transient highlight area.

Images