Digital visual control system and method for automation equipment
By using a digital visualization control system to precisely sense and intelligently predict the flow state of powder or granular materials, the problem of insufficient sensing and coarse control in existing technologies is solved. This enables proactive early warning and precise intervention for flow anomalies, improving the stability and accuracy of the conveying and packaging process.
Patent Information
- Application Number
- CN202510773167.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-11
- Publication Date
- 2025-10-28
AI Technical Summary
Existing technologies lack in-depth perception of the actual flow state of materials in the automated conveying and packaging of powder or granular materials. They are unable to capture microscopic characteristics and dynamic behavior, resulting in poor control effects or excessive intervention. They are difficult to predict and respond to flow anomalies and lack the ability to provide early warning of potential blockages and bridging risks.
A digital visualization control system is adopted. Through image sequence acquisition and preprocessing, flow texture and velocity distribution are analyzed, a three-dimensional point cloud model is constructed, the characteristics of the accumulation surface are analyzed, and intelligent prediction and control parameter adjustment are performed by combining flow state and accumulation data, so as to achieve fine perception and proactive early warning of flow state.
It achieves in-depth and comprehensive perception of material flow status, provides high-quality basic data, and significantly improves the stability and accuracy of powder material conveying and packaging process through intelligent prediction and adaptive adjustment, reducing the risk of unplanned equipment downtime and damage.
Smart Images

Figure CN120848390A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial automation and process control technology, and in particular to a digital visualization control system and method for automated equipment. Background Technology
[0002] In the automated conveying and packaging of powder or granular materials, existing technologies mainly rely on simple weight or flow sensors for monitoring and control. However, this approach has the following significant drawbacks: it lacks in-depth perception of the actual flow state of the material, failing to capture key microscopic characteristics and complex dynamic behaviors such as surface texture changes, local velocity distribution, agglomeration, or stratification; it cannot effectively predict and respond to dynamic changes in material flow characteristics, only passively responding after material flow anomalies (such as a significant decrease or interruption in flow) occur, lacking early warning capabilities for potential blockages and bridging risks; and it lacks intelligent control capabilities for abnormal material states, making it difficult to make fine-grained and adaptive parameter adjustments based on complex and dynamically changing material behavior, resulting in poor control effects or excessive intervention.
[0003] In summary, existing technologies suffer from problems such as insufficient sensing, delayed prediction, and coarse control, and cannot effectively solve the abnormal flow phenomena that easily occur in powder or granular materials under complex and variable conditions, which urgently need to be addressed. Summary of the Invention
[0004] Therefore, it is necessary to provide a digital visualization control system and method for automated equipment to solve at least one of the above-mentioned technical problems.
[0005] To achieve the above objectives, a digital visualization control method for automated equipment includes the following steps:
[0006] Step S1: Acquire and preprocess image sequences of the automated material conveying equipment to obtain an enhanced flow image set; analyze the flow texture and velocity distribution of the enhanced flow image set to obtain a texture feature data set and a velocity field distribution map; extract flow characteristics from the texture feature data set and the velocity field distribution map to obtain a flow feature vector map.
[0007] Step S2: Construct a 3D point cloud of the material to obtain a stacked surface model; measure and partition the stationary angle of the stacked surface model to obtain a stationary angle distribution map; calculate the curvature and morphological features of the stacked surface model to obtain a surface feature map; analyze the evolution of material morphology using the surface feature map and the stationary angle distribution map to obtain a morphological change trend map; perform stability assessment and index calculation based on the morphological change trend map to obtain a particle stability index matrix.
[0008] Step S3: Perform intelligent prediction of flow risk on the flow eigenvector map and particle stability index matrix to obtain the spatiotemporal map of flow health;
[0009] Step S4: Perform risk assessment and adjustment of control parameters on the spatiotemporal diagram of the flow health to obtain a set of parameter adjustment schemes; execute the set of parameter adjustment schemes and evaluate and provide feedback on the effects to obtain the response curves of the control parameters.
[0010] This invention, through multi-point high-speed visual perception and advanced image processing, overcomes the limitations of traditional sensors, capturing microscopic flow details of materials (texture, velocity, early anomalies) to achieve precise and comprehensive perception of flow states and provide high-quality basic data. It utilizes 3D vision technology to accurately construct a packing morphology model, analyzing its static geometry (stationary angle, curvature) and dynamic evolution characteristics, quantifying material packing stability, and providing direct evidence for predicting anomalies such as bridging and collapse. By integrating flow and packing data and learning historical patterns, it intelligently predicts the risk probability and time window of various anomalies, achieving a shift from passive response to proactive early warning, and visually presenting the risk state through a flow health spatiotemporal map. Based on predicted risks, combined with material characteristics and parameter influences, it intelligently optimizes and adaptively adjusts equipment control parameters (vibration, airflow, etc.) to achieve precise intervention and coordinated control. Through continuous learning via effect evaluation, it improves control performance and effectively resolves flow anomalies. Therefore, this invention provides a digital visualization control method for automated equipment. By introducing a flow perception module (using machine vision to collect microscopic flow characteristics) and a packing morphology modeling module (using 3D perception technology to analyze packing stability), it overcomes the limitations of traditional perception, achieving in-depth and comprehensive perception of material flow states. Furthermore, by integrating these multi-source sensing data through the flow risk prediction module for intelligent analysis and time-series prediction, early warning of potential anomalies is achieved. Finally, based on accurate risk prediction, intelligent parameter adjustment is performed through the adaptive control module, realizing a shift in control paradigm from passive response to proactive prevention, significantly improving the stability and accuracy of powder material conveying and packaging processes.
[0011] Preferably, the present invention also provides a digital visualization control system for automated equipment, for executing the digital visualization control method for automated equipment as described above, the digital visualization control system for automated equipment comprising:
[0012] The flow perception module is used to acquire and preprocess image sequences of automated material conveying equipment to obtain an enhanced flow image set; extract flow texture features from the enhanced flow image set to obtain a texture feature data set; calculate the velocity distribution of the enhanced flow image set to obtain a velocity field distribution map; and extract flow features from the velocity field distribution map and the texture feature data set to obtain a flow feature vector map.
[0013] The packing analysis module is used to construct a 3D point cloud of the material to obtain a packing surface model; to measure and partition the packing surface model to obtain a static angle distribution map; to calculate the curvature and morphological characteristics of the packing surface model to obtain a surface feature map; to analyze the evolution of material morphology using the surface feature map and the static angle distribution map to obtain a morphological change trend map; and to perform stability assessment and index calculation based on the morphological change trend map to obtain a particle stability index matrix.
[0014] The risk prediction module is used to intelligently predict flow risks from the flow pattern feature vector map and particle stability index matrix, and obtain a spatiotemporal map of flow health.
[0015] The parameter control module is used to perform risk assessment and adjustment of control parameters on the spatiotemporal diagram of flow health, and obtain a set of parameter adjustment schemes; execute the set of parameter adjustment schemes and perform effect evaluation and feedback to obtain the control parameter response curves.
[0016] This digital visualization control system for automated equipment deeply integrates four modules: flow perception, accumulation analysis, risk prediction, and parameter control. It constructs a complete closed loop encompassing comprehensive perception from microscopic material flow to macroscopic accumulation morphology, learning from historical data to intelligent prediction of future risks, and proactive adaptive control. Through high-resolution image acquisition and advanced algorithms, the system overcomes the limitations of traditional sensors in perceiving material flow states, achieving precise capture of texture, velocity, fluctuations, and early anomalies. Simultaneously, it utilizes 3D perception technology to accurately model material accumulation morphology, quantifying its geometric features, dynamic changes, and stability, providing deep information about the accumulation state that traditional methods cannot obtain. Based on this multi-source, high-dimensional perception data, the system can intelligently identify anomaly precursor patterns, calculate and visualize the flow health and risk probability of materials in time and space, and provide early warnings of potential blockages, bridging, and other anomalies, thus transforming post-event handling into pre-event prevention. Ultimately, based on accurate risk prediction results and combined with knowledge of material characteristics and parameter influences, the system automatically or recommends the optimal combination of control parameters to achieve adaptive and refined control of equipment such as vibration, airflow, and conveying speed. It continuously evaluates the effects, accumulates experience, effectively maintains the stability of material flow, significantly improves the uniformity and accuracy of conveying and packaging, and reduces the risk of unplanned equipment downtime and damage. Attached Figure Description
[0017] Figure 1 This is a flowchart illustrating the steps of a digital visualization control method for automated equipment.
[0018] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0019] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0020] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.
[0021] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0022] In this embodiment of the invention, reference Figure 1 The diagram shown is a flowchart illustrating the steps of the digital visualization control method for automated equipment according to the present invention. In this example, the digital visualization control method for automated equipment includes the following steps:
[0023] Step S1: Acquire and preprocess image sequences of the automated material conveying equipment to obtain an enhanced flow image set; analyze the flow texture and velocity distribution of the enhanced flow image set to obtain a texture feature data set and a velocity field distribution map; extract flow characteristics from the texture feature data set and the velocity field distribution map to obtain a flow feature vector map.
[0024] In this embodiment of the invention, high-speed industrial cameras are synchronously installed at key locations in the material conveying process to acquire high-definition image sequences at a frequency of 100 frames per second, forming an original flow image set. These images are then enhanced and noise-reduced using methods such as adaptive histogram equalization and median filtering to obtain an enhanced flow image set. A Gabor filter bank is used for multi-scale decomposition and texture statistics calculation (such as mean and variance), combined with methods such as local binary mode and gray-level co-occurrence matrix, to calculate the surface roughness, directional intensity, and texture statistics of the material, forming a texture statistical feature set. By analyzing the local spatial variations or temporal variations of texture statistical features in consecutive image frames, the dynamic fluctuation intensity of the material surface is calculated, resulting in a dynamic fluctuation feature map. Based on the fluctuation pattern, anomalous regions such as agglomeration and adhesion are identified, yielding a characteristic anomaly marker map. Using Shi-Tomasi corner detection and the pyramid Lucas-Kanade optical flow algorithm, the pixel displacements of feature points in the enhanced image set are calculated. After bidirectional consistency checks and temporal smoothing filtering, these displacement vectors are subjected to spatial inverse distance weighted interpolation to obtain the initial velocity field. The initial velocity field is corrected using the directional information in the texture feature data set to obtain the corrected velocity field. The velocity magnitude and directional variance within the region are then calculated to assess flow uniformity, resulting in a flow uniformity map. The corrected velocity field, flow uniformity map, roughness distribution map, directional analysis map, dynamic fluctuation feature map, and characteristic anomaly marker map obtained from texture analysis are fused using multi-dimensional features to form a flow characteristic vector map describing the microscopic flow state of the material, containing multi-dimensional information such as velocity, texture, fluctuation, and anomaly regions.
[0025] Step S2: Construct a 3D point cloud of the material to obtain a stacked surface model; measure and partition the stationary angle of the stacked surface model to obtain a stationary angle distribution map; calculate the curvature and morphological features of the stacked surface model to obtain a surface feature map; analyze the evolution of material morphology using the surface feature map and the stationary angle distribution map to obtain a morphological change trend map; perform stability assessment and index calculation based on the morphological change trend map to obtain a particle stability index matrix.
[0026] In this embodiment of the invention, a structured light projector and a binocular camera system are installed inside the silo to simultaneously acquire images of the material surface with specific structured light patterns. Using calibration parameters and the principle of triangulation, the three-dimensional spatial coordinates of each point on the material surface are calculated based on the stripe distortion and matching points in the image, resulting in an original accumulated point cloud. The original point cloud is then subjected to statistical outlier removal and voxel downsampling optimization, and a Poisson surface reconstruction algorithm is used to convert the discrete point cloud into a continuous triangular mesh accumulated surface model. The accumulated surface model is then translated and scaled in the coordinate system so that the bottom center is located at the origin, generating a standardized accumulated model. Two-dimensional profiles are extracted by radiating radial rays outward from the center, and the global stationary angle is calculated by fitting straight lines to the profiles. The surface is divided into polar coordinate mesh regions, and the angle between the normal vector of the local plane fitting and the Z-axis is calculated for each region as the local stationary angle, forming a local stationary angle table. Spatial interpolation is then performed to obtain a continuous distribution of stationary angles. Based on neighborhood quadratic surface fitting, the normal curvature tensor of each vertex is calculated to obtain the principal curvatures k1 and k2, the Gaussian curvature K = k1 × k2, and the average curvature H = (k1 + k2) / 2, forming curvature field data. Based on this, the surface is divided into planar, elliptical, hyperbolic, and parabolic morphological regions, resulting in a morphological partitioning map. Simultaneously, curvature extrema are extracted as morphological feature points, and principal direction vectors are traced to form a network of feature lines such as ridges and valleys. Based on simplified material mechanics principles, combined with curvature, morphology, and feature line information, the relative stress level at each point on the accumulation surface is estimated, resulting in a stress distribution map. Potential flow channels, bridging risk zones, and other shape characteristics are identified, resulting in a shape characteristic map. The surface feature maps and stationary angle distribution maps at multiple time points are aligned temporally and spatially to form a time-series feature data cube. The time-series rate of change, fluctuations, and abrupt change points of features such as stationary angle, curvature, and height (density) in the cube are analyzed to obtain stationary angle evolution feature maps, curvature evolution feature maps, and density evolution maps. By analyzing the mutual influence strength among various evolutionary characteristics through cross-correlation and causal relationship analysis, a parameter interaction influence diagram is obtained. Temporal patterns similar to historical anomaly precursor patterns are identified, resulting in a characteristic correlation time series diagram. Combining the parameter interaction influence diagram and the characteristic correlation time series diagram, a morphological change trend diagram is generated, describing the current dynamics of the packing state and predicting its evolution direction. Finally, based on the distribution of rest angle magnitude, curvature characteristics, estimated stress, and morphological change trends, a material packing stability assessment model is constructed, calculating the stability index and critical instability threshold for each region, forming a particle stability index matrix.
[0027] Step S3: Perform intelligent prediction of flow risk on the flow eigenvector map and particle stability index matrix to obtain the spatiotemporal map of flow health;
[0028] In this embodiment of the invention, the flow pattern feature vector map and the particle stability index matrix are spatially correlated and temporally aligned to form a unified data structure containing information on both flow pattern and accumulation morphology. Z-score standardization is then performed to obtain a state fusion dataset. Records of anomaly events marked in the historical operational database are acquired, and the state fusion dataset from a period prior to the anomaly occurrence is traced back to extract feature time series related to the anomaly location. An anomaly precursor pattern library containing precursor patterns of various anomaly types (blockage, bridging, etc.) is constructed. Dynamic time warping (DTW) similarity calculation is performed between the current and recent historical state fusion datasets and historical precursor patterns in the anomaly precursor pattern library to obtain an anomaly similarity matrix, quantifying the similarity between the current state and each anomaly precursor. Based on the anomaly similarity matrix and the current state fusion dataset, a pre-trained machine learning model (such as a neural network classifier) is used to calculate the risk probability of different types of anomalies occurring in different spatial regions, resulting in a risk probability distribution map. The risk probability and its rate of change are analyzed, and combined with anomaly similarity and the evolution speed of historical patterns, the expected time window for anomaly events in high-risk areas is predicted, resulting in an anomaly early warning timetable. Finally, based on the risk probability and the urgency of the warning time, a health score between 0 and 100 is constructed to form a data structure that reflects the quantitative indicators of material flow status in time and space. The data is then visualized through color coding, icon overlay, and other methods to obtain a spatiotemporal map of flow health.
[0029] Step S4: Perform risk assessment and adjustment of control parameters on the spatiotemporal diagram of the flow health to obtain a set of parameter adjustment schemes; execute the set of parameter adjustment schemes and evaluate and provide feedback on the effects to obtain the response curves of the control parameters;
[0030] In this embodiment of the invention, the spatiotemporal map of flow health is analyzed. Based on the health score and warning time, risk areas with health scores below a set threshold are identified and classified into emergency, high, medium, and low risk levels. Simultaneously, the main anomaly types and their impact ranges are marked to form a risk intervention map. A list of adjustable control parameters for automated equipment is obtained, and a parameter impact knowledge base built based on historical data and process knowledge is consulted. The parameter impact weights and directions corresponding to the anomaly types and areas involved in the current risk intervention map are extracted to generate a parameter impact matrix, quantifying the expected impact of each parameter on different risks. Based on the risk intervention map and the parameter impact matrix, the sensitivity of each risk area to each control parameter is calculated, resulting in a parameter sensitivity ranking table. Physical property data of the currently processed material is obtained, and referring to the physical property data and the parameter sensitivity ranking table, initial adjustment ranges and suggested values for various control parameters such as vibration (frequency, amplitude, duration), airflow (pressure, flow rate, pulse), mechanical conveying (speed), and silo pressure (wall pressure, internal pressure) are defined or suggested. Starting with the recommended parameter values, a multi-objective optimization problem is constructed based on the risk intervention map and parameter influence matrix. Under the premise of meeting equipment constraints and energy consumption limits, the system searches for parameter combinations that maximize overall health or prioritize the resolution of high-risk areas, resulting in a set of collaborative parameter combination schemes. Based on the intervention priority and anomaly warning schedule in the risk intervention map, the execution sequence of the collaborative parameter combination schemes is planned, determining the start time, target value, and rate of change for each parameter adjustment, forming a set of parameter adjustment schemes containing detailed operational instructions. The schemes in the parameter adjustment scheme set are evaluated for effectiveness (predicted risk improvement, energy consumption, load, and stability), and calibrated using historical parameter response data. Based on preset rules, a decision is made whether to automatically execute the scheme with the highest evaluation score or recommend it to the operator through a visual interface, resulting in execution decision instructions. Following the execution decision instructions, control commands are sent to the equipment via the industrial communication interface to gradually adjust parameter values, and the actual values of the parameters and the corresponding changes in key indicators of the flow health spatiotemporal map are recorded in real time, forming a parameter execution trajectory. By comparing the spatiotemporal diagrams of flow health before and after parameter adjustment, the improvement effects such as enhanced health and reduced risk areas are quantified. The time lag and duration between parameter changes and state response are analyzed, and a quantitative model or relationship between parameters and responses is established. These effect evaluation results and parameter-response relationship data are stored to form control parameter response curves, which are used to optimize control strategies and accumulate experience.
[0031] Of particular importance is the extraction of flowing texture features, specifically as follows:
[0032] Multi-scale decomposition and feature domain transformation are performed on the enhanced flow image set to obtain a multi-scale feature map set;
[0033] Texture statistics are calculated on the multi-scale feature map to obtain the texture statistical feature set;
[0034] The surface roughness index of the material is calculated from the texture statistical feature set to obtain a roughness distribution map;
[0035] Directional features are extracted from the texture statistical feature set to obtain a directional analysis map;
[0036] Dynamic texture temporal analysis of the enhanced flow image set is performed based on the texture statistical feature set to obtain a dynamic fluctuation feature map;
[0037] Based on the dynamic fluctuation feature map, special material characteristics are identified in the enhanced flow image set to obtain a characteristic anomaly marker map;
[0038] By integrating roughness distribution map, directionality analysis map, dynamic fluctuation feature map, and characteristic anomaly marker map, a texture feature data set is obtained;
[0039] In this embodiment of the invention, an enhanced flow image set is obtained. For each enhanced image in the image set, a set of Gabor filters is applied for convolution. A Gabor filter is a linear filter whose kernel function is a sinusoidal plane wave with a Gaussian envelope, exhibiting good locality in both the frequency and spatial domains, making it suitable for texture analysis. A set of Gabor filters with different parameters is used, covering multiple scales (e.g., wavelengths set to 4, 8, 16, and 32 pixels, representing different texture fineness) and multiple directions (e.g., directions set to 0°, 45°, 90°, and 135°). For each selected scale and direction, a corresponding Gabor filter kernel is constructed and convolved with the input image. The convolution result is a response image, whose pixel values reflect the similarity between the input image and the filter kernel at that location, scale, and direction. The Gabor filter response images for all scales and directions are collected to form a multi-scale feature map set. For example, using 4 scales and 4 directions, each input image will generate 16 Gabor response images. To acquire a multi-scale feature map set, for each response image in the set, its space is divided into non-overlapping regions of fixed size (e.g., 32×32 pixel patches). Within each patch, statistics of the pixel values are calculated, such as the mean and variance. The mean reflects the dominant response intensity of the region at a specific scale and orientation, while the variance reflects the dispersion of the response values. The mean and variance of each patch across all scales and orientations are organized to form the texture statistical feature vector for that patch. The texture statistical feature vectors of all patches are collected to form the texture statistical feature set. The texture statistical feature set is a multi-dimensional data set containing spatial location information, such as a three-dimensional array with dimensions of [image height / 32] × [image width / 32] × [number of scales × number of orientations × 2 (mean and variance)]. The roughness of a material surface is correlated with the intensity of high-frequency components in the image. In the texture statistical feature set, the variance of the Gabor filter response corresponding to smaller scales (higher frequencies, such as wavelengths of 4 and 8 pixels) typically reflects the complexity and roughness of the local texture. For each spatial patch, calculate the weighted sum of its Gabor response variances at a selected high-frequency scale, which serves as the roughness index for that patch. For example, roughness index = w1 × Var(Scale4, 0°) + w2 × Var(Scale4, 45°) + ... + w n×Var(Scale8, 135°), where Var represents the response variance at the corresponding scale and direction, and wi is a preset weighting coefficient. The calculated roughness indices are normalized to a range of 0 to 1, where 0 represents the smoothest surface and 1 represents the roughest surface. These normalized roughness indices are arranged according to the location of spatial blocks to form a roughness distribution map. The roughness distribution map is a two-dimensional image or matrix, where each pixel or cell represents the roughness index of the corresponding region. A texture statistical feature set is obtained; the directionality of material flow or the anisotropy of surface texture can be measured by comparing the intensity of Gabor filter responses in different directions. For each spatial block and each scale, the mean Gabor response in different directions (0°, 45°, 90°, 135°) at that scale is compared. The direction with the largest mean is identified as the dominant direction for that block at that scale. The difference or ratio between the mean response in the dominant direction and the mean response in other directions at that scale is calculated as the directional intensity of that block at that scale. For example, directional intensity = (MaxMean - MinMean) / (MaxMean + MinMean). By integrating directional information at different scales (e.g., weighted averaging or analysis at a specific scale), the overall principal direction and directional intensity of each block are determined. The principal direction angle and directional intensity values of each block are arranged spatially to form a directional analysis map. The directional analysis map contains two channels: one channel stores the principal direction angle, and the other channel stores the directional intensity. Based on the texture statistical feature set and the enhanced flow image set, dynamic texture refers to the pattern of texture change over time, reflecting fluctuations, ripples, or pulsations on the material surface. For each spatial block, the changes in its texture statistical features (e.g., the overall response mean or the response mean at a specific scale / direction) are tracked over a continuous time series (e.g., the most recent 10 frames). The variance or mean absolute difference of these time series is calculated as the dynamic fluctuation intensity of that block. For example, fluctuation intensity = temporal variance(Mean(Block,t), t∈[t0-9,t0]). Alternatively, on an enhanced flow image set, the average absolute difference in pixel grayscale between two consecutive frames within each small block is calculated; this value reflects the degree of instantaneous change in that region. These fluctuation intensity values are arranged according to the spatial block positions to form a dynamic fluctuation feature map. The dynamic fluctuation feature map is a two-dimensional image or matrix, where each pixel or cell represents the dynamic fluctuation intensity of the corresponding region. By acquiring the dynamic fluctuation feature map and the enhanced flow image set, and analyzing the texture patterns within both sets, patterns associated with specific material anomalies can be identified. For example, regions with consistently low fluctuation intensity (fluctuation intensity below a set threshold ε3) indicate material agglomeration or adhesion to the wall surface; low-fluctuation regions with clear boundaries that contrast sharply with surrounding high-fluctuation regions indicate the formation of stable bridging arches. Regions with periodic high fluctuation intensity are associated with pulsed flow.Using pre-established rules or a trained classifier (based on anomaly samples such as clumps, bridging, and stratification marked in historical data), dynamic fluctuation feature maps and texture statistical feature sets are analyzed and pattern matched. Regions conforming to specific anomaly patterns (e.g., specific fluctuation intensity ranges, specific texture combinations, specific spatial morphologies) are identified. These identified regions are labeled and assigned anomaly type tags (e.g., "suspected clumping," "potential bridging area," "signs of stratification"). This set of labeled regions and tags constitutes a characteristic anomaly label map. Texture-related feature data from different sources and of different types—roughness distribution maps, directional analysis maps, dynamic fluctuation feature maps, and characteristic anomaly label maps—are integrated. For each spatial location (e.g., the center of each 32×32 pixel patch), its corresponding roughness index, principal direction angle, directional intensity, dynamic fluctuation intensity, and characteristic anomaly label information (e.g., anomaly type represented by a one-hot encoded vector) are combined to form a multi-dimensional feature vector. These feature vectors from spatial locations are organized into a structured dataset. For example, a multi-channel image can be constructed, where each channel represents a texture feature map, or a grid-like data structure can be built, where each grid cell stores a vector containing all texture features. This integrated dataset is called the texture feature dataset, which comprehensively and quantitatively describes the microstructure, dynamic behavior, and potential anomalies of the material surface.
[0040] Most importantly, the feature point selection is specifically as follows:
[0041] Pyramid optical flow calculations are performed on candidate feature points to obtain point displacement vectors;
[0042] A vector reliability assessment is performed on the point displacement vector to obtain the selected displacement vector;
[0043] The selected displacement vectors are smoothed over time to obtain smoothed displacement vectors.
[0044] The initial velocity field is obtained by interpolating the smooth displacement vector in vector space.
[0045] The initial velocity field is corrected by texture-guided velocity correction using texture feature data sets to obtain the corrected velocity field;
[0046] A flow uniformity analysis was performed on the modified velocity field to obtain a flow uniformity diagram;
[0047] The velocity field data is organized from the corrected velocity field and flow uniformity map to obtain a velocity field distribution map;
[0048] In this embodiment of the invention, the current frame and the previous frame images from the enhanced flow image set are acquired, along with a set of candidate feature points detected on the material surface in the current frame (e.g., approximately 1000 high-quality corner points detected by the Shi-Tomasi corner detector). An iterative Lucas-Kanade sparse optical flow algorithm based on image pyramids is used to calculate the pixel displacement of these candidate feature points between the previous and current frames. First, a Gaussian image pyramid is constructed for the current and previous frames, for example, generating a 3-layer pyramid, with each layer's image size being 0.5 times that of the previous layer. The tracking process begins at the top layer of the pyramid (the coarsest scale). For each candidate feature point, a fixed-size window (e.g., 21×21 pixels) is used at its corresponding position in the previous frame (considering pyramid scaling). The initial displacement is estimated by minimizing the sum of squares of the pixel grayscale differences within the window. This displacement estimate is then passed to the next layer of the pyramid (a finer scale) as the initial guess for tracking at that layer, and iterative optimization is performed. This process is repeated until the bottom layer of the pyramid. The algorithm outputs the pixel displacement vector (Δx, Δy) of each candidate feature point between the previous frame and the current frame, along with a status flag indicating whether the point has been successfully tracked. The set of displacement vectors of these successfully tracked points constitutes the point displacement vector. The calculated point displacement vectors and their corresponding tracking status flags are obtained. A reliability check is performed on these displacement vectors. First, points whose tracking status flags indicate failure are removed. Then, a bidirectional optical flow consistency check is performed: for the displacement vector (Δx, Δy) calculated from the previous frame to the current frame, the optical flow vector (Δx', Δy') calculated from the current frame to the previous frame is also calculated. If the Euclidean distance between the original displacement vector (Δx, Δy) and the reverse displacement vector (-Δx', -Δy') is greater than a set threshold (e.g., 1.0 pixels), the displacement vector is considered unreliable and is removed. The Euclidean distance calculation formula is... In addition, the magnitude of the displacement vectors can be checked, and outliers that are too large or too small (e.g., greater than 10% of the image diagonal length or less than 0.1 pixels) can be removed. After these reliability assessments, the set of retained displacement vectors is the filtered displacement vector. The filtered displacement vectors for the current frame, as well as the historical filtered displacement vectors of the same batch of feature points in previous consecutive frames (e.g., the past 5 frames), are obtained. For each feature point filtered and retained in the current frame, a temporal smoothing filter is applied to its historical filtered displacement vectors to reduce instantaneous velocity fluctuations caused by inter-frame noise. An exponential moving average (EMA) filter is used for smoothing. For the displacement vector v of point p at time t... pt =(Δx) pt ,Δy pt Its smooth displacement vector V pt The calculation formula is: V pt =α×v pt +(1-α)×V pt-1Where α is a smoothing factor (e.g., set to 0.5), V pt-1 This is the smoothed displacement vector at time t-1. For points appearing for the first time or with insufficient historical records, the current filtered displacement vector can be directly used as the smoothed displacement vector. The set of displacement vectors after time smoothing is the smoothed displacement vector. The smoothed displacement vector exists only at the tracked feature point locations. To obtain a continuous velocity field covering the entire material flow region, spatial interpolation is required. First, a regular grid (e.g., one grid point every 10 pixels) is defined within the region of the enhanced flow image. For each grid point P, interpolation is performed using the smoothed displacement vector within its neighborhood (e.g., within a radius of 50 pixels). The inverse distance weighted (IDW) interpolation method is used. The initial displacement vector V of grid point P. p Equal to the smooth displacement vector V of all neighborhood feature points i i Weighted average: V p =∑ i (w i ×V i ) / ∑ i w i where the weight w i It is inversely proportional to the distance d(P,i) from feature point i to grid point P, for example, w i = 1 / d(P,i) 2Feature points that are farther away contribute less to the interpolation. For regions without sufficient neighborhood feature points, they can be marked as unknown or interpolated over a wider range. The interpolation result is a dense or semi-dense grid of displacement vectors, representing the estimated instantaneous displacement of the material at each grid point. Dividing the displacement vector by the inter-frame time interval (e.g., 1 / 100 sec = 0.01 sec) converts it to a velocity vector (pixels / sec or millimeters / sec, if the camera is calibrated), forming the initial velocity field. Based on the initial velocity field and the texture feature data set (particularly the directional analysis plot within it), texture directionality can provide additional information about the local flow direction of the material. For each grid point in the initial velocity field, its corresponding spatial location is found in the principal direction angle and directional intensity value in the texture feature data set. In regions with high texture directionality intensity, if the initial velocity vector deviates significantly from the principal direction, correction is required. For example, assuming the material tends to flow along the principal direction of the texture (e.g., particles sliding on a textured surface). The initial velocity vector V is decomposed into components parallel to the principal direction V / / and perpendicular to the principal direction V⊥. The corrected velocity vector V' can be calculated using V' = V / / + λ × V⊥, where λ is a correction factor inversely proportional to the texture directionality (the stronger the directionality, the closer λ is to 0, and the more the vertical component is suppressed). Alternatively, the direction of the velocity vector can be directly adjusted to be closer to the dominant direction while maintaining its magnitude or adjusting its magnitude according to texture uniformity. The correction process can be based on a pre-trained model or heuristic rules. The velocity field after texture-guided correction is the corrected velocity field. To assess the uniformity of material flow, the area covered by the corrected velocity field is divided into smaller analysis windows (e.g., 64 × 64 pixels). Within each analysis window, the standard deviation of the velocity vector magnitude (velocity modulus) σ|V| and the circular square deviation of the velocity vector direction σθ are calculated. The standard deviation of the velocity magnitude reflects the consistency of the flow rate within the region, and the circular square deviation of the direction reflects the consistency of the flow direction within the region. For example, the circular square deviation σθ = 1 - |∑ i exp(j×θ i )| / N, where θ iσ|V| is the direction angle of the i-th velocity vector within the window, N is the number of velocity vectors within the window, and j is the imaginary unit. Combining σ|V| and σθ (e.g., by weighted averaging or multiplication) yields the flow non-uniformity index for each analysis window. This index is normalized (e.g., between 0 and 1), and the uniformity index Uniformity = 1 - NonUniformityIndex is calculated. These uniformity indices are arranged according to the position within the analysis window to form a flow uniformity map. The flow uniformity map is a two-dimensional image or matrix, where each pixel or cell represents the degree of flow uniformity in the corresponding region. The modified velocity field (a grid containing two-dimensional velocity vectors) is used as the primary data layer. The flow uniformity map is used as a supplementary data layer. This data is organized into a unified data structure that represents the velocity vector at each spatial location (grid point or analysis window) and the flow uniformity information for that region. For example, a two-dimensional grid containing velocity vector information can be created, and an attribute field can be attached to each cell in the grid to store the corresponding flow uniformity index. Alternatively, a multi-channel data structure can be created, with the first two channels storing the X and Y components of the velocity vector, and the third channel storing the flow uniformity index. This well-organized data structure, including velocity vectors and uniformity information, constitutes a velocity field distribution map. The velocity field distribution map comprehensively describes the spatial velocity distribution and uniformity characteristics of the material flow.
[0049] Preferably, step S1 specifically includes:
[0050] Step S11: Install high-speed cameras at key locations of the automated material conveying equipment to synchronously acquire high-definition image sequences of material flow at a frequency of 60-120 frames per second, forming an original flow image set;
[0051] Step S12: Perform image enhancement and noise reduction on the original flow image set to obtain an enhanced flow image set;
[0052] Step S13: Extract flow texture features from the enhanced flow image set to obtain a texture feature data set;
[0053] Step S14: Calculate the velocity distribution of the enhanced flow image set based on the texture feature data set to obtain a velocity field distribution map;
[0054] Step S15: Detect clusters and abnormal regions in the velocity field distribution map and texture feature data set to obtain an abnormal region marker map;
[0055] Step S16: Perform multi-dimensional feature fusion and map generation on the texture feature data group, velocity field distribution map and abnormal region marker map to obtain the flow state feature vector map.
[0056] In this embodiment of the invention, industrial high-speed cameras (e.g., Baslerac A2040-120kc) are used at five key locations in the automated material handling equipment, including the feed inlet, pipe bends, and silo outlets. These cameras are mounted on custom brackets, and the shooting angles are adjusted to cover the main areas of material flow. Each camera is equipped with an LED ring light to provide uniform and stable illumination. The cameras are set to a frame rate of 100 frames per second and a resolution of 1920×1080 pixels. Multi-camera synchronous acquisition software on an industrial computer ensures that all cameras begin acquiring data simultaneously and stores the images as a continuous sequence of PNG files. The file names include a location identifier (e.g., "feed inlet"), an acquisition timestamp (accurate to milliseconds), and a frame number, such as "feed inlet_20230101_100000_00001.png". This collection of PNG files constitutes the original flow image set.
[0057] The acquired raw flow image set was preprocessed using an image processing program. First, an Adaptive Histogram Equalization (CLAHE) algorithm was applied to each image to enhance local contrast and compensate for uneven illumination. The CLAHE algorithm divides the image into multiple small regions and performs histogram equalization within each region. Then, bilinear interpolation is used to smooth the region boundaries, with parameters set to a block size of 8×8 pixels and a contrast limiting factor of 2.0. Next, gamma correction was applied to adjust the overall contrast, with a gamma value set to 0.8, making details in the darker areas of the image clearer. Finally, a median filter with a kernel size of 3×3 pixels was used to remove noise from the image, eliminating salt-and-pepper noise generated during acquisition. After these processing steps, an enhanced flow image set was generated, with improved image quality and easier identification of material boundaries and textures.
[0058] Texture features of material surfaces are extracted using an enhanced flow image set. For each enhanced image, it is first converted to a grayscale image. Then, the Gray-Level Co-occurrence Matrix (GLCM) is calculated to quantify the statistical properties of the texture. The GLCM is calculated in four directions (0°, 45°, 90°, 135°) and at a distance of one pixel. Based on the GLCM, four texture statistics—contrast, correlation, energy, and uniformity—are calculated. For example, contrast = ∑ i ∑ j (ij) 2×P(i,j), where P(i,j) is the probability that gray levels i and j co-occur at a specified direction and distance. Simultaneously, a Local Binary Pattern (LBP) operator with a radius of 1 and 8 neighborhood points is used to calculate the LBP value for each pixel, generating an LBP histogram for the entire image. This histogram reflects the distribution of local texture patterns in the image. Furthermore, the average or variance of the absolute difference between pixel gray levels in two consecutive frames is calculated as an indicator of surface fluctuation intensity. These GLCM statistics, LBP histograms, and fluctuation intensity indicators are combined to form a texture feature data set.
[0059] Based on an enhanced flow image set and texture feature dataset, the flow velocity distribution of the material is calculated. The Shi-Tomasi corner detection algorithm is used to detect approximately 1000 trackable corners with good features in the current frame. Then, the Lucas-Kanade sparse optical flow algorithm is used to track the positions of these corners in the next frame. The Lucas-Kanade algorithm estimates the optical flow vector by minimizing the sum of squared pixel differences in the neighborhood. For each successfully tracked feature point, its pixel displacement vector (δ) between two consecutive frames is calculated. x ,δ γ By combining camera calibration parameters and frame rate, pixel displacements are converted into velocity vectors in real space. For regions where no feature points are detected, interpolation estimation is performed using the velocities of neighboring feature points. Utilizing local texture uniformity information from the texture feature data set, the feature point detection density is increased in regions with rich texture variations, improving the accuracy of velocity calculation. Finally, a grid-like velocity field distribution map is generated, with each grid point containing a two-dimensional velocity vector, displaying the magnitude and direction of the material's velocity on the image plane.
[0060] Using velocity field distribution maps and texture feature datasets, agglomeration, bridging, or stagnation areas in material flow are detected. First, a velocity threshold (e.g., below 10% of the overall average velocity) is set to identify potential stagnation areas. Pixels with velocity vector sizes below this threshold are marked as low-velocity areas. Simultaneously, local anomalies in texture features are analyzed; for example, the variance of texture uniformity (GLCMEnergie) within a small neighborhood (50×50 pixels) is calculated. When the variance exceeds a set threshold, it indicates abnormal texture changes in the area, suggesting the presence of agglomeration or surface solidification. A logical OR operation is performed on the low-velocity area markers and texture anomaly area markers to obtain anomaly candidate areas. Connectivity component analysis is performed on the anomaly candidate areas to identify independent anomaly regions. The pixel area of each anomaly region is calculated and converted to an actual physical area based on camera calibration. The duration of these regions in consecutive frames is tracked; regions whose duration exceeds a set threshold (e.g., 0.5 seconds) are identified as persistent anomaly regions. Based on the specific performance of the velocity field and texture features, the anomaly regions are classified as agglomeration, bridging, or stagnation. Finally, an anomaly region marker map is generated, which is a data structure containing a list of identified anomaly region bounding boxes, types (such as "clumping" or "stagnation"), location coordinates, actual area, and duration.
[0061] The texture feature data set obtained in step S13, the velocity field distribution map generated in step S14, and the abnormal area marker map detected in step S15 are fused to construct a comprehensive feature vector map describing the material flow state. For each monitoring point (e.g., the feed inlet), at each acquisition time stamp, the texture features (such as GLCM statistics, LBP histogram mean), key statistics of the velocity field distribution map (such as average velocity, velocity variance, maximum velocity), and information of the abnormal area marker map (such as the proportion of total abnormal area, the area of the largest abnormal area, and the number of abnormal areas) at that time are extracted. These values are combined into a high-dimensional feature vector. For example, a feature vector may contain [GLCM contrast mean, LBP histogram kurtosis, surface fluctuation intensity, average velocity magnitude, velocity vector direction variance, proportion of total abnormal area, and area of the largest agglomeration]. These feature vectors calculated at different monitoring points and at different time stamps are spatiotemporally organized to form a multi-dimensional data structure. For example, a tensor can be constructed with dimensions of [number of monitoring points] × [time series length] × [feature vector dimension]. This structured dataset is the flow pattern feature vector map, which provides comprehensive and quantitative flow pattern perception information for subsequent accumulation pattern analysis and flow risk prediction.
[0062] Preferably, the 3D point cloud construction in step S2 specifically involves:
[0063] A specific structured light pattern is projected onto the material surface by a projector, and the camera captures images of the material surface with the structured light pattern from different angles. Then, the spatial coordinate data of each point on the material surface is calculated using the principle of triangulation.
[0064] Construct the original accumulation point cloud of the material accumulation surface based on spatial coordinate data;
[0065] Point cloud optimization and surface reconstruction are performed on the original stacked point cloud to obtain a stacked surface model.
[0066] In this embodiment of the invention, a DLP (Digital Light Processing) projector, such as the Optoma HD28HDR, is installed inside the silo or above the outlet. This projector, controlled by a computing unit, projects a series of stripe patterns according to a preset sequence, such as phase-shifted stripe patterns in the horizontal and vertical directions (a total of 12 patterns, 6 in each direction, including phase shift and grayscale encoding). Simultaneously, two high-resolution industrial cameras, such as the Hikvision MV-CE200-10GM, are installed on either side of the projector. A fixed baseline distance (e.g., 500 mm) exists between the two cameras, and their optical axes form a specific solid angle with the projector's optical axis. The cameras and projector work synchronously via hardware trigger signals, ensuring that both cameras simultaneously acquire images when the projector projects each pattern. The acquired image sequence includes images of the material surface under different structured light patterns. Using calibrated projector and binocular camera system parameters (including the intrinsic parameter matrix Kcam)... i Where i = 1, 2 represents the camera number; the extrinsic parameter matrix [R] i |t i ] represents the rotation R of the camera coordinate system relative to the world coordinate system. i Peaceful movement i The image with striped patterns is processed using the equivalent parameters of the projector (and the projector's parameters). Distorted stripes in the image are identified, and the phase value corresponding to each pixel is calculated using a phase-shift decoding algorithm. The pixel coordinates (u) of the same material surface point in the images from the two cameras are then compared. i ,v iThe corresponding relationship is found through phase matching. Based on the triangulation principle of binocular vision, the world coordinates (X, Y, Z) of a point in three-dimensional space are calculated using known camera intrinsic and extrinsic parameters and the coordinates of the matched pixels. The core of the triangulation principle is that the projection points p1 and p2 of a three-dimensional point P in the images of two cameras are coplanar with the optical centers O1 and O2 of the two cameras. The spatial position of P is determined by solving the geometric relationship formed by these four points. For example, for a pair of matched points p1 and p2, their spatial coordinates (X, Y, Z) satisfy the camera imaging equations: s1p1 = K1[R1|t1]P and s2p2 = K2[R2|t2]P, where s1 and s2 are depth scale factors. The (X, Y, Z) coordinates of P are calculated by minimizing the reprojection error or by direct analytical methods. This process is repeated to calculate a large number of matched points in the image to obtain a set of spatial coordinate data for each point on the material surface.
[0067] The spatial coordinates (X, Y, Z) of each point on the material surface, calculated using the triangulation principle, are organized into an unordered set of points, each representing a discrete sampling point on the material surface. This set of points constitutes the original accumulated point cloud. The point cloud data is stored in a standard format, such as a PLY or PCD file, where each point contains at least (X, Y, Z) coordinate information and may also contain color or intensity information (this embodiment primarily utilizes geometric information). The original accumulated point cloud faithfully records the shape of the material surface obtained through structured light scanning, including issues such as noise points, outliers, or density inhomogeneities.
[0068] The original stacked point cloud is optimized. Statistical filtering methods, such as Statistical Outlier Removal, are applied, with the number of neighboring points K = 50 and the standard deviation multiplier StddevMulThresh = 1.0, to remove outliers whose average distance from their neighbors is too large, reducing noise. Next, VoxelGrid filtering is applied for downsampling, with a voxel size of 5 mm to reduce the number of point clouds, improve subsequent processing efficiency, and make the point cloud density more uniform. Then, the normal vector of each point is estimated, for example, using a covariance analysis method based on K-nearest neighbors (K = 20) to obtain the normalized surface normal vector. Based on the optimized point cloud, an improved Poisson Surface Reconstruction algorithm is used for surface reconstruction. This algorithm treats the point cloud as a gradient field sampling of a certain implicit function, solves a Poisson equation to recover the implicit function, and its zero isosurface is the reconstructed surface. The reconstruction depth is set to depth = 9, the normal vector consistency weight scale = 1.1, and the minimum tree depth min_depth = 5. The algorithm outputs a continuous three-dimensional mesh model, consisting of vertices, edges, and triangular patches, which accurately describes the geometry of the material accumulation. This model is the accumulation surface model, which is used for subsequent analysis of morphological features such as rest angle and curvature.
[0069] Preferably, the stationary angle measurement and zoning process in step S2 specifically includes:
[0070] A coordinate system is established and the center is located for the stacked surface model to obtain a standardized stacked model.
[0071] Extract the radial profile set of the standardized stacking model;
[0072] Global stationary angles are calculated on the radial profile set to obtain global stationary angle data;
[0073] The surface region of the standardized stacking model is divided to obtain a region division map;
[0074] Based on the regional division map, the local rest angle of the standardized stacking model is calculated to obtain the local rest angle table;
[0075] Based on the local static angle table, spatial interpolation of the static angles is performed on the region division map to obtain a continuous distribution of static angles;
[0076] Critical region identification is performed based on the continuous distribution of stationary angles, resulting in critical region markers.
[0077] A static angle distribution map is generated based on the continuous distribution of static angles, critical region markings, and global static angle data.
[0078] In this embodiment of the invention, a stacked surface model (a triangular mesh model) is obtained. First, the bottom of the model is analyzed (e.g., by identifying the vertex with the lowest Z-coordinate or fitting a plane to the vertices of the lowest part). Assuming the bottom of the silo is a horizontal plane, the projection area of the material stack on the horizontal plane is determined. The centroid of this projection area is calculated as the horizontal center point (X0, Y0) of the stack. The entire 3D model is translated along the X and Y axes (-X0, -Y0) so that the horizontal center of the stack is aligned with the origin (0, 0) of the world coordinate system. The Z-coordinate of the lowest point of the stack is set to 0, and the entire model is translated along the Z-axis so that the lowest point is located at or close to the Z=0 plane. Through the above translation operation, a standardized coordinate system is established with the center of the bottom of the stack as the origin and the Z-axis pointing vertically upward. The stacked surface model under this standardized coordinate system is the standardized stacked model. This operation eliminates the influence of the sensor installation position and the shape of the bottom of the silo, facilitating subsequent unified analysis.
[0079] In the standardized packing model, rays are emitted outward from the horizontal center point (0,0) at fixed angular intervals (e.g., every 1 degree) on the horizontal plane. A total of 360 rays are emitted, covering a 360-degree range. For each ray, its intersection with the surface of the standardized packing model's 3D mesh is calculated. The distances of these intersections in the ray direction and their corresponding Z-coordinates (heights) form a 2D sequence. These 2D sequences are combined to form a set of radial profiles. Each profile represents the lateral profile of the material packing along a specific horizontal direction.
[0080] For each two-dimensional profile in the radial profile set (consisting of a series of (distance, height) points), points near the center (e.g., distances less than 10% of the stack radius) and the top (e.g., heights greater than 90% of the maximum height) are ignored because these areas are significantly affected by the feed inlet shape and material flow inertia and do not represent stable stationary angle characteristics. For the remaining set of points, a straight line (height = a × distance + b) is fitted using linear regression, where the arctangent of the slope a, arctan(a), is the stationary angle in that radial direction. To increase robustness, the RANSAC (Random Sample Consensus) algorithm can be used for line fitting to exclude some outliers. The average of the stationary angles in all 360 radial directions is calculated as the global stationary angle of the material stack. The global stationary angle data is a single value representing the overall stacking characteristics of this batch of material under current conditions.
[0081] To analyze local rest angles, the 3D surface of the standardized packing model is divided into multiple sub-regions. The division can be based on a polar coordinate grid; for example, the horizontal angle range (0-360°) can be divided into 16 sectors, and the radial distance range (from the center to the maximum radius) into 3 annular zones. This results in 16 × 3 = 48 regions on the entire packing surface. Each region consists of a set of triangular mesh faces or vertices belonging to that region. This division method considers the centrosymmetry of the packing and facilitates the analysis of local characteristics in different directions and radial positions. The region partitioning diagram is a data structure that records the boundary definition of each region and the mesh face / vertices indices belonging to that region.
[0082] For each sub-region in the region partitioning map, extract all vertices belonging to that region. For these vertex sets, use Principal Component Analysis (PCA) or a plane fitting algorithm to calculate a best-fit plane to approximate the local surface of the region. Calculate the normal vector of this fitted plane. The angle between this normal vector and the Z-axis of the normalized coordinate system is the local rest angle of the region. Organize the calculated local rest angle values for all sub-regions into a table or matrix, where rows represent radial rings, columns represent angular sectors, and table cells store the local rest angle value for the corresponding region. This table is the local rest angle table.
[0083] The local stationary angle table provides average stationary angle information for discrete regions. To obtain an estimate of the stationary angle at any point on the entire packing surface, spatial interpolation is required. The stationary angle values at the center points of each region in the local stationary angle table are used as known data points. An inverse distance weighted (IDW) interpolation method is employed to calculate the stationary angles for all vertices of the standardized packing model. The interpolated stationary angle value for a vertex is equal to the weighted average of the stationary angle values at the known region center points, with the weights inversely proportional to the negative power (e.g., square) of the distance to that vertex. The interpolation result is a continuously distributed stationary angle field, associating a stationary angle value with each vertex of the standardized packing model. This continuously distributed stationary angle data is the stationary angle continuous distribution.
[0084] Set a critical rest angle threshold (e.g., 38 degrees based on historical material data or experience). Identify regions (composed of continuous grid patches) within a continuous distribution of rest angles where all rest angle values are greater than or equal to this critical threshold. Perform connectivity component analysis on these regions to identify independent regions with rest angles above the threshold. These regions indicate that the material accumulation is in an unstable state, with potential risks of collapse or bridging. Mark each identified critical region, recording its spatial location, area, maximum rest angle value, and deviation from the critical threshold. This set of marked data constitutes the critical region label.
[0085] Visualize the continuous distribution data of the stationary angle, for example, using a color map where different colors represent different stationary angle values, visually showing the spatial variation of the stationary angle across the entire stack surface. Overlay critical region markers onto the visualization, highlighting high-risk areas with striking colors or boundaries. Simultaneously, include global stationary angle values in the illustration or accompanying data structure. Integrating this information forms a stationary angle distribution map, a comprehensive data product that fully describes the overall and local stationary angle characteristics of the material stack, as well as potentially unstable regions.
[0086] Preferably, the calculation of curvature and morphological features in step S2 is specifically as follows:
[0087] Differential geometric parameters are calculated on the stacked surface model to obtain curvature field data;
[0088] Surface morphology is classified based on curvature field data to obtain a morphology partition map;
[0089] Feature points are extracted and analyzed from the morphological partition map to obtain morphological feature points;
[0090] Feature lines are extracted from the curvature field data to obtain a feature line network;
[0091] Stress distribution is estimated based on curvature field data, morphological partitioning diagram, and characteristic line network to obtain a stress distribution map.
[0092] Shape characteristic analysis is performed on curvature field data, morphological partitioning diagram, and stress distribution diagram to obtain shape characteristic diagram;
[0093] Generate surface feature maps based on shape property maps and stress distribution maps.
[0094] In this embodiment of the invention, a stacked surface model (a triangular mesh model) is obtained. For each vertex v of the model... i Fit a quadratic surface, such as a parabola z = ax, within its neighborhood (e.g., using a spherical neighborhood with a radius of 10 mm or a neighborhood containing its k = 10 nearest neighbor vertices). 2 +by 2 +cxy+dx+ey+f, where (x,y,z) is v i The local coordinates of the neighborhood points are calculated. The fitting parameters a, b, c, d, e, and f are solved using the least squares method. Based on the coefficients of the fitted surface, the normal curvature tensor at each vertex is calculated. Eigenvalue decomposition is performed on the normal curvature tensor to obtain two principal curvatures k1 and k2 (by convention |k1|≤|k2|) and their corresponding orthogonal principal direction vectors. The Gaussian curvature K = k1 × k2 and the mean curvature H = (k1 + k2) / 2 are calculated based on the principal curvatures. The principal curvatures, Gaussian curvatures, mean curvatures, and principal direction vectors calculated for each vertex are stored to form curvature field data.
[0095] The surface of the stacked surface model is classified using the calculated curvature field data (Gaussian curvature K and mean curvature H) for each vertex. A small threshold ε1 = 10 is set. -4 and ε2 = 10 -4 For each vertex, it is classified according to the signs of K and H: if |K| < ε1 and |H| < ε2, it is classified as a planar region; if K > ε1, it is classified as an elliptical region (convex if H > ε2, concave if H < -ε2); if K < -ε1, it is classified as a hyperbolic region (saddle point); if |K| < ε1 and |H| ≥ ε2, it is classified as a parabolic region (cylindrical or grooved surface). Each vertex is labeled with its morphological category (e.g., integers 1-5 represent planar, convex elliptical, concave elliptical, hyperbolic, and parabolic). This set of vertex labels constitutes the morphological partitioning map.
[0096] Based on morphological zoning maps and curvature field data, feature points with special geometric significance are identified. For example, curvature extrema are identified: vertices of convex elliptical regions where Gaussian curvature K has a local maximum (e.g., K > 0.01 and is the largest in the neighborhood) are marked as "peaks"; vertices of concave elliptical regions where Gaussian curvature has a local maximum are marked as "valleys" (e.g., the depression at the bottom of a silo). The center points or points with the largest absolute curvature values of hyperbolic regions (saddle points) are identified; these points are often located in areas with drastic slope changes. The spatial distribution density of these feature points, their relative positional relationships, and associated curvature values are analyzed. The identified feature points, their types, spatial coordinates, and associated curvature values are recorded to form morphological feature point data.
[0097] Based on the principal direction vectors and principal curvature values in the curvature field data, feature lines are extracted from the surface. For example, ridges and valleys are extracted. A ridge is a curve on the surface where, along one principal direction, the other principal curvature reaches a local maximum; a valley is a curve on the surface where, along one principal direction, the other principal curvature reaches a local minimum. Methods based on principal direction vector field tracing or on zero-crossing point detection based on curvature derivatives can be used. For example, for each vertex, tracing along the two principal directions, if the corresponding normal curvature (i.e., the other principal curvature) in that direction is a local extremum, then that point belongs to a feature line. Connecting the traced points forms a series of polylines composed of 3D points; these polylines constitute the feature line network, including ridge and valley networks.
[0098] Based on a simplified material mechanics model, the relative stress level at each point on the accumulation surface is estimated. It is assumed that the stress is related to the material's gravitational load and local geometry. In convex regions (marked as convex ellipses in the morphological partitioning diagram), especially near ridges, higher normal compressive stress is experienced due to the accumulation of material above. The stress estimate can be positively correlated with the local mean curvature H (when H>0). In concave regions (marked as concave ellipses in the morphological partitioning diagram) or near valleys, the stress is relatively low, forming flow channels. In hyperbolic regions (saddle points) or areas with drastic slope changes, higher shear stress exists. A simplified stress estimation model calculates a scalar value S for each vertex: S = c1 × max(0,H) + c2 × |K| + c3 × z', where z' is the height of the vertex in the normalized coordinate system, c1, c2, and c3 are empirical coefficients, max(0,H) indicates that only the compressive stress component caused by the convexity is considered, and |K| reflects geometric instability. The calculated S value for each vertex is stored to form a stress distribution map.
[0099] By comprehensively utilizing the aforementioned data, macroscopic shape characteristics related to material flow and anomalies are identified. For example, potential "flow channels" are identified: connected areas composed of concave elliptical or parabolic regions in the morphological zoning diagram, with low stress values in the stress distribution diagram and coinciding with or near the valley network. Potential "high-risk bridging areas" are identified: areas located above the silo outlet, close to the wall, composed of convex elliptical regions in the morphological zoning diagram, with high stress values in the stress distribution diagram and intersecting with or near the ridge network. Potential "rat hole" formation areas are identified: typically areas with steep sidewalls (judged based on local normal vectors), relatively smooth surfaces (small curvature changes), and stress distribution diagrams showing signs of stress concentration or release. These identified areas are marked on the 3D model and assigned corresponding shape characteristic labels (e.g., "potential flow channels," "bridging risk areas"). This set of marked areas and labels constitutes the shape characteristic map.
[0100] All calculated geometric and stress information is integrated into a unified data structure. This data structure is based on a packing surface model, and each vertex or facet is appended with its corresponding curvature value (k1, k2, K, H), morphological category label, estimated stress value, and label of its corresponding shape characteristic region. Simultaneously, the extracted list of morphological feature points and the feature line network (ridges and valleys) are included as additional information. This multi-level, multi-attribute dataset constitutes the surface feature map, comprehensively describing the detailed geometry, local characteristics, and macroscopic structure related to flow stability of the material packing surface.
[0101] Preferably, the material morphology change and evolution analysis in step S2 specifically includes:
[0102] Time-series data alignment and sampling are performed on the surface feature map and the stationary angular distribution map to obtain a time-series feature data cube;
[0103] The static angle variation characteristics of the time-series feature data cube are analyzed to obtain the static angle evolution feature map.
[0104] The curvature change characteristics of the time-series feature data cube are analyzed to obtain a curvature evolution feature map.
[0105] The volume change characteristics of the time-series feature data cube are analyzed to obtain the density evolution diagram;
[0106] The intensity of parameter interaction influence is calculated for the static angle evolution feature map, curvature evolution feature map and density evolution map to obtain the parameter interaction influence map;
[0107] Perform material property-related time series analysis on the time series feature data cube to obtain the property correlation time series diagram;
[0108] A morphological change trend diagram is generated based on the parameter interaction influence diagram and the characteristic correlation time series diagram.
[0109] In this embodiment of the invention, surface feature maps and stationary angle distribution maps are acquired and generated at continuous time points. These data contain geometric features (such as curvature, morphological partitioning, and stress estimation) and stationary angle information (such as local stationary angles and critical region markings) of the material accumulation at different times. For time-series analysis, these data first need to be aligned in the time dimension. If the acquisition frequency is inconsistent, a resampling technique (e.g., linear interpolation) is used to unify all feature data to a fixed time interval (e.g., once per minute). At the same time, the feature data in the spatial dimension is standardized, for example, using fixed sampling points on the region partitioning or standardized model defined in step S2. For each time point and each spatial location (region or sampling point), its corresponding feature values are extracted, including the average stationary angle, average Gaussian curvature, average average curvature, maximum principal curvature, estimated stress value, and critical region marking status (yes / no) of that region / point. These multidimensional feature values are organized by time, spatial location, and feature type to form a three-dimensional data structure, such as a tensor, whose dimension can be [number of time points] × [number of spatial locations] × [number of feature types]. This tensor is the time-series feature data cube, which provides the foundation for subsequent dynamic analysis.
[0110] Extract the stationary angle time series for each spatial location (region or sampling point) from the time-series feature data cube. For each location's stationary angle time series, calculate its rate of change (e.g., using the finite difference method to calculate the change in stationary angle between adjacent time points divided by the time interval, i.e., Δθ / Δt). Calculate the moving average and standard deviation of the stationary angle time series to reflect its smoothing trend and volatility. Identify moments of significant changes in stationary angle, for example, using change point detection algorithms based on statistical tests. Organize these change characteristics (such as average rate of change, maximum rate of change, volatility, and change point location) spatially to form a spatial distribution map, such as a two-dimensional grid map, where each cell represents a region, and the color intensity or arrow direction indicates the trend and rate of change of the stationary angle in that region. This spatial distribution map is the stationary angle evolution feature map.
[0111] Extract time series of curvature features (e.g., mean Gaussian curvature K, mean average curvature H) for each spatial location (region or sampling point) from the time series feature data cube. For the curvature time series at each location, calculate its rate of change (e.g., ΔK / Δt, ΔH / Δt). Analyze the correlation between curvature changes and morphological types (planar, convex, concave, hyperbolic); for example, an increase in the rate of H in a convex region predicts further bulging of material accumulation. Identify regions and time points where curvature abruptly occurs, which are related to the formation of local collapses or bridging. Organize the curvature change characteristics (e.g., mean rate of change, maximum rate of change, abrupt change location, and amplitude) by spatial location to form a curvature evolution feature map, for example, a spatial map using different colors or symbols to represent the curvature change trends and amplitudes in different regions, especially those regions with drastic curvature changes.
[0112] Extract the height time series (e.g., the average Z-coordinate of the region) for each spatial location (region or sampling point) from the time-series feature data cube. Calculate the rate of change of the average height for each region (e.g., ΔZ / Δt). A decrease in height generally indicates that the material has settled or compacted in that region. Relatively stable or slightly increasing height (in the absence of new material addition) suggests the formation of cavities or arches below. By analyzing the spatial distribution and temporal evolution of the height change rate, changes in material density can be indirectly inferred. For example, a sustained rate of height decrease indicates a continuous compaction process. Organize these height change characteristics (such as the average height change rate, maximum height decrease rate) by spatial location to form a density evolution map, for example, a spatial map where colors represent the average height change rate of regions, with red indicating rapid decrease (compaction) and blue indicating stable or slow change.
[0113] The relationships between the previously obtained evolution diagrams of rest angle, curvature, and density are analyzed. For example, the cross-correlation function between the time series of the rate of change of rest angle in different regions and the time series of the rate of change of curvature in adjacent regions is calculated. High cross-correlation with zero time lag indicates simultaneous changes, while cross-correlation with non-zero lag indicates causal or propagation effects (e.g., a change in the rest angle in the upper region leads to a change in the curvature in the lower region). Methods such as the Granger causality test are used to quantify the causal relationship strength between different characteristic changes and between different spatial regions. These interactive relationships (e.g., the influence strength and lag time of the change in rest angle in region A on the change in curvature in region B) are represented in a graphical structure or matrix to form a parametric interaction diagram, revealing the dynamic correlation between local characteristic changes in the material packing morphology.
[0114] Within the time-series feature data cube, specific time-series patterns are identified that are associated with known material behaviors or anomalous precursors (e.g., a sudden, rapid decrease in the rest angle after a sustained, slow increase indicates surface collapse; a local curvature H value turning from positive to negative, accompanied by a velocity field, indicates mouse hole formation). Pattern matching algorithms (e.g., Dynamic Time Warping (DTW)) or machine learning models (e.g., sequence classifiers based on Recurrent Neural Networks (RNNs)) are used to compare the patterns against a pre-established library of anomalous precursor patterns. Segments in the current time-series data similar to these patterns are identified, and their temporal and spatial locations, as well as the matching pattern type, are labeled. The identified pattern instances and their associated material behaviors (e.g., "collapse," "mouse hole formation") are recorded to form a feature-related time-series graph, which is either a list of events or a spatially labeled graph with a time axis.
[0115] By combining the dynamic relationships between features revealed by the integrated parameter interaction influence diagram and the key pattern instances identified by the characteristic correlation time series diagram, a final morphological change trend diagram is generated. The morphological change trend diagram is a dynamic visualization or data report. It displays the morphology of the material accumulation at the current time point (from the latest accumulation surface model), and overlays it with: the current rate of change of the stationary angle, curvature, and density of each region (from the evolution feature diagram), using arrows or colors to indicate trends; the critical state of the stationary angle in high-risk areas (from the critical region markers in the stationary angle distribution diagram); the location and type of specific abnormal precursor patterns detected (from the characteristic correlation time series diagram), for example, represented by icons; and the direction and intensity of the recent morphological change propagation predicted based on parameter interaction influence analysis. The morphological change trend diagram intuitively and comprehensively demonstrates the current dynamics, evolution direction, and potential risk development paths of the material accumulation state.
[0116] Preferably, step S3 specifically includes:
[0117] Step S31: Perform feature data fusion on the flow eigenvector map and the particle stability index matrix to obtain a state fusion dataset;
[0118] Step S32: Obtain historical operation records; extract historical anomaly patterns from the state fusion dataset based on the historical operation records to obtain anomaly precursor patterns;
[0119] Step S33: Calculate and match the similarity between the anomaly precursor pattern and the state fusion dataset to obtain the anomaly similarity matrix;
[0120] Step S34: Calculate the risk probability based on the anomaly similarity matrix and the state fusion dataset to obtain the risk probability distribution map;
[0121] Step S35: Based on the risk probability distribution map and the anomaly similarity matrix, predict the anomaly time window to obtain the anomaly early warning schedule;
[0122] Step S36: Construct and visualize health indicators based on the anomaly warning timetable and risk probability distribution map to obtain a spatiotemporal map of mobile health.
[0123] In this embodiment of the invention, the flow characteristic vector map contains two-dimensional flow characteristics of each monitoring point (e.g., feed inlet, elbow, outlet, etc.) in the material conveying process over a continuous time series, such as local velocity distribution, texture statistics, and abnormal area area. The particle stability index matrix contains three-dimensional packing morphology characteristics of each spatial region of the material accumulation (e.g., grid areas divided inside the silo) over a continuous time series, such as local rest angle, curvature, estimated stress, and stability index. To integrate these data, the two-dimensional monitoring points or their covered areas in the flow characteristic vector map are first spatially correlated with the three-dimensional spatial regions in the particle stability index matrix. For example, the two-dimensional region within the field of view of the silo outlet camera is mapped to a specific three-dimensional packing region at the silo outlet. For locations such as pipe elbows, they can be correlated with the region near the downstream silo inlet. Then, these feature data collected or calculated at different spatial locations and different time points are integrated. A unified data structure is constructed, such as a four-dimensional tensor, whose dimensions are [number of time points] × [number of spatial regions] × [feature type] × [feature dimension]. The "Number of Spatial Regions" corresponds to the fixed areas divided into material conveying and stacking spaces (e.g., feed port area, bend area 1, silo area A, etc.). "Feature Type" distinguishes between flow characteristics and stacking morphology characteristics. "Feature Dimension" includes specific numerical values (e.g., mean flow velocity, angle of repose, Gaussian curvature). Feature data from different sources and with different dimensions are standardized, for example, using Z-score standardization to ensure that the mean of each feature is 0 and the standard deviation is 1, eliminating the influence of dimensions. This unified data structure, after spatiotemporal alignment, fusion, and standardization, is the state fusion dataset.
[0124] Historical operation records from the automated equipment's operational database are retrieved. These records include changes in equipment operating parameters, operator logs, maintenance records, and clearly marked abnormal events with their occurrence time, type (e.g., silo outlet blockage, pipeline bridging), and location. Using this historical abnormal event information, historical data segments associated with the abnormal events are extracted from the state fusion dataset generated in step S31. For each historical abnormal event (e.g., a silo outlet blockage at a specific time on a certain day), the state fusion dataset is traced back to a fixed time window (e.g., the previous 30 minutes) preceding the event. Feature time series are extracted from the state fusion dataset corresponding to the location and related areas of the abnormality within this time window. For example, time series of features such as average flow velocity, flow velocity variance, agglomeration area ratio, local stationary angle, rate of curvature change, and stability index of the silo outlet area within 30 minutes before the blockage. These data segments with clear abnormal labels (e.g., "precursor to outlet blockage") and time series features are aggregated to form an abnormal precursor pattern library. This pattern library stores the typical feature evolution trajectories before the occurrence of various types of abnormal events.
[0125] For each spatial region (e.g., silo region A) and each known anomaly type (e.g., blockage, bridging) in the state fusion dataset, the feature time series within a fixed-length time window (e.g., the most recent 10 minutes) preceding the current moment of that region is extracted as the "current state trajectory". The similarity between the current state trajectory of each region and all historical precursor patterns of the corresponding anomaly type in the anomaly precursor pattern library is calculated. The Dynamic Time Warping (DTW) algorithm is used to calculate the distance between the two time series. The DTW distance measures the similarity between two similar sequences that have offsets or stretches on the time axis. The smaller the DTW distance, the more similar the current state trajectory is to the historical precursor pattern. For each spatial region and each anomaly type, its DTW distance to the most similar historical precursor pattern in the pattern library is recorded. These distance values are organized into a matrix, where rows represent spatial regions, columns represent anomaly types, and cell values are the corresponding DTW distances. This matrix is the anomaly similarity matrix, which quantifies the similarity between the current material state and different anomaly precursor patterns in different regions.
[0126] Based on this data, the probability of material blockages, bridging, and other anomalies occurring in different regions is calculated. A machine learning-based risk probability assessment model is established, such as a trained neural network classifier or a support vector machine model. The model's input includes the anomaly similarity matrix values for each region at the current moment and key feature values from the corresponding state fusion dataset (e.g., current average flow rate, current stability index). The model's training data comes from historical data, using past anomaly similarity matrices and state fusion datasets as input, and the output label being whether an anomaly will occur in the region within a certain period (e.g., within the next 10 minutes). After training, the current anomaly similarity matrix and state fusion dataset are input into the model. For each spatial region and each anomaly type, the model outputs a probability value between 0 and 1, representing the probability of that anomaly occurring within a future period. These probability values are organized spatially to form a spatial distribution map, with each spatial region having one or more probability values (corresponding to different anomaly types). This map is the risk probability distribution map.
[0127] For areas marked as high-risk (e.g., risk probability greater than 0.7) on the risk probability distribution map, it is necessary to predict the time window for anomaly occurrence. Analyze the trend (i.e., rate of change) of the risk probability or anomaly similarity values in these high-risk areas over a recent period. If the risk probability is rising rapidly, or the anomaly similarity is falling rapidly (approaching historical precursor patterns), the predicted anomaly occurrence time window is relatively close. Refer to the time length information from the appearance of historical patterns to the occurrence of anomalies in the anomaly precursor pattern library in step S32. Combine the current risk probability value, its rate of change, and the evolution speed of the matched historical patterns to estimate the time required for the risk probability to reach the critical threshold (e.g., 0.9). For example, if the current probability is 0.8, the rate of change is 0.05 / minute, and the critical threshold is 0.95, it is estimated that it will take (0.95-0.8) / (0.05 / minute) = 3 minutes to reach the critical state. Associate this estimated time information (e.g., the expected start time and duration of the anomaly) with the corresponding spatial region and anomaly type. Create a list or table to record the expected anomaly type, the expected time window of occurrence (e.g., "within the next 5-10 minutes"), and the confidence level or scope of impact for each high-risk area. This list constitutes the anomaly warning timetable.
[0128] Based on these two data points, a health index for material flow status is constructed. A health score is defined for each spatial region, for example, a value between 0 and 100, where 100 represents complete health and 0 represents a serious anomaly. The health calculation formula comprehensively considers risk probability and warning time. For example, for low-risk regions (risk probability less than 0.5), the health score is set to 100. For high-risk regions, the health score can be calculated as: Health = 100 × (1 - RiskProbability) × f(TimeToAnomaly), where f(TimeToAnomaly) is a function that decreases as the warning time shortens, for example, f(t) = min(1, t / T0), where T0 is a time constant (e.g., 15 minutes), indicating that if the warning time is less than T0, the health score will significantly decrease. The calculated health scores are assigned to the corresponding spatial regions. These spatial region data with health scores are organized by time (including the current state and the short-term predicted future state based on risk prediction) to form the data structure of the flow health spatiotemporal map. The flow health spatiotemporal map is displayed through a visualization interface. On the 3D model or 2D top view of the material conveying and stacking, color bars map health scores: green represents high health (80-100), yellow represents medium health (50-80), orange represents low health (20-50), and red represents critical / abnormal (0-20). Abnormality type icons and estimated warning times are overlaid in risk areas. A timeline sliding function allows users to view predicted short-term health trends, providing an intuitive understanding of risk development. This visualization interface and the data it carries constitute the flow health spatiotemporal map.
[0129] Preferably, the risk assessment and adjustment of control parameters in step S4 specifically includes:
[0130] Risk areas are located and classified in the spatiotemporal map of mobile health to obtain a risk intervention map;
[0131] Obtain equipment control parameters; perform control parameter impact analysis on equipment control parameters based on the risk intervention map to obtain the parameter impact matrix;
[0132] The sensitivity of risk area parameters in the risk intervention map is calculated to obtain a parameter sensitivity ranking table.
[0133] Obtain the physical property data of the material; define the range of vibration parameters based on the physical property data and the parameter sensitivity ranking table to obtain the vibration parameters;
[0134] Airflow parameters are obtained by adjusting airflow parameters based on physical property data and parameter sensitivity ranking tables.
[0135] The mechanical conveying parameters are adjusted based on physical property data and parameter sensitivity ranking table to obtain the mechanical conveying parameters.
[0136] Based on physical property data and parameter sensitivity ranking table, silo pressure control settings are obtained to obtain silo control parameters;
[0137] Multi-parameter collaborative optimization was performed on vibration parameters, airflow parameters, mechanical conveying parameters, and silo control parameters to obtain a collaborative parameter combination scheme;
[0138] A parameter adjustment timing plan is performed on the collaborative parameter combination scheme to obtain a set of parameter adjustment schemes.
[0139] In this embodiment of the invention, the spatiotemporal health map includes the health score and predicted anomaly warning time for each area of the material conveying and stacking space in the current and short-term future. A health threshold is set: a health score below 80 is marked as a concern area, below 50 as a risk area, and below 20 as a critical area. Risk areas are classified according to their health score and anomaly warning time. For example, a health score below 20 or a warning time of less than 5 minutes is marked as an emergency risk area; a health score below 50 and a warning time of less than 15 minutes is marked as a high-risk area; a health score below 80 and a warning time of less than 30 minutes is marked as a medium-risk area; and a health score below 100 but showing a downward trend or a warning time of more than 30 minutes is marked as a low-risk area. The main anomaly types associated with each risk area are identified (e.g., blockage, bridging, and stratification warning information from S36). Intervention priorities are determined based on risk level and anomaly type, with emergency risk areas having the highest priority. This information (area identifier, anomaly type, risk level, and intervention priority) is marked on the material conveying and stacking space model to form a risk intervention map. A risk intervention map is a data structure that contains a list of marked areas and their attributes.
[0140] Obtain a list of adjustable control parameters for the current automated equipment, such as vibrator frequency and amplitude, screw conveyor speed and start / stop status, pneumatic conveying system pressure and flow rate, and silo wall vibration or air cushion pressure. Establish a parameter impact knowledge base based on historical operational data (including control parameter adjustment records and corresponding material state change feedback, i.e., accumulated results from S47) and material handling process knowledge. This knowledge base stores the degree and direction of the impact of different control parameter adjustments on different types of anomalies (blockage, bridging, stratification, etc.) at different spatial locations (feed inlets, pipelines, silo outlets, etc.). For example, increasing the silo wall vibration amplitude has a strong positive impact on bridging at the silo outlet (reducing risk), but a smaller impact on blockage at pipeline bends; reducing the screw conveyor speed reduces the risk of material accumulation at the feed inlet, but increases the risk of blockage in downstream pipelines. Extract parameter impact data corresponding to the anomaly types and regions involved in the current risk intervention map from the knowledge base to construct a parameter impact matrix. The rows of this matrix represent control parameters, the columns represent risk areas or anomaly types, and the cell values represent the weight and direction of the corresponding parameter adjustment on the health of that risk area / anomaly type (for example, the weight value is between -1 and 1, a positive value indicates a positive effect, and a negative value indicates a negative effect).
[0141] Based on each risk area and its anomaly type identified in the risk intervention map, and combined with the parameter influence matrix, the sensitivity of that area to each control parameter is calculated. Sensitivity can be defined as the absolute value of the influence weight of the corresponding control parameter on that risk area / anomaly type in the parameter influence matrix. The higher the sensitivity, the greater the impact of adjusting the parameter on the improvement or deterioration of the risk in that area. For each risk area, the control parameters are sorted in descending order according to their parameter sensitivity. A parameter sensitivity ranking table is formed, which lists the most effective or most important control parameters and their sensitivity values for each risk area.
[0142] Obtain physical property data of the currently processed material, such as particle size distribution, humidity, density, internal friction angle, and cohesion. This data is partly derived from the analysis results in S13 and S26 or from external input. Refer to the material's physical property data and parameter sensitivity ranking table. For vibrator parameters that are highly sensitive to specific risk areas (e.g., bridging risk area at the silo outlet) in the parameter sensitivity ranking table, consult or calculate the minimum vibration acceleration or frequency range required to break bridging in the current stacking state of the material, based on its internal friction angle and cohesion. For example, for powders with high cohesion, higher frequency or larger amplitude vibration is required. Set an initial recommended range for vibration parameters, such as frequency F∈[50Hz, 100Hz], amplitude A∈[0.5mm, 2.0mm], duration T∈[2s, 10s], and interval I∈[30s, 120s]. Use these vibration parameter ranges and initial recommended values, defined based on material properties and sensitivity, as the vibration parameter data.
[0143] Refer to the material's physical properties data (e.g., permeability, minimum fluidization velocity) and a parameter sensitivity ranking table. For airflow parameters (e.g., pneumatic conveying pressure, silo air cushion pressure, or flow rate) that are highly sensitive to specific risk areas (e.g., pneumatic conveying pipeline blockage risk areas or silo bottom compaction areas) in the parameter sensitivity ranking table, set initial recommended values and adjustment ranges for the airflow parameters based on material characteristics and risk level. For example, for easily caking materials, silo air cushions require higher pressure or more frequent pulse blowing. In pneumatic conveying blockage risk areas, it is necessary to increase the conveying pressure or initiate a backflushing procedure. Use these airflow parameter ranges and initial recommended values set based on material characteristics and sensitivity as the airflow parameter data.
[0144] Referencing the physical properties of the material (e.g., bulk density, coefficient of friction) and a parameter sensitivity ranking table, for mechanical conveying parameters (e.g., screw conveyor speed) that are highly sensitive to specific risk areas (e.g., material accumulation risk area at the screw conveyor inlet) in the parameter sensitivity ranking table, initial recommended values and adjustment ranges for the mechanical conveying parameters are set based on material characteristics and risk level. For example, if the screw conveyor speed is too low when the risk of material accumulation at the feed inlet is high, the speed needs to be appropriately increased to increase conveying capacity, while simultaneously considering downstream processing capacity. These mechanical conveying parameter ranges and initial recommended values set based on material characteristics and sensitivity are used as the mechanical conveying parameter data.
[0145] Referencing the physical property data of the reference material (e.g., bulk density, internal friction angle) and a parameter sensitivity ranking table, for silo pressure control parameters (e.g., silo wall pressure pulses, internal air pressure / vacuum) that are highly sensitive to specific risk areas (e.g., bridging risk areas on the silo inner wall) in the parameter sensitivity ranking table, initial recommended values and adjustment ranges for the silo control parameters are set according to material characteristics and risk level. For example, for easily adhesive materials, periodic pressure pulses are required on the silo wall to prevent material adhesion and bridging. These silo control parameter ranges and initial recommended values set based on material characteristics and sensitivity are used as the silo control parameter data.
[0146] Based on all risk areas and their risk levels identified in the risk intervention map, and the parameter influence matrix, a multi-objective optimization problem is constructed. The objective function aims to maximize overall health (e.g., minimize the weighted sum of risk probabilities across all areas in the risk probability distribution map, with weights determined by risk levels), while considering constraints such as physical limitations of equipment parameters (maximum speed, maximum pressure, etc.), energy consumption limitations, and cooperative or conflicting relationships between parameters (e.g., some parameters cannot be adjusted simultaneously or in opposite directions). Optimization algorithms, such as gradient-based optimization methods or heuristic algorithms (e.g., simulated annealing), are used to search for the optimal parameter combination within the allowable range of each parameter. The optimization process iteratively adjusts the values of each parameter and predicts the impact of the adjustment on the health of each risk area based on the parameter influence matrix, selecting the parameter combination that significantly reduces overall risk. Multiple cooperative parameter combination schemes with different trade-offs (e.g., aggressive intervention to quickly eliminate risk vs. smooth adjustment to reduce system disturbances) are generated. This list of schemes constitutes the cooperative parameter combination scheme.
[0147] Choose one or more schemes from the collaborative parameter combination schemes as the basis. Based on the intervention priority and anomaly warning schedule for each risk area in the risk intervention map, plan the execution sequence of parameter adjustments. For urgent risks, parameter adjustments need to be executed immediately with a more aggressive adjustment rate. For low to medium risks, a more gradual adjustment rate can be used, or a suitable opportunity can be waited for (e.g., when equipment load is low). If multiple parameter adjustments target risks in different locations and they interact with each other, the adjustment sequence needs to be planned or performed synchronously to avoid creating new problems or reducing effectiveness. For example, first increase the vibration frequency at the silo outlet to break bridging, then adjust the downstream screw conveyor speed to match the flow rate. Define the start time, target value, and rate of change (e.g., how many units per second) for each parameter adjustment. Arrange these detailed adjustment steps in chronological order to form a complete parameter adjustment plan. This set of plans (containing multiple alternative plans) constitutes the parameter adjustment scheme set, which is an ordered list of operational instructions.
[0148] Preferably, the effect evaluation and feedback in step S4 specifically includes:
[0149] The parameter adjustment scheme set is evaluated and an execution decision is made to obtain the execution decision instruction;
[0150] Based on the execution decision instructions, parameter adjustments are performed and feedback is generated to obtain the parameter execution trajectory;
[0151] The effect of parameter execution trajectory and flow health spatiotemporal diagram is evaluated and response curves are constructed to obtain the control parameter response curves.
[0152] In this embodiment of the invention, the parameter adjustment scheme set includes multiple control parameter adjustment plans generated for the current material flow risk status. Each plan details the parameters to be adjusted, the target value, the adjustment range, and the execution sequence. Simultaneously, historical parameter response data is acquired, recording the actual response of the material flow status when similar parameter adjustments were performed in the past. Each scheme in the scheme set is evaluated from multiple dimensions. Evaluation dimensions include: the predicted risk improvement effect (estimated health improvement based on the parameter influence matrix and historical response data, the proportion of risk area reduction, and the extension of warning time), the expected energy consumption change, the impact on equipment operating load, and the stability of the adjustment process (whether the parameter change rate causes drastic fluctuations). The predicted effect is calibrated using historical parameter response data. For example, if historical data shows that similar parameter adjustments are typically 10% less effective in actual applications than the model prediction, the predicted effect of the current scheme is corrected accordingly. Based on preset decision rules, combined with the risk level (risk intervention map in S41) and the evaluation results, the final execution decision instruction is determined. For example, for urgent risks, the system automatically selects the highest-scoring option and sets it to automatic execution mode; for moderate risks, it selects the highest-scoring option and sets it to recommended mode, awaiting operator confirmation or modification; for low risks, no adjustments are made or only a minor adjustment option is recommended. The execution decision instruction is structured data that explicitly specifies the option to be executed (referencing the option ID or directly including option details), the execution mode (automatic / recommended), the start time, and the areas or indicators that need to be monitored during execution (e.g., silo outlet flow rate, agglomeration area in bend areas).
[0153] The system receives execution decision instructions. If the instruction is in recommended mode, the solution is displayed to the operator through the user interface, awaiting confirmation or modification before execution is triggered by the operator. If the instruction is in automatic execution mode, the system interacts directly with the automation equipment control system (e.g., PLC or DCS) via an industrial communication interface. Following the parameter adjustment timing plan specified in the execution decision instruction, control commands are sent to the corresponding actuators (e.g., vibrator controllers, frequency converters, valve controllers) to adjust parameter values. A specified adjustment rate is used during execution to ensure smooth parameter changes. For example, if the instruction requires adjusting the vibration frequency from 50Hz to 80Hz at a rate of 5Hz / second, the system sends instructions every second to increase the frequency by 5Hz until it reaches 80Hz. During this process, the system continuously reads the actual values of the current parameters from the equipment control system in real time and records the parameter change sequence with timestamps. Simultaneously, it continuously receives data updates from steps S1 (flow characteristic acquisition) and S2 (stacking morphology modeling), and updates the flow health spatiotemporal map in real time through step S3 (flow risk prediction). The actual values of these real-time monitored parameters and the changes in key indicators of the corresponding flow health spatiotemporal diagram (e.g., target area health, flow velocity, agglomeration area) over time are recorded. These records constitute the parameter execution trajectory, which details the actual process of parameter adjustment and its resulting real-time feedback on the material state.
[0154] Acquire parameter execution trajectories and flow health spatiotemporal diagrams before, during, and after parameter adjustment. Quantitatively evaluate the effectiveness of parameter adjustment: calculate the change in health indicators of the target risk area over a period after adjustment begins (e.g., percentage improvement in health, decrease in risk probability). Identify the time required for health indicators to reach a stable state or achieve maximum improvement, i.e., response time or time lag. Analyze whether health indicators remain at the improved level after adjustment, and the duration of this maintenance. Correlate each parameter value in the parameter execution trajectory with key health indicators or flow characteristic values (e.g., average flow velocity in the target area, agglomeration area ratio) in the flow health spatiotemporal diagram at the same time point or considering time lag. For example, plot the curve of vibration frequency versus time and the curve of health status of the silo outlet area versus time on the same coordinate system to visually demonstrate the relationship between parameter changes and effects. Furthermore, perform regression analysis or system identification-based methods on these correlated data to establish a quantitative model or response function between parameter changes and material flow state response. For example, a simple linear model is represented as ΔHealth = α × ΔParameter, where α is the response coefficient. More complex models may include time delays and nonlinear terms. These quantitative relationships, response times, durations of impact, and actual evaluation results of parameter adjustments (e.g., the actual health improved by X%, with a difference of Z% from the predicted value Y%) are stored in a structured database. This stored dataset constitutes the control parameter response curve data. It is not merely a graph, but rather the underlying data describing the parameter-response relationship, enriching the historical parameter response data knowledge base and guiding more precise parameter adjustments and optimized control strategies in the future.
[0155] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.
[0156] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.
Claims
1. A digital visualization control method for automated equipment, characterized in that, Includes the following steps: Step S1: Acquire and preprocess image sequences of the automated material conveying equipment to obtain an enhanced flow image set; analyze the flow texture and velocity distribution of the enhanced flow image set to obtain a texture feature data set and a velocity field distribution map; extract flow characteristics from the texture feature data set and the velocity field distribution map to obtain a flow feature vector map. Step S2: Construct a 3D point cloud of the material to obtain an accumulated surface model; measure and partition the stationary angle of the accumulated surface model to obtain a stationary angle distribution map; calculate the curvature and morphological features of the accumulated surface model to obtain a surface feature map; analyze the evolution of material morphology using the surface feature map and the stationary angle distribution map to obtain a morphological change trend map. Stability assessment and index calculation were performed based on the morphological change trend diagram to obtain the particle stability index matrix; Step S3: Perform intelligent prediction of flow risk on the flow eigenvector map and particle stability index matrix to obtain the spatiotemporal map of flow health; Step S4: Perform risk assessment and adjustment of control parameters on the spatiotemporal diagram of the flow health to obtain a set of parameter adjustment schemes; execute the set of parameter adjustment schemes and evaluate and provide feedback on the effects to obtain the response curves of the control parameters.
2. The digital visualization control method for automated equipment according to claim 1, characterized in that, Step S1 is as follows: Step S11: Install high-speed cameras at key locations of the automated material conveying equipment to synchronously acquire high-definition image sequences of material flow at a frequency of 60-120 frames per second, forming an original flow image set; Step S12: Perform image enhancement and noise reduction on the original flow image set to obtain an enhanced flow image set; Step S13: Extract flow texture features from the enhanced flow image set to obtain a texture feature data set; Step S14: Calculate the velocity distribution of the enhanced flow image set based on the texture feature data set to obtain a velocity field distribution map; Step S15: Detect clusters and abnormal regions in the velocity field distribution map and texture feature data set to obtain an abnormal region marker map; Step S16: Perform multi-dimensional feature fusion and map generation on the texture feature data group, velocity field distribution map and abnormal region marker map to obtain the flow state feature vector map.
3. The digital visualization control method for automated equipment according to claim 1, characterized in that, The 3D point cloud construction in step S2 is specifically as follows: A specific structured light pattern is projected onto the material surface by a projector, and the camera captures images of the material surface with the structured light pattern from different angles. Then, the spatial coordinate data of each point on the material surface is calculated using the principle of triangulation. Construct the original accumulation point cloud of the material accumulation surface based on spatial coordinate data; Point cloud optimization and surface reconstruction are performed on the original stacked point cloud to obtain a stacked surface model.
4. The digital visualization control method for automated equipment according to claim 1, characterized in that, The specific steps of stationary angle measurement and zoning in step S2 are as follows: A coordinate system is established and the center is located for the stacked surface model to obtain a standardized stacked model. Extract the radial profile set of the standardized stacking model; Global stationary angles are calculated on the radial profile set to obtain global stationary angle data; The surface region of the standardized stacking model is divided to obtain a region division map; Based on the regional division map, the local rest angle of the standardized stacking model is calculated to obtain the local rest angle table; Based on the local static angle table, spatial interpolation of the static angles is performed on the region division map to obtain a continuous distribution of static angles; Critical region identification is performed based on the continuous distribution of stationary angles, resulting in critical region markers. A static angle distribution map is generated based on the continuous distribution of static angles, critical region markings, and global static angle data.
5. The digital visualization control method for automated equipment according to claim 1, characterized in that, The calculation of curvature and morphological features in step S2 is as follows: Differential geometric parameters are calculated on the stacked surface model to obtain curvature field data; Surface morphology is classified based on curvature field data to obtain a morphology partition map; Feature points are extracted and analyzed from the morphological partition map to obtain morphological feature points; Feature lines are extracted from the curvature field data to obtain a feature line network; Stress distribution is estimated based on curvature field data, morphological partitioning diagram, and characteristic line network to obtain a stress distribution map. Shape characteristic analysis is performed on curvature field data, morphological partitioning diagram, and stress distribution diagram to obtain shape characteristic diagram; Generate surface feature maps based on shape property maps and stress distribution maps.
6. The digital visualization control method for automated equipment according to claim 1, characterized in that, The material morphology change and evolution analysis in step S2 is as follows: Time-series data alignment and sampling are performed on the surface feature map and the stationary angular distribution map to obtain a time-series feature data cube; The static angle variation characteristics of the time-series feature data cube are analyzed to obtain the static angle evolution feature map. The curvature change characteristics of the time-series feature data cube are analyzed to obtain a curvature evolution feature map. The volume change characteristics of the time-series feature data cube are analyzed to obtain the density evolution diagram; The intensity of parameter interaction influence is calculated for the static angle evolution feature map, curvature evolution feature map and density evolution map to obtain the parameter interaction influence map; Perform material property-related time series analysis on the time series feature data cube to obtain the property correlation time series diagram; A morphological change trend diagram is generated based on the parameter interaction influence diagram and the characteristic correlation time series diagram.
7. The digital visualization control method for automated equipment according to claim 1, characterized in that, Step S3 is as follows: Step S31: Perform feature data fusion on the flow eigenvector map and the particle stability index matrix to obtain a state fusion dataset; Step S32: Obtain historical operation records; extract historical anomaly patterns from the state fusion dataset based on the historical operation records to obtain anomaly precursor patterns; Step S33: Calculate and match the similarity between the anomaly precursor pattern and the state fusion dataset to obtain the anomaly similarity matrix; Step S34: Calculate the risk probability based on the anomaly similarity matrix and the state fusion dataset to obtain the risk probability distribution map; Step S35: Based on the risk probability distribution map and the anomaly similarity matrix, predict the anomaly time window to obtain the anomaly early warning schedule; Step S36: Construct and visualize health indicators based on the anomaly warning timetable and risk probability distribution map to obtain a spatiotemporal map of mobile health.
8. The digital visualization control method for automated equipment according to claim 1, characterized in that, The risk assessment and adjustment of control parameters in step S4 are as follows: Risk areas are located and classified in the spatiotemporal map of mobile health to obtain a risk intervention map; Obtain equipment control parameters; perform control parameter impact analysis on equipment control parameters based on the risk intervention map to obtain the parameter impact matrix; The sensitivity of risk area parameters in the risk intervention map is calculated to obtain a parameter sensitivity ranking table. Obtain the physical property data of the material; define the range of vibration parameters based on the physical property data and the parameter sensitivity ranking table to obtain the vibration parameters; Airflow parameters are obtained by adjusting airflow parameters based on physical property data and parameter sensitivity ranking tables. The mechanical conveying parameters are adjusted based on physical property data and parameter sensitivity ranking table to obtain the mechanical conveying parameters. Based on physical property data and parameter sensitivity ranking table, silo pressure control settings are obtained to obtain silo control parameters; Multi-parameter collaborative optimization was performed on vibration parameters, airflow parameters, mechanical conveying parameters, and silo control parameters to obtain a collaborative parameter combination scheme; A parameter adjustment timing plan is performed on the collaborative parameter combination scheme to obtain a set of parameter adjustment schemes.
9. The digital visualization control method for automated equipment according to claim 1, characterized in that, The specific steps for effect evaluation and feedback in step S4 are as follows: The parameter adjustment scheme set is evaluated and an execution decision is made to obtain the execution decision instruction; Based on the execution decision instructions, parameter adjustments are performed and feedback is generated to obtain the parameter execution trajectory; The effect of parameter execution trajectory and flow health spatiotemporal diagram is evaluated and response curves are constructed to obtain the control parameter response curves.
10. A digital visualization control system for automated equipment, characterized in that, For performing the digital visualization control method for automated equipment as described in claim 1, the digital visualization control system for automated equipment includes: The flow perception module is used to acquire and preprocess image sequences of automated material conveying equipment to obtain an enhanced flow image set; extract flow texture features from the enhanced flow image set to obtain a texture feature data set; calculate the velocity distribution of the enhanced flow image set to obtain a velocity field distribution map; and extract flow features from the velocity field distribution map and the texture feature data set to obtain a flow feature vector map. The packing analysis module is used to construct a 3D point cloud of the material to obtain a packing surface model; to measure and partition the packing surface model to obtain a static angle distribution map; to calculate the curvature and morphological characteristics of the packing surface model to obtain a surface feature map; to analyze the evolution of material morphology using the surface feature map and the static angle distribution map to obtain a morphological change trend map; and to perform stability assessment and index calculation based on the morphological change trend map to obtain a particle stability index matrix. The risk prediction module is used to intelligently predict flow risks from the flow pattern feature vector map and particle stability index matrix, and obtain a spatiotemporal map of flow health. The parameter control module is used to perform risk assessment and adjustment of control parameters on the spatiotemporal diagram of flow health, and obtain a set of parameter adjustment schemes; execute the set of parameter adjustment schemes and perform effect evaluation and feedback to obtain the control parameter response curves.
Citation Information
Patent Citations
Coal pile stability analysis and prediction method
CN114707721A
Gate-type bucket wheel machine automatic stockpiling method and system based on laser point cloud data
CN117533811A
Automatic powder loading device based on visual identification
CN119635649A
Material loading control method
CN120097034A
Powder flow characteristic estimating method, and powder flow characteristic testing device
JP2005241425A
Cited By
Anti-blocking and anti-collapse cooperative control method for safety protection system of intelligent bulk material bin
CN121167430A
Coal granularity analysis method and system in intelligent coal preparation process based on machine vision
CN121169921A
Clothing production workshop flow dynamic control method and system
CN121232765A
Method, system and equipment for detecting deformation of oil pipe of main oil pump of steam turbine and medium
CN121252677A
Feed production ingredient automatic control system and control method thereof
CN122499702A
