A screening machine posture monitoring method based on multi-modal fusion

Abstract

The application discloses a kind of screening machine posture monitoring methods based on multimodal fusion, it is related to screening equipment intelligent monitoring technical field, including the following steps: S1, acquisition image data, support state data and operation monitoring data, form multimodal operation data;S2, generate standardization posture monitoring dataset;S3, generate posture angle and coal flow eccentric load information by improving RT-DETR network;S4, generate screening machine main posture angle by effectiveness judgment and double-machine consistency check;S5, generate posture fusion feature vector by iTransformer model;S6, generate posture anomaly identification result in combination with posture trend prediction and physical consistency residual error judgment;S7, output screening machine posture monitoring result.The application uses improved RT-DETR network, iTransformer model and physical consistency residual error judgment, realizes screening machine posture real-time monitoring.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring technology for screening equipment, and in particular to a method for monitoring the attitude of a screening machine based on multimodal fusion. Background Technology

[0002] With the increasing demand for intelligent transformation of coal preparation plants and continuous and stable operation of screening equipment, multi-source monitoring technology for screening machine posture changes, coal flow eccentricity, and abnormal support status has received widespread attention. Existing screening machine status monitoring methods mainly rely on manual inspection, single-camera visual detection, vibration threshold alarms, or air spring support parameter monitoring for anomaly detection. However, these methods generally suffer from the following problems in practical applications: Traditional steel springs or ordinary rubber damping structures have limited vibration isolation capabilities and are prone to fatigue failure, increased noise, and unstable support after long-term operation. They are unable to adapt to the high-frequency vibration and off-center impact of the screening machine under full-load conditions. The feed coal flow is greatly affected by changes in feed rate, moisture content, and particle size, which can easily lead to local accumulation and unilateral off-center loading. This results in screen tilting, abnormal exciter stress, accelerated screen plate wear, and reduced screening efficiency. Manual inspection can only be carried out after the abnormality has become obvious, resulting in delayed early warning. Single-camera visual detection is prone to target loss or angle misjudgment under conditions of coal dust obscuring, vibration blurring, and partial obstruction, making it difficult to reliably obtain information on screen posture and coal flow distribution. Single vibration or temperature threshold alarms can only reflect local operational abnormalities and cannot distinguish the correlation between coal flow off-center loading, support abnormalities, and posture abnormalities. This leads to a lack of reliability verification and abnormality source judgment for alarm results, affecting the accuracy and traceability of screening machine posture monitoring.

[0003] Therefore, how to provide a screening machine attitude monitoring method based on multimodal fusion is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose a multimodal fusion-based method for monitoring the attitude of a screening machine. This invention employs an improved RT-DETR network, an iTransformer model, and a physical consistency residual judgment method. It acquires dual-view image data using a first intrinsically safe mining camera and a second intrinsically safe mining camera, and fuses the screening machine's support status data and operational monitoring data to achieve real-time monitoring of screen attitude changes and coal flow off-center loading. Furthermore, it reduces the impact of single-view occlusion and false detections through dual-view consistency constraints, improves the stability of feature extraction under complex coal dust environments by enhancing degraded samples caused by coal dust occlusion, and utilizes the iTransformer model to perform temporal fusion modeling of attitude angle, coal flow off-center loading, support status, and operational status. Combined with physical consistency residuals, it distinguishes between attitude anomalies, coal flow off-center loading anomalies, support anomalies, and operational monitoring anomalies. This method has the advantages of strong real-time monitoring, accurate anomaly identification, high alarm reliability, and applicability to continuous operation monitoring of screening machines in coal preparation plants.

[0005] A method for monitoring the attitude of a screening machine based on multimodal fusion according to an embodiment of the present invention includes the following steps: S1. Synchronously collect multimodal operation data during the operation of the screening machine. The multimodal operation data includes image data collected by the first intrinsically safe mining camera and the second intrinsically safe mining camera, as well as screening machine support status data and operation monitoring data. S2. Perform attitude benchmarking processing on the multimodal operation data to generate a standardized attitude monitoring dataset; S3. Input the image data from the standardized attitude monitoring dataset into the improved RT-DETR network, and extract visual attitude features and coal flow off-center loading features based on dual-view consistency constraints and coal dust occlusion degradation samples to generate first attitude angle, second attitude angle and coal flow off-center loading information. S4. Perform data validity judgment and dual-machine consistency verification on the first attitude angle, the second attitude angle and the coal flow off-center loading information, generate attitude data credibility, and generate the main attitude angle of the screening machine based on the attitude data credibility. S5. Input the main attitude angle of the screening machine, the coal flow off-center load information, the support status data of the screening machine and the operation monitoring data into the iTransformer model, perform multivariate time series modeling, and generate the screening machine attitude fusion feature vector. S6. Based on the posture fusion feature vector of the screening machine, perform posture trend prediction and introduce physical consistency residual judgment to generate posture anomaly identification results. S7. Generate the screening machine posture monitoring results based on the posture anomaly identification results, and send the screening machine posture monitoring results to the PLC or centralized control platform. At the same time, store the screening machine posture monitoring results and the corresponding historical operation data.

[0006] Optionally, S1 specifically includes: Based on the visible area of ​​the screen body and the coal flow passage area of ​​the screening machine, a first intrinsically safe mining camera and a second intrinsically safe mining camera are respectively installed so that the first intrinsically safe mining camera and the second intrinsically safe mining camera can collect image data of the screening machine during operation from different directions; The image data includes screen body image data for characterizing screen body posture changes and coal flow image data for characterizing coal flow distribution changes. The support status data of the screening machine is obtained, and the support status data of the screening machine is generated by the air spring pressure detection signal and the air spring height detection signal; The operation monitoring data is generated from vibration detection signals, temperature detection signals, and displacement detection signals during the operation of the screening machine. Image data, screening machine support status data, and operation monitoring data are bound together according to the same acquisition time to form multimodal operation data during the operation of the screening machine.

[0007] Optionally, S2 specifically includes: The camera calibration parameters corresponding to the first intrinsically safe mining camera and the second intrinsically safe mining camera are obtained. The camera calibration parameters are obtained by internal parameter calibration and external parameter calibration, and are used to determine the mapping relationship between the screening machine and the image coordinates in the image data. The attitude monitoring area is configured based on the visible area of ​​the screen body and the coal flow passage area in the image data. The attitude monitoring area is used to limit the extraction range of subsequent visual attitude features and coal flow off-center loading features. Under normal operating conditions of the screening machine, reference status data is extracted from image data, screening machine support status data, and operation monitoring data. The reference status data includes image reference information, support reference information, and operation reference information corresponding to the normal posture of the screening machine. Using the same acquisition time as the alignment benchmark, the multimodal operation data is time-aligned and format-unified. The camera calibration parameters, attitude monitoring area and reference state data are then written into the aligned multimodal operation data to generate a standardized attitude monitoring dataset.

[0008] Optionally, S3 specifically includes: The image data is read from the standardized attitude monitoring dataset and divided into first image data and second image data according to the acquisition source. The first image data is the image data acquired by the first intrinsically safe mining camera and the second image data is the image data acquired by the second intrinsically safe mining camera. The first image data and the second image data are input into the improved RT-DETR network. The improved RT-DETR network performs target localization and feature extraction on the visible area of ​​the screen and the coal flow passage area in the image data based on dual-view consistency constraints and coal dust occlusion degradation sample enhancement. Based on the target positioning and feature extraction results of the visible area of ​​the screen body, the visual posture features are generated, which are used to characterize the changes in the posture of the screen body during the operation of the screening machine. Based on the target location and feature extraction results of the area through which the coal flow passes, the coal flow off-load feature is generated. The coal flow off-load feature is used to characterize the coal flow distribution offset during the operation of the screening machine. The first attitude angle is generated based on the visual attitude features corresponding to the first image data, the second attitude angle is generated based on the visual attitude features corresponding to the second image data, and the coal flow off-center loading information is generated based on the coal flow off-center loading features.

[0009] Optionally, inputting the first image data and the second image data into the improved RT-DETR network specifically involves: The first image data and the second image data are input into the improved RT-DETR network to obtain a dual-view target feature map. The dual-view target feature map is used to characterize the target position, edge structure and regional distribution of the visible area of ​​the screen and the coal flow passage area under different acquisition views. The improved RT-DETR network includes a dual-view input embedding layer, a shared multi-scale feature extraction layer, a dual-view consistency constraint decoding layer, a degradation robust target detection layer, and a posture coal flow joint output layer. The dual-view input embedding layer is used to receive the first image data and the second image data respectively, and to perform scale normalization and position encoding on the first image data and the second image data to generate first-view embedding features and second-view embedding features. The shared multi-scale feature extraction layer is used to extract multi-scale features with shared parameters from the first viewpoint embedding features and the second viewpoint embedding features, generating first viewpoint multi-scale features and second viewpoint multi-scale features. The dual-view consistency constraint decoding layer embeds the dual-view consistency constraint, performs geometric correspondence matching between the same screen body target and the same coal flow target in the first-view multi-scale features and the second-view multi-scale features, and constrains the decoding direction of the target query vector according to the matching result to generate dual-view consistent target features; The degradation robust target detection layer is trained based on training samples enhanced by the coal dust occlusion degradation samples. It is used to perform target detection on the dual-view consistent target features under coal dust occlusion, partial occlusion and vibration blur conditions, and generate degradation robust target detection results. The attitude and coal flow joint output layer outputs the visual attitude features and the coal flow off-center load features based on the degraded robust target detection results, and uses the visual attitude features and the coal flow off-center load features to generate the first attitude angle, the second attitude angle, and the coal flow off-center load information.

[0010] Optionally, S4 specifically includes: Read the first attitude angle, the second attitude angle, and the coal flow off-center loading information, and judge the validity of the first attitude angle and the second attitude angle according to the preset effective angle range. The preset effective angle range is determined by the allowable attitude change range of the screening machine. The validity of the data is determined based on the changes in the coal flow off-load information during the continuous acquisition time. When the coal flow off-load information has a sudden change, is missing, or has an abnormal correspondence with the image data, it is determined that the coal flow off-load information does not meet the validity requirements. Calculate the angle deviation between the first attitude angle and the second attitude angle, and compare the angle deviation with a preset dual-camera consistency threshold to complete the dual-camera consistency verification. The preset dual-camera consistency threshold is determined by the installation position and camera calibration parameters of the first intrinsically safe mining camera and the second intrinsically safe mining camera. Based on the data validity judgment results of the first attitude angle, the data validity judgment results of the second attitude angle, the data validity judgment results of the coal flow off-center loading information, and the dual-machine consistency verification results, the attitude data credibility is generated. When the confidence level of the attitude data meets the preset confidence threshold, the main attitude angle of the screening machine is generated based on the first attitude angle and the second attitude angle. When the confidence level of the attitude data does not meet the preset confidence level threshold, the main attitude angle of the screening machine corresponding to the previous acquisition time is retained, or the attitude data is output as invalid.

[0011] Optionally, S5 specifically includes: The main attitude angle of the screening machine, the coal flow off-center load information, the support status data of the screening machine, and the operation monitoring data are read, and attitude time series samples are constructed according to the same acquisition time. The attitude time series samples are a set of data related to the attitude change of the screening machine within a continuous acquisition time. The attitude time series sample is divided into variables to obtain attitude angle variables, coal flow off-center load variables, support status variables and operation monitoring variables. The attitude angle variables are generated by the main attitude angle of the screening machine, the coal flow off-center load variables are generated by the coal flow off-center load information, the support status variables are generated by the support status data of the screening machine, and the operation monitoring variables are generated by the operation monitoring data. The attitude angle variable, the coal flow off-center load variable, the support state variable, and the operation monitoring variable are converted into a variable token sequence, which is used to characterize the change relationship of different variables over a continuous acquisition time. The variable token sequence is input into the iTransformer model, which includes an inverted variable embedding layer, a variable correlation modeling layer, a temporal variation encoding layer, and a fusion output layer. The variable token sequence is embedded and represented by the inverted variable embedding layer. The correlation between the attitude angle variable, the coal flow off-center load variable, the support state variable and the operation monitoring variable is learned by the variable correlation modeling layer. The attitude change trend features within the continuous acquisition time are extracted by the time series change encoding layer. The attitude fusion feature vector of the screening machine is generated by the fusion output layer.

[0012] Optionally, converting the attitude angle variable, the coal flow off-center load variable, the support state variable, and the operation monitoring variable into a variable token sequence specifically involves: The attitude angle variable, the coal flow off-center load variable, the support state variable, and the operation monitoring variable are divided into sliding windows according to the continuous acquisition time, generating multiple time window variable segments; For each time window variable segment, numerical normalization and missing data imputation are performed to generate standardized variable segments, which are used to eliminate differences in the units of different variables. Variable identification codes are configured for the attitude angle variable, the coal flow off-center load variable, the support state variable, and the operation monitoring variable, respectively. The variable identification codes are used to distinguish the sources of different variables. Configure time position codes for the standardized variable segments, the time position codes being used to characterize the chronological order of the standardized variable segments within a continuous acquisition time; The standardized variable fragment, the variable identifier code, and the time position code are concatenated to generate an attitude angle variable token corresponding to the attitude angle variable, a coal flow off-center load variable token corresponding to the coal flow off-center load variable, a support state variable token corresponding to the support state variable, and an operation monitoring variable token corresponding to the operation monitoring variable. The attitude angle variable token, the coal flow off-center load variable token, the support status variable token, and the operation monitoring variable token are arranged according to the order of variable source and the order of continuous acquisition time to generate the variable token sequence.

[0013] Optionally, S6 specifically includes: Read the attitude fusion feature vector of the screening machine, and parse the time-series correlation features corresponding to the main attitude angle of the screening machine, the coal flow off-center loading information, the support status data of the screening machine and the operation monitoring data from the attitude fusion feature vector of the screening machine; Based on the temporal correlation features, attitude trend prediction is performed to obtain attitude trend prediction results. The attitude trend prediction results are used to characterize the attitude change direction and attitude change amplitude of the screening machine in the subsequent collection time. A physical constraint reference relationship is constructed based on the attitude response relationship under normal operating conditions of the screening machine. The physical constraint reference relationship is obtained from the correspondence between the main attitude angle of the screening machine, the support status data of the screening machine, and the operation monitoring data. Based on the physical constraint benchmark relationship, the physical consistency residual is judged on the attitude trend prediction result, the deviation between the attitude trend prediction result and the physical constraint benchmark relationship is calculated, and physical consistency residual is generated. When the physical consistency residual does not exceed the preset residual threshold, an attitude anomaly identification result is generated based on the attitude trend prediction result; When the physical consistency residual exceeds the preset residual threshold, the abnormal source is distinguished by combining the coal flow off-center loading information, the screening machine support status data and the operation monitoring data, and an attitude abnormality identification result including at least one type of attitude abnormality, coal flow off-center loading abnormality, support abnormality or operation monitoring abnormality is generated.

[0014] Optionally, S7 specifically includes: Read the posture anomaly identification result, and determine the posture state, anomaly type, and anomaly level of the screening machine at the current collection time based on the posture anomaly identification result; Based on the attitude state, the anomaly type, the anomaly level, the main attitude angle of the screening machine, the coal flow off-center loading information, and the attitude data reliability, the attitude monitoring result of the screening machine is generated. When the posture anomaly identification result indicates that the screening machine is in a normal posture state, the posture monitoring result of the screening machine is marked as a normal monitoring state; When the posture anomaly identification result indicates that the screening machine has an abnormal posture state, the posture monitoring result of the screening machine is marked as an alarm monitoring state, and corresponding alarm information is generated according to the anomaly type and the anomaly level. The screening machine attitude monitoring results are sent to the PLC or centralized control platform according to a preset communication format, wherein the preset communication format is determined by the data receiving rules of the PLC or centralized control platform. The historical operating data is generated by associating and storing the screening machine posture monitoring results, the multimodal operation data, the standardized posture monitoring dataset, the posture anomaly identification results, and the corresponding acquisition time.

[0015] The beneficial effects of this invention are: This invention addresses the challenges of timely detection of coal flow imbalance during screening machine operation, including difficulty in detecting coal flow imbalance, significant impact of coal dust obstruction on single-view imaging, unstable attitude angle judgment, and difficulty in distinguishing the source of anomalies, by fusing dual-view image acquisition, support status monitoring, and operational monitoring data. It employs a first intrinsically safe mining camera and a second intrinsically safe mining camera to simultaneously acquire image data, and combines this with screening machine support status data and operational monitoring data to construct multimodal operational data, achieving complete acquisition of screening machine attitude monitoring information. In the visual feature extraction stage, this invention introduces an improved RT-DETR network, using dual-view consistency constraints to perform geometric correspondence verification on the same screen body target and the same coal flow target, and incorporating enhancement based on coal dust obstruction and degradation samples. This invention improves the stability of visual attitude features and coal flow off-center loading features extraction under conditions of coal dust obstruction, partial obstruction, and vibration ambiguity. In the attitude angle determination stage, attitude data credibility is generated through data validity judgment and dual-machine consistency verification, and the main attitude angle of the screening machine is generated based on the attitude data credibility, reducing angle deviation caused by false detection from a single viewpoint. In the time-series fusion stage, the invention uses the iTransformer model to perform variable-level time-series modeling of the main attitude angle of the screening machine, coal flow off-center loading information, screening machine support status data, and operation monitoring data, generating a screening machine attitude fusion feature vector. Physical consistency residual judgment is introduced to ensure that the attitude anomaly identification results simultaneously satisfy the data trend relationship and the actual support response relationship of the screening machine. Ultimately, this invention achieves real-time output of screening machine attitude monitoring results, anomaly source differentiation, alarm push notifications, and historical data traceability, offering advantages such as strong real-time monitoring, high identification accuracy, strong anti-interference capability, and applicability to continuous operation conditions in coal preparation plants. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of a screening machine attitude monitoring method based on multimodal fusion proposed in this invention; Figure 2 This is a diagram of the improved RT-DETR network structure proposed in this invention; Figure 3This is a schematic diagram of the iTransformer time-series fusion and physical consistency residual judgment proposed in this invention. Detailed Implementation

[0017] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0018] refer to Figures 1-3 A method for monitoring the attitude of a screening machine based on multimodal fusion includes the following steps: S1. Synchronously collect multimodal operation data during the operation of the screening machine. The multimodal operation data includes image data collected by the first intrinsically safe mining camera and the second intrinsically safe mining camera, as well as screening machine support status data and operation monitoring data. S2. Perform attitude benchmarking processing on the multimodal operation data to generate a standardized attitude monitoring dataset; S3. Input the image data from the standardized attitude monitoring dataset into the improved RT-DETR network, and extract visual attitude features and coal flow off-center loading features based on dual-view consistency constraints and coal dust occlusion degradation samples to generate first attitude angle, second attitude angle and coal flow off-center loading information. S4. Perform data validity judgment and dual-machine consistency verification on the first attitude angle, the second attitude angle and the coal flow off-center loading information, generate attitude data credibility, and generate the main attitude angle of the screening machine based on the attitude data credibility. S5. Input the main attitude angle of the screening machine, the coal flow off-center load information, the support status data of the screening machine and the operation monitoring data into the iTransformer model, perform multivariate time series modeling, and generate the screening machine attitude fusion feature vector. S6. Based on the posture fusion feature vector of the screening machine, perform posture trend prediction and introduce physical consistency residual judgment to generate posture anomaly identification results. S7. Generate the screening machine posture monitoring results based on the posture anomaly identification results, and send the screening machine posture monitoring results to the PLC or centralized control platform. At the same time, store the screening machine posture monitoring results and the corresponding historical operation data.

[0019] In this embodiment, S1 specifically refers to: Based on the visible area of ​​the screen body and the coal flow passage area of ​​the screening machine, a first intrinsically safe mining camera and a second intrinsically safe mining camera are respectively installed so that the first intrinsically safe mining camera and the second intrinsically safe mining camera can collect image data of the screening machine during operation from different directions; The image data includes screen image data used to characterize changes in screen posture and coal flow image data used to characterize changes in coal flow distribution. The support status data of the screening machine is obtained. The support status data of the screening machine is generated by the air spring pressure detection signal and the air spring height detection signal. The operation monitoring data is generated from vibration detection signals, temperature detection signals, and displacement detection signals during the operation of the screening machine. Image data, screening machine support status data, and operation monitoring data are bound together at the same acquisition time to form multimodal operation data during the operation of the screening machine; In the specific implementation process, the first intrinsically safe mining camera is set above the front side of the screening machine, and the second intrinsically safe mining camera is set above the side of the screening machine. The two form image acquisition angles in different directions. The acquisition frame rate is set to 20fps~30fps, and the image resolution is not less than 1920×1080, which is used to acquire image data. The support status data of the screening machine is obtained through air spring pressure sensor and air spring height sensor. The acquisition range of air spring pressure detection signal is set to 0MPa~0.8MPa, and the measurement resolution of air spring height detection signal is no greater than 1mm. Operational monitoring data is acquired through vibration sensors, temperature sensors, and displacement sensors. The sampling frequency of the vibration detection signal is set to 200Hz to 1000Hz, the sampling frequency of the temperature detection signal is set to 1Hz to 5Hz, and the sampling frequency of the displacement detection signal is set to 20Hz to 100Hz. Using the frame time of the image data as the same acquisition time, the image data, screening machine support status data, and operation monitoring data are timestamped and bound together. The time synchronization error is controlled within 20ms. The binding is completed when the time difference between different data does not exceed 50ms, forming multimodal operation data.

[0020] In this embodiment, S2 specifically refers to: The camera calibration parameters corresponding to the first intrinsically safe mining camera and the second intrinsically safe mining camera are obtained. The camera calibration parameters are obtained by internal parameter calibration and external parameter calibration, and are used to determine the mapping relationship between the screening machine and the image coordinates in the image data. The attitude monitoring area is configured based on the visible area of ​​the screen and the coal flow passage area in the image data. The attitude monitoring area is used to limit the extraction range of subsequent visual attitude features and coal flow off-center loading features. Under normal operating conditions of the screening machine, baseline status data is extracted from image data, screening machine support status data, and operation monitoring data. The baseline status data includes image baseline information, support baseline information, and operation baseline information corresponding to the normal posture of the screening machine. Using the same acquisition time as the alignment benchmark, the multimodal operation data is time-aligned and format-unified. The camera calibration parameters, attitude monitoring area and reference state data are written into the aligned multimodal operation data to generate a standardized attitude monitoring dataset. In the specific implementation process, after the first and second intrinsically safe mining cameras are installed, internal and external parameter calibrations are performed using calibration boards or fixed structural points on site to obtain camera calibration parameters. The camera calibration parameters include focal length, principal point coordinates, distortion coefficients, rotation matrix, and translation vector. The reprojection error after calibration is controlled within 0.5 pixels. The attitude monitoring area is configured according to the edge of the screen body, the position of the screen frame and the coverage of the coal flow in the image data. The boundary of the attitude monitoring area is 5% to 15% larger than the visible area of ​​the screen body and the area through which the coal flow passes, in order to avoid the target from exceeding the monitoring range due to screen body vibration or coal flow fluctuation. Data was continuously collected for 3 to 10 minutes under the conditions of stable no-load operation or normal material operation of the screening machine. The average value of normal posture position, average value of support status data of the screening machine and average value of operation monitoring data in the image data were calculated and used as image reference information, support reference information and operation reference information, respectively, and together they formed the reference status data. Based on the same acquisition time, data with a time difference of no more than 50ms in the multimodal operation data are grouped into the same data frame. The image size is unified for image data, and the units and field formats are unified for screening machine support status data and operation monitoring data. The camera calibration parameters, attitude monitoring area and reference status data are bound to the corresponding data frames to generate a standardized attitude monitoring dataset.

[0021] In this embodiment, S3 specifically refers to: Image data is read from the standardized attitude monitoring dataset and divided into first image data and second image data according to the acquisition source. The first image data is the image data acquired by the first intrinsically safe mining camera and the second image data is the image data acquired by the second intrinsically safe mining camera. The first and second image data are input into the improved RT-DETR network. The improved RT-DETR network is based on dual-view consistency constraints and coal dust occlusion degradation sample enhancement to perform target localization and feature extraction on the visible area of ​​the sieve and the coal flow passage area in the image data. Based on the target localization and feature extraction results of the visible area of ​​the screen body, visual posture features are generated. These visual posture features are used to characterize the changes in the posture of the screen body during the operation of the screening machine. Based on the target location and feature extraction results of the coal flow through the area, coal flow off-load characteristics are generated. These characteristics are used to characterize the coal flow distribution offset during the operation of the screening machine. A first attitude angle is generated based on the visual attitude features corresponding to the first image data, a second attitude angle is generated based on the visual attitude features corresponding to the second image data, and coal flow off-center loading information is generated based on the coal flow off-center loading features. In the specific implementation process, image data is read from the standardized attitude monitoring dataset at the same acquisition time, and the image data is divided into first image data and second image data according to the acquisition source identifier. The first image data and second image data are uniformly adjusted to 640×640 pixels or 960×960 pixels, the pixel values ​​are normalized to 0 to 1, and distortion correction is performed according to the camera calibration parameters to ensure that the screen edge and coal flow area maintain geometric consistency before input. The first and second image data are input into the improved RT-DETR network. The target detection confidence threshold is set to 0.45 to 0.65. Target localization results below the target detection confidence threshold are not included in subsequent calculations. The improved RT-DETR network outputs the target bounding box of the screen body, the edge response map of the screen body, and the set of attitude feature points corresponding to the visible area of ​​the screen body. At the same time, it outputs the target bounding box of the coal flow through the area, the coal flow distribution mask, and the coal flow boundary response map. Visual pose features are generated based on the target bounding box of the sieve body, the edge response map of the sieve body, and the set of pose feature points. Specifically, pose feature points with a confidence level greater than 0.6 are selected from the set of pose feature points, and the sieve body pose baseline is fitted in combination with the edge response map of the sieve body. The sieve body pose baseline is obtained by fitting using the least squares method. The fitted straight line is represented as y=kx+b, and the current tilt angle of the sieve body is calculated based on arctan(k). For the first image data, the difference between the current screen body tilt angle and the first reference tilt angle in the reference state data is used to generate the first attitude angle; for the second image data, the difference between the current screen body tilt angle and the second reference tilt angle in the reference state data is used to generate the second attitude angle; when the number of attitude feature points is less than 4 or the fitting error of the screen body attitude baseline is greater than 2 pixels, the corresponding attitude angle is marked as a state to be verified. The coal flow off-center loading characteristics are generated based on the coal flow target frame, coal flow distribution mask, and coal flow boundary response map. Specifically, the pixel distribution of the coal flow distribution mask in the width direction of the screen surface is statistically analyzed, and the offset direction and offset ratio of the coal flow distribution center relative to the coal flow reference center in the reference state data are calculated. The offset ratio is the ratio of the offset distance of the coal flow distribution center to the effective width of the screen surface. When the offset ratio is less than 5%, it is judged as no obvious off-center loading; when the offset ratio is 5% to 15%, it is judged as slight off-center loading; and when the offset ratio is greater than 15%, it is judged as obvious off-center loading. The offset direction, offset ratio, and off-center loading level are combined to generate coal flow off-center loading information.

[0022] In this embodiment, the first image data and the second image data are input into the improved RT-DETR network, specifically as follows: The first and second image data are input into the improved RT-DETR network to obtain dual-view target feature maps. The dual-view target feature maps are used to characterize the target position, edge structure and regional distribution in the visible area of ​​the screen and the coal flow area under different acquisition views. The improved RT-DETR network includes a dual-view input embedding layer, a shared multi-scale feature extraction layer, a dual-view consistency-constrained decoding layer, a degradation-robust target detection layer, and a posture-coal flow joint output layer; The dual-view input embedding layer is used to receive the first image data and the second image data respectively, and to perform scale normalization and position encoding on the first image data and the second image data to generate first-view embedding features and second-view embedding features. A shared multi-scale feature extraction layer is used to extract multi-scale features with shared parameters from the first-viewpoint embedded features and the second-viewpoint embedded features, generating first-viewpoint multi-scale features and second-viewpoint multi-scale features. The dual-view consistency constraint decoding layer embeds dual-view consistency constraints, performs geometric correspondence matching between the same screen body target and the same coal flow target in the first-view multi-scale features and the second-view multi-scale features, and constrains the decoding direction of the target query vector according to the matching results to generate dual-view consistent target features; The degraded robust target detection layer is trained on training samples enhanced by coal dust occlusion degraded samples. It is used to detect target features with consistent dual-viewpoint under coal dust occlusion, partial occlusion and vibration blur conditions, and generate degraded robust target detection results. The attitude and coal flow joint output layer outputs visual attitude features and coal flow off-center loading features based on the degraded robust target detection results, and uses the visual attitude features and coal flow off-center loading features to generate the first attitude angle, the second attitude angle and coal flow off-center loading information. In the specific implementation process, the improved RT-DETR network inherits the end-to-end target detection framework of the RT-DETR network in its overall structure, including feature extraction, feature encoding, query decoding and target output process. However, in view of the problems of dual-view acquisition, coal dust occlusion, local occlusion and vibration blur in the screening machine operation site, dual-view consistency constraint is introduced in the decoding stage, and coal dust occlusion degradation sample enhancement is introduced in the training stage. First, the first and second image data are uniformly adjusted to 640×640 pixels, the pixel values ​​are normalized to 0 to 1, and then input into the dual-view input embedding layer respectively. The dual-view input embedding layer performs embedding encoding on the two images by sharing linear mapping parameters, with the embedding dimension set to 256, to generate first-view embedding features and second-view embedding features. Then, the first-view embedding features and the second-view embedding features are input into a shared multi-scale feature extraction layer to extract target features at three scales: 1 / 8, 1 / 16, and 1 / 32. The number of channels is set to 128, 256, and 512, respectively. After scale fusion, the first-view multi-scale features and the second-view multi-scale features are generated. Next, the dual-view consistency constraint decoding layer performs geometric correspondence matching on the same screen body target and the same coal flow target according to the camera calibration parameters, and the number of target query vectors is set to 100-300; when the geometric residual between the center point of the first view target and the center point of the second view target is less than 3-8 pixels, it is determined to be a dual-view consistent target and dual-view consistent target features are generated; when the geometric residual exceeds this range, the confidence of the corresponding target query vector is reduced. The degradation robust target detection layer is trained using training samples enhanced by coal dust occlusion degradation samples. The coal dust atomization enhancement transparency coefficient is set to 0.15–0.55, the random occlusion area accounts for 5%–25% of the image area, the vibration blur convolution kernel size is set to 3–9 pixels, and the target detection confidence threshold is set to 0.45–0.65, thereby generating degradation robust target detection results. Finally, the attitude and coal flow joint output layer generates a dual-view target feature map based on the degraded robust target detection results, and outputs visual attitude features and coal flow off-center loading features. Among them, the visual attitude features consist of the target position of the screen body, the edge structure of the screen body, and attitude feature points, which are used to fit the screen body attitude baseline and calculate the first attitude angle and the second attitude angle. The coal flow off-center loading features consist of the target position of the coal flow, the coal flow distribution area, and the coal flow boundary structure, which are used to calculate the offset ratio of the coal flow distribution center relative to the coal flow reference center and generate coal flow off-center loading information. Compared to the traditional RT-DETR model, the improved RT-DETR network reduces the impact of single-view occlusion or false detection on target localization results through dual-view consistency constraints, and enhances the network's adaptability to coal dust, occlusion and vibration-blurred scenes through coal dust occlusion degradation sample enhancement, thereby improving the extraction stability of visual pose features and coal flow off-center load features.

[0023] In this embodiment, S4 specifically refers to: Read the first attitude angle, the second attitude angle, and the coal flow off-center loading information, and judge the validity of the first attitude angle and the second attitude angle according to the preset effective angle range. The preset effective angle range is determined by the allowable attitude change range of the screening machine. The validity of the data is judged based on the changes in the coal flow off-center loading information during the continuous acquisition time. When there are sudden changes, missing information or abnormal correspondence with image data in the coal flow off-center loading information, the coal flow off-center loading information is determined to not meet the validity requirements. Calculate the angle deviation between the first attitude angle and the second attitude angle, and compare the angle deviation with the preset dual-camera consistency threshold to complete the dual-camera consistency verification. The preset dual-camera consistency threshold is determined by the installation position and camera calibration parameters of the first intrinsically safe mining camera and the second intrinsically safe mining camera. Based on the data validity judgment results of the first attitude angle, the data validity judgment results of the second attitude angle, the data validity judgment results of the coal flow off-center loading information, and the dual-machine consistency verification results, the attitude data credibility is generated. When the confidence level of the attitude data meets the preset confidence threshold, the main attitude angle of the screening machine is generated based on the first attitude angle and the second attitude angle. When the confidence level of the attitude data does not meet the preset confidence level threshold, retain the main attitude angle of the screening machine corresponding to the previous acquisition time, or output the attitude data as invalid. In the specific implementation process, the preset effective angle range is set to -6° to 6°. When the first attitude angle or the second attitude angle exceeds this range, the corresponding attitude angle is marked as invalid; when the attitude angle is within this range and the change does not exceed 1.5° in three consecutive acquisition time periods, the corresponding attitude angle is marked as valid. When judging the continuity of coal flow off-center loading information, the direction of coal flow offset, the offset ratio, and the off-center loading level are read. When the offset ratio changes by more than 20% in adjacent acquisition time, or when coal flow off-center loading information is missing for two consecutive acquisition time, the coal flow off-center loading information is judged to not meet the validity requirements. When the coal flow offset direction is inconsistent with the position change of the coal flow distribution area in the image data, it is also judged to be an abnormal correspondence with the image data. Calculate the angular deviation between the first attitude angle and the second attitude angle. The angular deviation is the absolute value of the difference between the two. The preset dual-machine consistency threshold is set to 0.5°~1.0°. When the angular deviation does not exceed the preset dual-machine consistency threshold, the dual-machine consistency verification is deemed to have passed. The attitude data reliability is generated by weighting the validity of the first attitude angle, the validity of the second attitude angle, the validity of the coal flow off-center loading information, and the consistency verification results of the two machines. The weights of each item are set to 0.3, 0.3, 0.2, and 0.2, respectively. When a certain item passes the judgment, the corresponding value is 1, and when it fails, the corresponding value is 0. The attitude data reliability is obtained by weighted summation. The preset confidence threshold is set to 0.75. When the confidence of the attitude data is not less than 0.75, the main attitude angle of the screening machine is generated based on the weighted average of the first attitude angle and the second attitude angle. When the confidence of the two attitude angles is the same, the arithmetic mean of the two is taken. When the confidence of the attitude data is less than 0.75, the main attitude angle of the screening machine corresponding to the previous acquisition time is retained, and the attitude data invalid status is output.

[0024] In this embodiment, S5 specifically refers to: Read the main attitude angle of the screening machine, coal flow off-center load information, screening machine support status data and operation monitoring data, and construct attitude time series samples according to the same acquisition time. The attitude time series samples are a set of data related to the attitude change of the screening machine within a continuous acquisition time. The attitude time series samples are divided into variables to obtain attitude angle variables, coal flow off-center load variables, support status variables and operation monitoring variables. The attitude angle variable is generated by the main attitude angle of the screening machine, the coal flow off-center load variable is generated by the coal flow off-center load information, the support status variable is generated by the support status data of the screening machine, and the operation monitoring variable is generated by the operation monitoring data. The attitude angle variable, coal flow off-center load variable, support status variable and operation monitoring variable are converted into variable token sequence. The variable token sequence is used to characterize the change relationship of different variables during continuous acquisition time. The variable token sequence is input into the iTransformer model, which includes an inverted variable embedding layer, a variable correlation modeling layer, a temporal variation encoding layer, and a fusion output layer. The variable token sequence is embedded and represented by an inverted variable embedding layer. The correlation between attitude angle variables, coal flow off-center load variables, support state variables and operation monitoring variables is learned by a variable correlation modeling layer. The attitude change trend features within the continuous acquisition time are extracted by a time series change coding layer. Finally, the attitude fusion feature vector of the screening machine is generated by a fusion output layer. In the specific implementation process, a collection interval of 100ms to 500ms is used to combine the main attitude angle of the screening machine, coal flow off-center load information, screening machine support status data and operation monitoring data within 2s to 10s into an attitude time series sample, and the sliding step size is set to 1 to 5 collection intervals. The attitude angle variable is generated by the main attitude angle of the screening machine and the angle change in adjacent acquisition time. The angle change is the difference between the current main attitude angle of the screening machine and the main attitude angle of the screening machine in the previous acquisition time. The coal flow off-center load variable is generated by the coal flow offset direction, offset ratio and off-center load level. The coal flow offset direction can be encoded as -1, 0 and 1, which represent left offset, no obvious offset and right offset, respectively. The support state variables are generated from the air spring pressure value, air spring height value, pressure difference between the left and right supports, and height difference between the left and right supports; the operation monitoring variables are generated from the effective value of vibration, peak vibration value, temperature value, temperature rise change, displacement, and displacement change rate. Normalize each variable, with the normalization range set to 0 to 1, and fill in missing values ​​with valid values ​​from the previous collection time; convert the normalized variables into a variable token sequence according to their source and time order, and input them into the iTransformer model; The inverted variable embedding layer uses different variables as modeling objects, with embedding dimensions set to 64–256; the variable correlation modeling layer adopts a 2–4 ​​layer attention encoding structure, with the number of attention heads set to 4–8, to calculate the correlation weights between attitude angle variables, coal flow off-center load variables, support state variables, and operation monitoring variables; the time-series change encoding layer extracts the change trend within the continuous acquisition time; and the fusion output layer concatenates and linearly maps the hidden features corresponding to each variable to generate a screening machine attitude fusion feature vector with dimensions of 128–512. By modeling variables as tokens, we can more directly learn the influence of coal flow eccentricity, support status changes, and operational monitoring fluctuations on the main attitude angle of the screening machine, thereby improving the stability of subsequent attitude trend prediction.

[0025] In this embodiment, the attitude angle variable, coal flow off-center load variable, support state variable, and operation monitoring variable are converted into a variable token sequence, specifically as follows: Based on the continuous acquisition time, the attitude angle variable, coal flow off-center load variable, support status variable and operation monitoring variable are divided into sliding windows to generate multiple time window variable segments. For each time window variable segment, numerical normalization and missing data imputation are performed to generate standardized variable segments, which are used to eliminate differences in the units of different variables. Variable identification codes are configured for attitude angle variables, coal flow off-center load variables, support status variables, and operation monitoring variables. The variable identification codes are used to distinguish the sources of different variables. Configure time position codes for standardized variable segments. Time position codes are used to represent the chronological order of standardized variable segments within a continuous acquisition time. The standardized variable fragments, variable identifier codes, and time location codes are spliced ​​together to generate attitude angle variable tokens corresponding to attitude angle variables, coal flow off-center load variable tokens corresponding to coal flow off-center load variables, support state variable tokens corresponding to support state variables, and operation monitoring variable tokens corresponding to operation monitoring variables. Arrange the attitude angle variable tokens, coal flow off-center load variable tokens, support status variable tokens, and operation monitoring variable tokens according to the order of variable source and the order of continuous acquisition time to generate a variable token sequence; In the specific implementation process, the acquisition interval is 100ms to 500ms. The attitude angle variable, coal flow off-center load variable, support status variable and operation monitoring variable are divided into sliding windows. The sliding window length is set to 2s to 10s, and the sliding step size is set to 1 to 5 acquisition intervals. Each sliding window contains a segment of attitude angle variable, coal flow off-center load variable, support status variable and operation monitoring variable within the same time range, thereby generating multiple time window variable segments. For each time window variable segment, numerical normalization is performed using max-min normalization to map the variable values ​​to the range of 0 to 1. When a variable is missing in a single acquisition time, the valid value of the previous acquisition time is used to fill it in. When the consecutive missing time exceeds 3 acquisition intervals, linear interpolation is performed using adjacent valid values ​​within the same sliding window, and a missing mark is added to the corresponding position to generate a standardized variable segment. Variable identification codes are configured for different variable sources. Different variable numbers or learnable embedding vectors are configured for attitude angle variables, coal flow off-center load variables, support status variables and operation monitoring variables. The variable identification code dimension is set to 8 to 32 to enable the subsequent model to distinguish different variable sources. Configure time position codes for standardized variable segments. The time position codes are generated in the order of collection time, and the dimension is set to 16-64 to represent the sequential position of each standardized variable segment within the sliding window. Standardized variable fragments, variable identifier codes, and time location codes are concatenated and converted into variable tokens of a unified dimension through linear mapping. The unified dimension is set to 64-256, generating attitude angle variable tokens, coal flow off-center load variable tokens, support status variable tokens, and operation monitoring variable tokens, respectively. Finally, according to the source order of the attitude angle variable, coal flow off-center load variable, support state variable and operation monitoring variable, and combined with the time order of each variable within the continuous acquisition time, the variable tokens are arranged to generate a variable token sequence; the variable token sequence retains both the source differences between different variables and the change relationship of the same variable within the continuous acquisition time.

[0026] In this embodiment, S6 specifically refers to: Read the posture fusion feature vector of the screening machine, and analyze the time-series correlation features corresponding to the main posture angle of the screening machine, coal flow off-center loading information, support status data of the screening machine and operation monitoring data from the posture fusion feature vector of the screening machine; Attitude trend prediction is performed based on temporal correlation features to obtain attitude trend prediction results. The attitude trend prediction results are used to characterize the attitude change direction and attitude change amplitude of the screening machine in the subsequent collection time. The physical constraint reference relationship is constructed based on the attitude response relationship of the screening machine under normal operating conditions. The physical constraint reference relationship is obtained from the correspondence between the main attitude angle of the screening machine, the support status data of the screening machine, and the operation monitoring data. Based on the physical constraint benchmark relationship, the physical consistency residual of the attitude trend prediction result is judged, the degree of deviation between the attitude trend prediction result and the physical constraint benchmark relationship is calculated, and the physical consistency residual is generated. When the physical consistency residual does not exceed the preset residual threshold, the attitude anomaly identification result is generated based on the attitude trend prediction result; When the physical consistency residual exceeds the preset residual threshold, the source of the anomaly is distinguished by combining the coal flow off-center loading information, the screening machine support status data and the operation monitoring data, and an attitude anomaly identification result is generated, which includes at least one type of attitude anomaly, coal flow off-center loading anomaly, support anomaly or operation monitoring anomaly. In the specific implementation process, after reading the posture fusion feature vector of the screening machine, the posture fusion feature vector of the screening machine is decomposed into posture angle correlation feature, coal flow off-center load correlation feature, support status correlation feature and operation monitoring correlation feature through the linear mapping layer. The output dimension of the linear mapping layer is set to 64~256. Based on temporal correlation features, the attitude change within the next 0.5s to 2s is predicted to obtain the attitude trend prediction result. The attitude trend prediction result includes the predicted attitude angle, attitude change direction and attitude change amplitude. The attitude change direction is determined by the difference between the predicted attitude angle and the current main attitude angle of the screening machine. When the difference is positive, it means that the screening machine is tilted in a preset positive direction. When the difference is negative, it means that the screening machine is tilted in a preset negative direction. The preset positive direction and preset negative direction are determined by the installation coordinate system in the width direction of the screening machine. The physical constraint reference relationship is obtained by fitting data under normal operating conditions of the screening machine. 30-120 minutes of continuous normal operating data are selected. The main attitude angle of the screening machine is used as the output, and the air spring pressure difference, air spring height difference, vibration effective value, and displacement are used as the input. Least square fitting is used to obtain the physical reference angle calculation relationship. The absolute value of the difference between the predicted attitude angle and the physical reference angle in the attitude trend prediction result is calculated as the physical consistency residual. The preset residual threshold is set to 0.3°~0.8°. When the physical consistency residual does not exceed the preset residual threshold, the attitude anomaly identification result is generated according to the predicted attitude angle. When the physical consistency residual exceeds the preset residual threshold, further distinction is made based on the abnormality source judgment rules: when the predicted attitude angle exceeds ±3°, it is judged as attitude abnormality; when the offset ratio in the coal flow off-center loading information exceeds 15%, it is judged as coal flow off-center loading abnormality; when the air spring pressure difference exceeds 0.05MPa~0.12MPa or the air spring height difference exceeds 5mm~15mm, it is judged as support abnormality; when the effective vibration value increases by more than 30% relative to the reference value or the temperature value increases by more than 10℃~20℃ relative to the reference value, it is judged as operation monitoring abnormality. By introducing physical consistency residual judgment, we can avoid misjudgment caused by relying solely on the prediction results of time series models, and make the posture anomaly identification results simultaneously satisfy the data trend relationship and the actual support response relationship of the screening machine, thereby improving the reliability of anomaly source differentiation.

[0027] In this embodiment, S7 specifically refers to: Read the attitude anomaly identification results and determine the attitude state, anomaly type, and anomaly level of the screening machine at the current acquisition time based on the attitude anomaly identification results; Based on the attitude status, anomaly type, anomaly level, main attitude angle of the screening machine, coal flow off-center loading information, and attitude data reliability, the attitude monitoring results of the screening machine are generated. When the posture anomaly identification result indicates that the screening machine is in a normal posture state, the posture monitoring result of the screening machine is marked as a normal monitoring state; When the posture anomaly identification result indicates that the screening machine has an abnormal posture state, the posture monitoring result of the screening machine is marked as an alarm monitoring state, and corresponding alarm information is generated according to the anomaly type and anomaly level. The screening machine attitude monitoring results are sent to the PLC or centralized control platform according to the preset communication format, which is determined by the data receiving rules of the PLC or centralized control platform. The screening machine posture monitoring results, multimodal operation data, standardized posture monitoring dataset, posture anomaly identification results, and corresponding acquisition time are associated and stored to generate historical operation data; In the specific implementation process, after reading the abnormal posture identification results, the posture status of the screening machine is divided into three states: normal, warning, and alarm. When the main posture angle of the screening machine is less than 1° and there are no abnormalities in coal flow off-center loading, support, or operation monitoring, it is determined to be in a normal state. When the main posture angle of the screening machine is between 1° and 3°, or the offset ratio in the coal flow off-center loading information is between 5% and 15%, it is determined to be in a warning state. When the main posture angle of the screening machine exceeds 3°, or the offset ratio in the coal flow off-center loading information exceeds 15%, or the air spring pressure difference, vibration effective value, or temperature value exceeds the corresponding threshold, it is determined to be in an alarm state. The attitude monitoring results of the screening machine are generated based on the attitude status, anomaly type, anomaly level, main attitude angle of the screening machine, coal flow off-center loading information, and attitude data reliability. The attitude monitoring results include the acquisition time, attitude status code, anomaly type code, anomaly level, main attitude angle of the screening machine, coal flow offset direction, coal flow offset ratio, and attitude data reliability. The attitude status code can be set to 0, 1, and 2, which represent normal, warning, and alarm, respectively. When an alarm is generated, a Level 1 alarm corresponds to a slight attitude deviation or a slight coal flow imbalance, a Level 2 alarm corresponds to a significant attitude deviation or a significant coal flow imbalance, and a Level 3 alarm corresponds to a support abnormality or an operational monitoring abnormality combined with an attitude abnormality. The alarm information includes the alarm time, alarm location, alarm cause, suggested handling method, and current monitoring data. The screening machine's attitude monitoring results are sent to the PLC or centralized control platform via Modbus TCP, Socket TCP, or Industrial Ethernet. The sending period is set to 100ms to 1000ms. An event report is triggered immediately in abnormal conditions. The communication fields include Status_Word, Angle_Main, Bias_Ratio, Confidence, Alarm_Level, and Timestamp. The screening machine posture monitoring results, multimodal operation data, standardized posture monitoring dataset, posture anomaly identification results and corresponding acquisition time are associated and stored according to the equipment number. The data retention period is set to no less than 30 days. Alarm data is indexed separately to generate historical operation data. By converting the abnormal posture recognition results into screening machine posture monitoring results that can be directly read by PLC or centralized control platform, it is possible to realize real-time output of screening machine posture status, alarm traceability and operation data retention, reduce the problem of lag in manual inspection, and provide data basis for subsequent screening machine posture adjustment and fault analysis.

[0028] Example 1: To verify the feasibility of this invention in practice, it was applied to the posture monitoring scenario of a screening machine in a coal preparation plant's screening production line. During continuous operation, the feed coal flow of this screening machine is prone to uneven distribution from left to right. At certain times, due to changes in the moisture content of the coal flow and fluctuations in the feed rate, short-term accumulation occurs on one side of the screen surface, causing a slight tilt of the screen body. At the same time, there is a large amount of coal dust on site, and traditional single-camera visual detection is easily affected by obstruction. Manual inspection also has the problem of delayed detection. Usually, it can only be determined that the screening machine has an off-center load or abnormal posture after the screen body vibration increases significantly, the screen plate wears faster, or the air spring support is abnormal, making it difficult to make timely adjustments.

[0029] In this scenario, a first intrinsically safe mining camera is positioned above and in front of the feed end of the screening machine, while a second intrinsically safe mining camera is positioned above and to the side of the screening machine. This allows the two cameras to acquire image data of the screen body and coal flow from different directions. Pressure and height sensors are installed at the air spring support of the screening machine to obtain support status data. Vibration, temperature, and displacement sensors are installed at key locations on the screen body to acquire operational monitoring data. The system binds the image data, screening machine support status data, and operational monitoring data at the same acquisition time to generate multimodal operational data, and then generates a standardized attitude monitoring dataset through attitude benchmarking processing.

[0030] In actual operation, the improved RT-DETR network performs target localization and feature extraction on the visible area of ​​the screen and the coal flow passage area in the dual-view images, generating a first attitude angle, a second attitude angle, and coal flow off-center loading information. Subsequently, the first and second attitude angles are checked for dual-machine consistency to generate the main attitude angle of the screening machine. Then, the main attitude angle of the screening machine, coal flow off-center loading information, screening machine support status data, and operation monitoring data are input into the iTransformer model to generate a screening machine attitude fusion feature vector. Combined with physical consistency residual judgment, the output attitude anomaly identification result is determined. After receiving the screening machine attitude monitoring results, the PLC or centralized control platform can prompt operators to adjust the feeding, check the air spring support status, or arrange equipment maintenance based on the alarm level.

[0031] In the initial stage of operation, the normal material-carrying state of the screening machine was used as the baseline state to calibrate the viewing angle, screen body reference angle, coal flow reference center, air spring pressure reference value, and vibration reference value of the two intrinsically safe mining cameras. Once the screening machine entered continuous operation, the system continuously updated the main attitude angle of the screening machine and coal flow off-center loading information according to a fixed acquisition cycle, and simultaneously saved the image frames, support status data, and operational monitoring data before and after each alarm. In this way, the system can not only determine whether there is an attitude anomaly, but also retain the changes before and after the anomaly occurs, facilitating subsequent analysis of the correlation between coal flow off-center loading, support status changes, and screen body tilt.

[0032] During the continuous operation period, the screening machine's throughput remained between 420 t / h and 560 t / h. The image acquisition frame rate was 25 fps, and the dual-view image resolution was 3840×2160. The system acquired approximately 648,000 valid image frames and recorded approximately 129,000 sets of multimodal data. After manual verification and comparison with equipment operation records and alarm records, the system was able to reliably identify normal posture, slight coal flow imbalance, significant coal flow imbalance, abnormal posture angle, abnormal air spring support, and combined vibration and temperature rise anomalies. Specific operational statistics are shown in Table 1.

[0033] Table 1. Statistical Table of Implementation and Operation of Attitude Monitoring of the Screening Machine of the Present Invention

[0034] As shown in Table 1, the present invention maintains good real-time performance, synchronization, and recognition stability even under conditions of coal dust interference, screen vibration, and coal flow fluctuation. The average data synchronization error is 16.4 ms, significantly lower than the 50 ms threshold set for multimodal data binding, indicating that image data, screening machine support status data, and operation monitoring data can form an effective correspondence at the same acquisition time, providing a reliable data foundation for subsequent multimodal fusion analysis. The average attitude recognition response time is 173 ms, which meets the real-time monitoring requirements of the screening machine under continuous operation, enabling the PLC or centralized control platform to receive attitude monitoring results and trigger early warnings or control prompts in a short time.

[0035] In terms of recognition accuracy, the average error of the attitude angle is 0.21°, indicating that by using a dual-view acquisition system with both intrinsically safe mining cameras and a dual-view consistency constraint for verification, the angular deviation caused by coal dust obstruction, partial obstruction, or vibration blurring of a single camera can be effectively reduced. The coal flow off-center loading recognition accuracy reaches 96.4%, indicating that the improved RT-DETR network can accurately extract coal flow off-center loading characteristics and has a strong ability to identify coal flow distribution center shift, coal flow boundary changes, and off-center loading degree. The average early warning time is 1.2s, indicating that the system can identify the development trend of coal flow off-center loading or abnormal support status before the screen body attitude deteriorates significantly, providing time margin for operators to adjust the feed rate, check the air spring support status, or take pre-shutdown measures. Overall, this invention can not only output the judgment result of whether the screening machine is abnormal, but also improve the reliability of abnormal judgment by combining attitude angle, coal flow off-center loading information, and operation monitoring data, reducing the problems of lag in manual inspection and false judgment of single threshold alarms.

[0036] To further verify the beneficial effects of this invention compared to existing monitoring methods, it was compared with manual inspection, single-camera visual detection, and single vibration threshold alarm methods. During the comparison, all methods were used for the same type of screening machine posture anomaly monitoring scenario. Statistical items included response time, angle error, accuracy of coal flow off-center loading identification, ability to distinguish anomaly sources, false negative rate, and reduction in the number of off-center loading-related shutdowns. The comparison results are shown in Table 2.

[0037] Table 2 Comparison of different posture monitoring methods for screening machines

[0038] Table 2 shows that different screening machine attitude monitoring methods differ significantly in response speed, identification accuracy, and ability to determine the source of anomalies. Manual inspection relies primarily on maintenance personnel periodically observing the screen's operating status, coal flow distribution, and equipment vibration. The average response time is 8–15 minutes, significantly lagging behind the development of abnormal screening machine attitudes. Manual inspection is only able to detect problems when coal flow imbalance has already caused screen tilting, localized screen plate wear, or abnormal air spring support conditions. Therefore, the accuracy rate of coal flow imbalance identification is only 72.5%, with a false negative rate of 13.8%, making it difficult to meet the real-time early warning requirements for continuous operation scenarios.

[0039] The average response time of the single-camera visual inspection method is 310ms, which is significantly shorter than that of manual inspection. It can also output the attitude angle. However, because it relies on only a single acquisition perspective, it is prone to problems such as incomplete identification of the screen edge, coal flow boundary offset, and increased attitude angle error when there is coal dust obstruction, partial obstruction, lens contamination, or limited perspective. Therefore, the average attitude angle error is 0.47°, and the accuracy rate of coal flow off-center loading identification is 88.6%, which still has some false detections and false negatives. Although the single vibration threshold alarm method can respond to abnormal equipment vibration, it can only determine whether the vibration value exceeds the threshold. It cannot directly output the main attitude angle of the screening machine, nor can it identify the direction and degree of coal flow off-center loading. Therefore, it has obvious shortcomings in coal flow off-center loading identification, attitude angle output, and differentiation of abnormal sources.

[0040] This invention acquires dual-view image data using a first intrinsically safe mining camera and a second intrinsically safe mining camera, and integrates the support status data and operational monitoring data of the screening machine. This enables the system to simultaneously analyze changes in the screen's posture, coal flow distribution deviation, air spring support status, and equipment operating status. The average response time of this invention is 173ms, faster than single-camera visual detection and single vibration threshold alarm methods. The average posture angle error is reduced to 0.21°, indicating that dual-view consistency constraints can effectively suppress angle deviations caused by false detections from a single viewpoint. The accuracy rate of coal flow off-center loading identification is improved to 96.4%, demonstrating that the improved RT-DETR network has a better ability to extract coal flow off-center loading characteristics in complex coal dust environments. Simultaneously, the false negative rate of this invention is only 2.2%, and the number of off-center loading-related shutdowns is reduced by approximately 71%, indicating that multivariate time-series modeling using the iTransformer model, combined with physical consistency residual judgment, can more accurately distinguish between posture anomalies, coal flow off-center loading anomalies, support anomalies, and operational monitoring anomalies. Overall, this invention not only improves the real-time performance and accuracy of the screening machine's attitude monitoring, but also enhances the interpretability and traceability of alarm results. It can effectively solve the problems of delayed early warning, unstable angle judgment, insufficient differentiation of abnormal sources, and frequent downtime for maintenance in existing methods.

[0041] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for monitoring the attitude of a screening machine based on multimodal fusion, characterized in that, Includes the following steps: S1. Synchronously collect multimodal operation data during the operation of the screening machine. The multimodal operation data includes image data collected by the first intrinsically safe mining camera and the second intrinsically safe mining camera, as well as screening machine support status data and operation monitoring data. S2. Perform attitude benchmarking processing on the multimodal operation data to generate a standardized attitude monitoring dataset; S3. Input the image data from the standardized attitude monitoring dataset into the improved RT-DETR network, and extract visual attitude features and coal flow off-center loading features based on dual-view consistency constraints and coal dust occlusion degradation samples to generate the first attitude angle, the second attitude angle and coal flow off-center loading information. S4. Perform data validity judgment and dual-machine consistency verification on the first attitude angle, the second attitude angle and the coal flow off-center loading information, generate attitude data credibility, and generate the main attitude angle of the screening machine based on the attitude data credibility. S5. Input the main attitude angle of the screening machine, the coal flow off-center load information, the support status data of the screening machine and the operation monitoring data into the iTransformer model, perform multivariate time series modeling, and generate the screening machine attitude fusion feature vector. S6. Based on the posture fusion feature vector of the screening machine, perform posture trend prediction and introduce physical consistency residual judgment to generate posture anomaly identification results. S7. Generate the screening machine posture monitoring results based on the posture anomaly identification results, and send the screening machine posture monitoring results to the PLC or centralized control platform. At the same time, store the screening machine posture monitoring results and the corresponding historical operation data.

2. The method for monitoring the attitude of a screening machine based on multimodal fusion according to claim 1, characterized in that, Specifically, S1 is: Based on the visible area of ​​the screen body and the coal flow passage area of ​​the screening machine, a first intrinsically safe mining camera and a second intrinsically safe mining camera are respectively installed so that the first intrinsically safe mining camera and the second intrinsically safe mining camera can collect image data of the screening machine during operation from different directions; The image data includes screen body image data for characterizing screen body posture changes and coal flow image data for characterizing coal flow distribution changes. The support status data of the screening machine is obtained, and the support status data of the screening machine is generated by the air spring pressure detection signal and the air spring height detection signal; The operation monitoring data is generated from vibration detection signals, temperature detection signals, and displacement detection signals during the operation of the screening machine. Image data, screening machine support status data, and operation monitoring data are bound together according to the same acquisition time to form multimodal operation data during the operation of the screening machine.

3. The method for monitoring the attitude of a screening machine based on multimodal fusion according to claim 1, characterized in that, Specifically, S2 is: The camera calibration parameters corresponding to the first intrinsically safe mining camera and the second intrinsically safe mining camera are obtained. The camera calibration parameters are obtained by internal parameter calibration and external parameter calibration, and are used to determine the mapping relationship between the screening machine and the image coordinates in the image data. The attitude monitoring area is configured based on the visible area of ​​the screen body and the coal flow passage area in the image data. The attitude monitoring area is used to limit the extraction range of subsequent visual attitude features and coal flow off-center loading features. Under normal operating conditions of the screening machine, reference status data is extracted from image data, screening machine support status data, and operation monitoring data. The reference status data includes image reference information, support reference information, and operation reference information corresponding to the normal posture of the screening machine. Using the same acquisition time as the alignment benchmark, the multimodal operation data is time-aligned and format-unified. The camera calibration parameters, attitude monitoring area and reference state data are then written into the aligned multimodal operation data to generate a standardized attitude monitoring dataset.

4. The method for monitoring the attitude of a screening machine based on multimodal fusion according to claim 1, characterized in that, Specifically, S3 is: The image data is read from the standardized attitude monitoring dataset and divided into first image data and second image data according to the acquisition source. The first image data is the image data acquired by the first intrinsically safe mining camera and the second image data is the image data acquired by the second intrinsically safe mining camera. The first image data and the second image data are input into the improved RT-DETR network. The improved RT-DETR network performs target localization and feature extraction on the visible area of ​​the screen and the coal flow passage area in the image data based on dual-view consistency constraints and coal dust occlusion degradation sample enhancement. Based on the target positioning and feature extraction results of the visible area of ​​the screen body, the visual posture features are generated, which are used to characterize the changes in the posture of the screen body during the operation of the screening machine. Based on the target location and feature extraction results of the area through which the coal flow passes, the coal flow off-load feature is generated. The coal flow off-load feature is used to characterize the coal flow distribution offset during the operation of the screening machine. The first attitude angle is generated based on the visual attitude features corresponding to the first image data, the second attitude angle is generated based on the visual attitude features corresponding to the second image data, and the coal flow off-center loading information is generated based on the coal flow off-center loading features.

5. The method for monitoring the attitude of a screening machine based on multimodal fusion according to claim 4, characterized in that, The step of inputting the first image data and the second image data into the improved RT-DETR network specifically involves: The first image data and the second image data are input into the improved RT-DETR network to obtain a dual-view target feature map. The dual-view target feature map is used to characterize the target position, edge structure and regional distribution of the visible area of ​​the screen and the coal flow passage area under different acquisition views. The improved RT-DETR network includes a dual-view input embedding layer, a shared multi-scale feature extraction layer, a dual-view consistency constraint decoding layer, a degradation robust target detection layer, and a posture coal flow joint output layer. The dual-view input embedding layer is used to receive the first image data and the second image data respectively, and to perform scale normalization and position encoding on the first image data and the second image data to generate first-view embedding features and second-view embedding features. The shared multi-scale feature extraction layer is used to extract multi-scale features with shared parameters from the first viewpoint embedding features and the second viewpoint embedding features, generating first viewpoint multi-scale features and second viewpoint multi-scale features. The dual-view consistency constraint decoding layer embeds the dual-view consistency constraint, performs geometric correspondence matching between the same screen body target and the same coal flow target in the first-view multi-scale features and the second-view multi-scale features, and constrains the decoding direction of the target query vector according to the matching result to generate dual-view consistent target features; The degradation robust target detection layer is trained based on training samples enhanced by the coal dust occlusion degradation samples. It is used to perform target detection on the dual-view consistent target features under coal dust occlusion, partial occlusion and vibration blur conditions, and generate degradation robust target detection results. The attitude and coal flow joint output layer outputs the visual attitude features and the coal flow off-center load features based on the degraded robust target detection results, and uses the visual attitude features and the coal flow off-center load features to generate the first attitude angle, the second attitude angle, and the coal flow off-center load information.

6. The method for monitoring the attitude of a screening machine based on multimodal fusion according to claim 1, characterized in that, Specifically, S4 is: Read the first attitude angle, the second attitude angle, and the coal flow off-center loading information, and judge the validity of the first attitude angle and the second attitude angle according to the preset effective angle range. The preset effective angle range is determined by the allowable attitude change range of the screening machine. The validity of the data is determined based on the changes in the coal flow off-load information during the continuous acquisition time. When the coal flow off-load information has a sudden change, is missing, or has an abnormal correspondence with the image data, it is determined that the coal flow off-load information does not meet the validity requirements. Calculate the angle deviation between the first attitude angle and the second attitude angle, and compare the angle deviation with a preset dual-camera consistency threshold to complete the dual-camera consistency verification. The preset dual-camera consistency threshold is determined by the installation position and camera calibration parameters of the first intrinsically safe mining camera and the second intrinsically safe mining camera. Based on the data validity judgment results of the first attitude angle, the data validity judgment results of the second attitude angle, the data validity judgment results of the coal flow off-center loading information, and the dual-machine consistency verification results, the attitude data credibility is generated. When the confidence level of the attitude data meets the preset confidence threshold, the main attitude angle of the screening machine is generated based on the first attitude angle and the second attitude angle. When the confidence level of the attitude data does not meet the preset confidence level threshold, the main attitude angle of the screening machine corresponding to the previous acquisition time is retained, or the attitude data is output as invalid.

7. The method for monitoring the attitude of a screening machine based on multimodal fusion according to claim 1, characterized in that, Specifically, S5 is: The main attitude angle of the screening machine, the coal flow off-center load information, the support status data of the screening machine, and the operation monitoring data are read, and attitude time series samples are constructed according to the same acquisition time. The attitude time series samples are a set of data related to the attitude change of the screening machine within a continuous acquisition time. The attitude time series sample is divided into variables to obtain attitude angle variables, coal flow off-center load variables, support status variables and operation monitoring variables. The attitude angle variables are generated by the main attitude angle of the screening machine, the coal flow off-center load variables are generated by the coal flow off-center load information, the support status variables are generated by the support status data of the screening machine, and the operation monitoring variables are generated by the operation monitoring data. The attitude angle variable, the coal flow off-center load variable, the support state variable, and the operation monitoring variable are converted into a variable token sequence, which is used to characterize the relationship between the changes of different variables over a continuous acquisition period. The variable token sequence is input into the iTransformer model, which includes an inverted variable embedding layer, a variable correlation modeling layer, a temporal variation encoding layer, and a fusion output layer. The variable token sequence is embedded and represented by the inverted variable embedding layer. The correlation between the attitude angle variable, the coal flow off-center load variable, the support state variable and the operation monitoring variable is learned by the variable correlation modeling layer. The attitude change trend features within the continuous acquisition time are extracted by the time series change encoding layer. The attitude fusion feature vector of the screening machine is generated by the fusion output layer.

8. The method for monitoring the attitude of a screening machine based on multimodal fusion according to claim 7, characterized in that, The process of converting the attitude angle variable, the coal flow off-center load variable, the support state variable, and the operation monitoring variable into a variable token sequence specifically involves: The attitude angle variable, the coal flow off-center load variable, the support state variable, and the operation monitoring variable are divided into sliding windows according to the continuous acquisition time, generating multiple time window variable segments; For each time window variable segment, numerical normalization and missing data imputation are performed to generate standardized variable segments, which are used to eliminate differences in the units of different variables. Variable identification codes are configured for the attitude angle variable, the coal flow off-center load variable, the support state variable, and the operation monitoring variable, respectively. The variable identification codes are used to distinguish the sources of different variables. Configure time position codes for the standardized variable segments, the time position codes being used to characterize the chronological order of the standardized variable segments within a continuous acquisition time; The standardized variable fragment, the variable identifier code, and the time position code are concatenated to generate an attitude angle variable token corresponding to the attitude angle variable, a coal flow off-center load variable token corresponding to the coal flow off-center load variable, a support state variable token corresponding to the support state variable, and an operation monitoring variable token corresponding to the operation monitoring variable. The attitude angle variable token, the coal flow off-center load variable token, the support status variable token, and the operation monitoring variable token are arranged according to the order of variable source and the order of continuous acquisition time to generate the variable token sequence.

9. The method for monitoring the attitude of a screening machine based on multimodal fusion according to claim 1, characterized in that, Specifically, S6 is: Read the attitude fusion feature vector of the screening machine, and parse the time-series correlation features corresponding to the main attitude angle of the screening machine, the coal flow off-center loading information, the support status data of the screening machine and the operation monitoring data from the attitude fusion feature vector of the screening machine; Based on the temporal correlation features, attitude trend prediction is performed to obtain attitude trend prediction results. The attitude trend prediction results are used to characterize the attitude change direction and attitude change amplitude of the screening machine in the subsequent collection time. A physical constraint reference relationship is constructed based on the attitude response relationship of the screening machine under normal operating conditions. The physical constraint reference relationship is obtained from the correspondence between the main attitude angle of the screening machine, the support status data of the screening machine, and the operation monitoring data. Based on the physical constraint benchmark relationship, the physical consistency residual is judged on the attitude trend prediction result, the deviation between the attitude trend prediction result and the physical constraint benchmark relationship is calculated, and physical consistency residual is generated. When the physical consistency residual does not exceed the preset residual threshold, an attitude anomaly identification result is generated based on the attitude trend prediction result; When the physical consistency residual exceeds the preset residual threshold, the abnormal source is distinguished by combining the coal flow off-center loading information, the screening machine support status data and the operation monitoring data, and an abnormality identification result including at least one type of abnormality, such as abnormality of posture, abnormality of coal flow off-center loading, abnormality of support or abnormality of operation monitoring is generated.

10. The method for monitoring the attitude of a screening machine based on multimodal fusion according to claim 1, characterized in that, Specifically, S7 is: Read the posture anomaly identification result, and determine the posture state, anomaly type, and anomaly level of the screening machine at the current collection time based on the posture anomaly identification result; Based on the attitude state, the anomaly type, the anomaly level, the main attitude angle of the screening machine, the coal flow off-center loading information, and the attitude data reliability, the attitude monitoring result of the screening machine is generated. When the posture anomaly identification result indicates that the screening machine is in a normal posture state, the posture monitoring result of the screening machine is marked as a normal monitoring state; When the posture anomaly identification result indicates that the screening machine has an abnormal posture state, the posture monitoring result of the screening machine is marked as an alarm monitoring state, and corresponding alarm information is generated according to the anomaly type and the anomaly level. The screening machine attitude monitoring results are sent to the PLC or centralized control platform according to a preset communication format, wherein the preset communication format is determined by the data receiving rules of the PLC or centralized control platform. The historical operating data is generated by associating and storing the screening machine posture monitoring results, the multimodal operation data, the standardized posture monitoring dataset, the posture anomaly identification results, and the corresponding acquisition time.

Images