Real-time estimation system and method for cutting load of coal mining machine based on multi-sensor fusion

Abstract

The application discloses a kind of based on multi-sensor fusion's coal winning machine cutting load real-time estimation system and method, system includes cutting state information acquisition module, current processing module, inclination processing module, video processing module and cutting load estimation module.Cutting state information acquisition module includes current sensor, inclination sensor and binocular camera.The method comprises: feature calculation and noise reduction processing are carried out to three-phase current signal;Real-time cutting height and cutting height change rate are calculated;Video data is denoised and coal-rock section geometric feature is extracted;Current, inclination, video three types of feature sequences are time-space aligned and fused calculation, time series modeling is carried out using long short-term memory network with attention mechanism to output cutting load estimation value, and network output layer parameters are updated online based on newly collected data using Adam optimization algorithm.

Description

Technical Field

[0001] This invention relates to the fields of coal mine safety production and intelligent sensing technology, and in particular to a real-time estimation system for coal mining machine cutting load based on multi-sensor fusion and a real-time estimation method for coal mining machine cutting load based on multi-sensor fusion. Background Technology

[0002] The coal mining machine is the core equipment in a fully mechanized coal mining face. Real-time and accurate estimation of its cutting load is crucial for intelligent control, fault warning, and safe production. The cutting load directly reflects the mechanical state of the interaction between the coal mining machine and the coal and rock, and is a key basis for adjusting the speed of the coal mining machine drum, optimizing traction speed, monitoring cutter wear, and implementing overload protection.

[0003] Currently, the methods for estimating the cutting load of coal mining machines mainly include the following categories: (1) Method based on a single current sensor: The cutting load is indirectly reflected by monitoring the operating current of the cutting motor. This method is simple to implement and has a low cost, but it is difficult to distinguish whether the load change is caused by the change in coal and rock hardness or the change in drum posture by relying solely on the current signal. In addition, the current signal is easily affected by power grid fluctuations and changes in motor parameters, resulting in poor estimation accuracy and robustness.

[0004] (2) Methods based on multiple sensors but with a single information dimension: Some existing technologies use acceleration sensors, rotation speed sensors and current sensors in combination, but the information obtained by these sensors is still limited to the operating status of the electromechanical system, lacks direct perception of the geometric features of the coal-rock interface, and cannot make full use of the spatial information of the coal-rock cross section to assist in load estimation.

[0005] (3) Vision-based methods without deep fusion: Some existing technologies use infrared cameras or monocular vision to identify coal and rock or evaluate the cutting effect. However, these methods usually only use visual information for coal and rock classification or state identification, without deeply fusing visual features with time-series features such as current and attitude, and without establishing a quantitative mapping relationship between visual features and cutting load.

[0006] (4) Methods based on shallow neural networks: In existing technologies, shallow models such as BP neural networks, ELM extreme learning machines, and RBF radial basis function networks are mostly used for soft measurement or prediction of cutting load. These models have simple structures and fast training speeds, but they are difficult to capture the temporal dynamic characteristics and long-term dependencies of the cutting process. Moreover, once the model is trained, it becomes fixed and cannot adapt to the dynamic changes in the working conditions of the coal mining machine.

[0007] (5) Lack of online adaptive update mechanism: Most load estimation models in the existing technology are fixed after offline training. When the geological conditions of the coal mining face change or the equipment status drifts, the accuracy of the model will drop significantly, and data needs to be collected again and trained offline, which seriously affects the practicality and reliability of the system.

[0008] In summary, existing technologies suffer from the following main problems: limited sensor information dimensions, insufficient depth of multi-source information fusion, limited temporal modeling capabilities, and a lack of adaptive update capabilities. Therefore, there is an urgent need for a real-time estimation system and method for coal mining machine cutting load that can fuse heterogeneous information from multiple sources, including current, tilt angle, and vision, and possesses temporal dynamic modeling capabilities and online adaptive update capabilities. Summary of the Invention

[0009] This invention aims to at least partially address one of the technical problems in related technologies. To this end, the first objective of this invention is to propose a real-time estimation system for coal mining machine cutting load based on multi-sensor fusion. This system collects multi-source feature information in real time during the cutting process of a 10kV coal mining machine, simultaneously processing cutting current, drum spatial attitude, and visual features of the coal and rock cross-section. By combining dynamic weight fusion, LSTM temporal modeling, and online updates, it achieves real-time and accurate estimation of the coal mining machine cutting load, improving the stability, adaptability, and reliability of cutting load sensing.

[0010] The second objective of this invention is to propose a real-time estimation method for the cutting load of a coal mining machine based on multi-sensor fusion.

[0011] To achieve the above objectives, a first aspect of the present invention proposes a real-time estimation system for coal mining machine cutting load based on multi-sensor fusion, comprising: The cutting status information acquisition module includes a current sensor, a tilt sensor, and a binocular camera. The current sensor is connected to the frequency converter of the cutting motor to acquire the three-phase current signal of the cutting motor. The tilt sensor is installed on the cutting rocker arm of the coal mining machine to acquire the spatial attitude angle signal of the cutting drum. The binocular camera is installed on the body of the coal mining machine, facing the cutting drum, to acquire video data of the coal-rock interface. The current processing module performs feature calculation and noise reduction on the three-phase current signal output by the current sensor, and outputs the cutting current feature sequence; the tilt angle processing module calculates the real-time mining height and mining height change rate of the cutting drum based on the spatial attitude angle signal output by the tilt angle sensor, and outputs the tilt angle feature sequence; the video processing module performs noise reduction and feature extraction on the video data output by the binocular camera, and outputs a visual feature sequence containing the geometric features of the coal and rock cross section. The truncation load estimation module is used to perform spatiotemporal alignment and fusion calculation on the feature sequences output by each processing module, and to perform temporal modeling using a long short-term memory network with attention mechanism to output truncation load estimates; and to update the output layer parameters of the long short-term memory network online using the Adam optimization algorithm based on newly acquired data.

[0012] In addition, the real-time estimation system for coal mining machine cutting load based on multi-sensor fusion according to the above embodiments of the present invention may also have the following additional technical features: According to one embodiment of the present invention, the process of processing the cut-off current in the current processing module is as follows: Current sensor and frequency Obtain the three-phase current signal of the cutting motor , , The current processing module performs a Clarke transform on the three-phase current signal using the following formula to obtain the current components in the two-phase stationary coordinate system. and : ; The magnitude of the instantaneous current composite vector is calculated using the following formula. : ; right Perform a moving average filter with a window length of L, and obtain the denoised cutoff current characteristic sequence using the following formula: }; ; Where i is the sequence number of the corresponding time, k is the summation index variable, and L is the moving average window length.

[0013] According to one embodiment of the present invention, the tilt angle processing process of the rocker arm in the tilt angle processing module is as follows: With frequency Read the output value of the rocker arm tilt sensor at time t The real-time cutting height of the cutting drum center is calculated using the following formula. : ; in, It is the vertical distance from the center of the rocker arm's rotational hinge to the reference surface of the coal seam floor in the roadway where the coal mining machine is located; It is the straight-line distance from the center of the rocker arm's rotational hinge to the center of the cutting drum; It is the installation offset angle; The cutting height variation rate is calculated by differentiating the real-time sampling height at the center of the cutting drum using the following formula. : ; in, It is the sampling time interval of the tilt sensor. ; The output tilt angle feature sequence is { , }

[0014] According to one embodiment of the present invention, the processing procedure for the coal mining machine's cut video information in the video processing module is as follows: (1) Gaussian denoising: for stereo cameras, at frame rate The acquired video stream is frame-by-frame extracted at fixed intervals ΔT to obtain the left image of the synchronized frame. And the right image Using standard deviation as A 3×3 Gaussian kernel is used for convolutional denoising, with the center of the Gaussian kernel at the origin. The coordinates of the left and right images of the synchronized frame are substituted into the following two-dimensional Gaussian function to calculate the weight value at each position in the Gaussian kernel: ; The following formula is used to calculate and output each pixel in each frame of the extracted image using convolution: ; in, It is the pixel value at the corresponding position in the extracted frame image; Is it a Gaussian convolution kernel in The weight value of the location; This yields the denoised left image. And the right image ; (2) Semi-global stereo matching: Semi-global stereo matching is performed on the left and right images of the denoised synchronization frame. The disparity map D(u,v) is calculated by minimizing the energy function E(D). The energy function is: ; in, It represents the raw video pixels of a certain frame in the cropping process, where q is the neighborhood N of pixel p. p The disparity of point q, which is an inner neighbor of another point, is D. q ;C(p,D p ) indicates that the disparity corresponding to pixel p is D p The matching cost function is P1, which represents the penalty factor for smoothness constraints, and P2, which represents the penalty factor for edge constraints; T[·] is an indicator function, which takes the value 1 when the condition is true and 0 otherwise. The disparity map with minimum E(D) is obtained by dynamic programming optimization. ; (3) 3D point cloud reconstruction: based on camera parameters focal length f, baseline B and optical center Convert the disparity map D(u,v) into 3D point cloud coordinates P(X,Y,Z): ; in, , It converts the pixel coordinate system of the disparity map into a coordinate system with the optical center as the origin; (4) Point cloud segmentation of coal and rock cross section: Set the region of interest (ROI) relative to the theoretical cutting surface of the cutting drum, and segment the point cloud cluster Q(t) belonging to the coal and rock cross section from the ROI according to the Random Sampling Consistency (RANSAC) principle. (5) Section feature extraction: Calculate the average height of the point cloud cluster. 3D axial bounding box volume V box Point cloud density The coal drop volume is obtained by multiplying the average projected area of ​​two consecutive frames by the cutting advance distance d. .

[0015] According to one embodiment of the present invention, in semi-global stereo matching, the value range of the smoothness constraint penalty factor P1 is 5≤P1≤20, and the value range of the edge constraint penalty factor P2 is 50≤P2≤200.

[0016] According to one embodiment of the present invention, in the truncated load estimation module, the data fusion and load estimation process is as follows: (1) Calculate the weighted fusion eigenvalues : Establish a unified timestamp for the characteristic sequence of the cutting current. Inclination feature sequence { } and visual feature sequences Perform synchronous interpolation to generate aligned, discretized multi-source feature vectors. ; Right now ; Adaptive weights are assigned to each feature using the following formula, calculated based on the inverse of the variance of each feature within the sliding time window; features with smaller variances have larger weights: ; in, yes The dynamic weight of the m-th feature. ; It is the variance of the m-th feature within a single sliding time window; The weighted fusion eigenvalue is calculated using the following formula. And used as input for the time series model: ; (2) Calculate the hidden state output vector : LSTM networks in processing fused feature sequences During the process, the hidden state output vector generated at each time step is calculated using the following formula. : ; in, It is the output gate vector; It represents the element-wise multiplication of two vectors of the same dimension; It is the cell state vector updated at the current time step; It is an activation function; (3) Calculate the correlation score between the current state and each historical state. : Calculated using the following formula : ; in, It is the hidden state at the current cut time n; It is the hidden state of the truncated historical moment t (t≤n); It is the trained weight matrix, used to... and Projected onto the same feature space; It is the transpose of the trained weight vector, used to compress the projected combined features into a scalar score; (4) Use the Softmax function to normalize the scores of all historical time t according to the following formula to obtain the weight distribution: ; in, This represents the weight of the cutoff information at historical time t in predicting the cutoff load at time n; (5) Using attention weights The context vector for the LSTM attention mechanism is calculated by weighted summation of all historical hidden states using the following formula. : ; (6) Transfer the context vector Mapped to the final cut-off load estimate : ; in, It is the weight matrix of the LSTM output layer; It is the bias term of the output layer; , All of these are learned from data during the model training process.

[0017] According to one embodiment of the present invention, the LSTM model training and online update process in the truncation load estimation module is as follows: (1) Collect labeled historical datasets ,in The actual load value is fed into the constructed LSTM model, and the parameters of each mechanism gate and each layer are automatically calculated from the training set. (2) After the system obtains a new set of real-time data, it first calculates the prediction error of the model on that data and calculates the loss value using the following formula. : ; in, It is the number of new data samples used to update the model; These are the theoretical load calculation values ​​from the calibrated cut physical model; (3) Calculation parameter update: The output layer parameters are respectively calculated using the coefficient formula. and Perform gradient calculation: ; ; in, It is the loss function applied to the output layer weight matrix. The gradient matrix; It is the loss function with respect to the output layer bias vector The gradient; It is the first The context vector obtained after a sample passes through the attention layer; (4) Based on the calculated gradient values, the LSTM model parameters are iteratively updated using the Adam optimization algorithm, as shown in the following formula: ; ; ; ; ; in, Represents updatable training parameters ; The first moment estimate of the gradient at time step The updated value, For second-order moment estimation at time step The updated value is the internal state maintained by the Adam algorithm; It is the exponential decay rate estimated by the matrix, taking... ; Yes Deviation correction, Yes Deviation correction is used to compensate for the initial time step; It is the base learning rate, taken as... ; It is the equilibrium constant, take ; These are the updated parameter values.

[0018] To achieve the above objectives, a second aspect of the present invention proposes a real-time estimation method for coal mining machine cutting load based on multi-sensor fusion. The real-time estimation system for coal mining machine cutting load based on multi-sensor fusion, as described in the above embodiment, includes the following steps: Acquire three-phase current signals, rocker arm tilt angle signals, and video data of the coal-rock interface during the coal cutting process of the coal mining machine; The three-phase current signal is subjected to feature calculation and noise reduction to obtain the cut-off current feature sequence; The real-time mining height and mining height change rate of the cutting drum are calculated based on the rocker arm tilt angle signal to obtain the tilt angle characteristic sequence; Denoising and feature extraction were performed on the video data to obtain a visual feature sequence containing the geometric features of the coal and rock cross section; Spatiotemporal alignment and fusion calculations are performed on each feature sequence, and a long short-term memory network with attention mechanism is used for temporal modeling to output the truncation load estimate in real time. The output layer parameters of the Long Short-Term Memory network are updated online using the Adam optimization algorithm based on the newly acquired data.

[0019] Compared with the prior art, the present invention has the following significant advantages: (1) Deep fusion of multi-source information leads to more comprehensive and accurate load estimation. This invention is the first to deeply fuse three heterogeneous information sources—three-phase current, rocker arm tilt angle, and binocular vision—for cutting load estimation. Current characteristics reflect the electromagnetic load state of the motor, tilt angle characteristics reflect the spatial attitude change of the drum, and visual characteristics reflect the geometric features of the coal and rock cross-section. These three factors characterize the cutting conditions from different dimensions, and adaptive weighted fusion achieves information complementarity, significantly improving the accuracy and robustness of load estimation. Experiments show that compared with the single current sensor method, the root mean square error of load estimation in this invention is reduced by approximately 35% to 45%.

[0020] (2) Adaptive weighted fusion improves system reliability. This invention adopts an adaptive weighted fusion strategy based on the inverse variance, which dynamically adjusts the weights of each feature according to their stability within the sliding time window. Features with smaller variance (more stable and reliable) automatically receive greater weights. When a sensor malfunctions or causes data anomalies due to environmental interference, its weight is automatically reduced, thereby effectively suppressing the impact of abnormal data on the estimation results and improving the system's reliability under complex operating conditions.

[0021] (3) Attention Mechanism LSTM Improves Temporal Modeling Accuracy. This invention uses a Long Short-Term Memory network with an attention mechanism to perform temporal modeling of the truncation process. The LSTM network can effectively capture the long-term dependencies of the truncation process, while the attention mechanism enables the model to dynamically pay attention to information at key historical truncation moments (such as when encountering geological anomalies such as rock inclusions or faults). Compared with ordinary LSTM and BP neural networks, the estimation accuracy is improved by about 15% to 25%.

[0022] (4) Online update mechanism improves model adaptability. This invention uses the Adam optimization algorithm to update the output layer parameters of the LSTM network online, enabling the model to continuously optimize based on newly acquired real-time data. When the coal seam hardness changes, the cutting teeth wear out, or the equipment parameters drift, the model can adaptively adjust to maintain long-term estimation accuracy, solving the problem of accuracy decay caused by fixed models in the prior art.

[0023] (5) Specific optimization for 10kV coal mining machines. This invention targets the electrical characteristics of the cutting motor of a 10kV high-voltage coal mining machine. It uses Clarke transform to extract the composite vector amplitude of the three-phase current, which can more accurately reflect the electromagnetic torque state of the motor. Compared with simple calculation of the effective value of the current, it has better adaptability to the nonlinear characteristics of the 10kV frequency conversion drive system.

[0024] (6) Robustness of visual processing. In view of the harsh environment of high dust, low light and strong vibration in underground mine, this invention adopts robust algorithms such as Gaussian denoising, semi-global stereo matching and RANSAC point cloud segmentation, and sets the ROI area relative to the theoretical cutting surface of the cutting drum, which effectively eliminates background interference and improves the reliability of coal and rock cross-section feature extraction.

[0025] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of a real-time estimation system for coal mining machine cutting load based on multi-sensor fusion according to an embodiment of the present invention. Detailed Implementation

[0027] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0028] The following description, with reference to the accompanying drawings, describes the real-time estimation system and method for coal mining machine cutting load based on multi-sensor fusion proposed in the embodiments of the present invention.

[0029] like Figure 1 As shown, the real-time estimation system for coal mining machine cutting load based on multi-sensor fusion according to an embodiment of the present invention includes: a cutting status information acquisition module 10, a current processing module 20, an inclination angle processing module 30, a video processing module 40, and a cutting load estimation module 50.

[0030] The cutting status information acquisition module 10 includes a current sensor 11, a tilt sensor 12, and a binocular camera 13. The current sensor 11 is connected to the frequency converter of the cutting motor and is used to acquire the three-phase current signal of the cutting motor. The tilt sensor 12 is installed on the cutting arm of the coal mining machine and is used to acquire the spatial attitude angle signal of the cutting drum. The binocular camera 13 is installed on the coal mining machine body, facing the cutting drum, and is used to acquire video data of the coal-rock interface. The current processing module 20 performs feature calculation and noise reduction processing on the three-phase current signal output by the current sensor 11 and outputs a cutting current feature sequence. The tilt processing module 30 calculates the real-time mining height and mining height change rate of the cutting drum based on the spatial attitude angle signal output by the tilt sensor 12 and outputs a tilt feature sequence. The video processing module 40 performs noise reduction and feature extraction on the video data output by the binocular camera 13 and outputs a visual feature sequence containing the geometric features of the coal-rock cross-section. The truncation load estimation module 50 is used to perform spatiotemporal alignment and fusion calculation on the feature sequences output by each processing module, and to perform temporal modeling using a long short-term memory network with attention mechanism to output truncation load estimates; and to update the output layer parameters of the long short-term memory network online based on newly acquired data using the Adam optimization algorithm.

[0031] Specifically, the real-time estimation system for coal mining machine cutting load based on multi-sensor fusion in this embodiment of the invention adopts a modular architecture design. The modules are connected through data interfaces to realize a complete process of information acquisition, processing, fusion, and estimation. The cutting status information acquisition module 10 is responsible for acquiring raw sensor data from multiple dimensions of the coal mining machine cutting system; the current processing module 20, tilt angle processing module 30, and video processing module 40 respectively preprocess and extract features from the raw data of their respective sensors; the cutting load estimation module 50 receives the feature sequences output by the three processing modules, performs spatiotemporal alignment, adaptive weighted fusion, temporal modeling, and load estimation, and outputs the final estimated cutting load value.

[0032] The cutting status information acquisition module 10 includes three types of sensors: a current sensor 11, a tilt sensor 12, and a binocular camera 13. These three sensors perceive the cutting status from different dimensions: the current sensor 11 perceives the motor load status from an electrical perspective, the tilt sensor 12 perceives the drum's spatial attitude from a mechanical perspective, and the binocular camera 13 perceives the geometric characteristics of the coal and rock cross-section from a visual perspective. The information from these three sensors complements each other, together forming a complete information foundation for cutting load estimation.

[0033] Specifically, the workflow of the real-time estimation system for coal mining machine cutting load based on multi-sensor fusion according to embodiments of the present invention includes the following steps: Step S101: System power-on initialization. Each module loads configuration parameters, establishes data connections, and initializes LSTM network model parameters.

[0034] Step S102: Multi-source information acquisition. Current sensor 11 acquires three-phase current signals from the inverter of the cutting motor at frequency fa; tilt sensor 12 acquires spatial attitude angle signals from the cutting rocker arm at frequency fb; binocular camera 13 acquires video data of the cutting coal-rock interface at frame rate fc.

[0035] Step S103: Parallel feature processing. The current processing module 20 performs Clarke transform, synthetic vector amplitude calculation, and moving average filtering on the three-phase current signal, outputting a truncated current feature sequence; the tilt angle processing module 30 performs real-time height calculation and height change rate difference on the tilt angle signal, outputting a tilt angle feature sequence; the video processing module 40 performs Gaussian denoising, semi-global stereo matching, 3D point cloud reconstruction, and cross-sectional feature extraction on the video data, outputting a visual feature sequence.

[0036] Step S104: Multi-source feature fusion and load estimation. The truncated load estimation module 50 establishes a unified timestamp, performs time synchronization and interpolation alignment on the three feature sequences, and generates multi-source feature vectors; it calculates adaptive weights based on the inverse of variance, performs weighted fusion to obtain fused feature values; it inputs the fused feature values ​​into an LSTM network with an attention mechanism, and outputs truncated load estimates.

[0037] Step S105: Online Model Update. Based on the newly acquired real-time data, calculate the prediction error and loss function, and use the Adam optimization algorithm to update the weight matrix and bias terms of the LSTM network output layer online.

[0038] Step S106: Execute repeatedly. Return to step S102 and continue to estimate the truncated load in real time.

[0039] According to one embodiment of the present invention, the process of processing the cut-off current in the current processing module 20 is as follows: Current sensor 11 and frequency Obtain the three-phase current signal of the cutting motor , , The current processing module 20 performs a Clarke transform on the three-phase current signal using the following formula to obtain the current components in the two-phase stationary coordinate system. and : .

[0040] Specifically, the cutting motor of the 10kV coal mining machine adopts a three-phase AC variable frequency drive, and its three-phase current signal , , The original three-phase current signal contains information about the electromagnetic load state of the motor. However, the original three-phase current signal suffers from problems such as three-phase coupling, high-frequency noise, and power grid fluctuation interference. The Clarke transform converts the three-phase current to a two-phase stationary coordinate system, eliminating the coupling relationship between the three phases. The extracted composite vector amplitude has an approximately linear relationship with the motor's electromagnetic torque, which can more accurately reflect the cut-off load. Moving average filtering further removes high-frequency noise, resulting in stable and reliable cut-off current characteristics.

[0041] It should be noted that the physical meaning of the Clark transform is to convert the three scalars in a three-phase equilibrium system into two orthogonal complex plane components. and .in Coaxial with phase a current, Advanced 90°. This transformation eliminates the coupling between the three phases, making subsequent feature extraction more direct.

[0042] Furthermore, the magnitude of the instantaneous current composite vector is calculated using the following formula. : .

[0043] Specifically, the magnitude of the synthesized vector is proportional to the electromagnetic torque of the motor. For a 10kV variable frequency drive system using SVPWM control, the magnitude of the synthesized vector directly reflects the electromagnetic load state of the motor, and compared with simple calculation of the effective value of the current, it has better robustness to three-phase imbalance and phase shift.

[0044] Furthermore, on Perform a moving average filter with a window length of L, and obtain the denoised cutoff current characteristic sequence using the following formula: }; ; Where i is the sequence number of the corresponding time, k is the summation index variable, and L is the moving average window length, which can be 5≤L≤50.

[0045] Specifically, the principle of moving average filtering is to take the arithmetic mean of the composite vector amplitude at the current time and the previous L-1 time steps, so that high-frequency noise cancels each other out during the averaging process, while preserving the slow changing trend of the truncated load. The choice of window length L needs to be balanced between response speed and filtering effect: if L is too small (e.g., L<5), the filtering effect will be poor and more high-frequency noise will remain; if L is too large (e.g., L>50), the response delay will increase, and it will be unable to track rapid changes in the load in a timely manner. This invention limits 5≤L≤50, which can achieve fast response while ensuring filtering effect.

[0046] According to one embodiment of the present invention, the tilt angle processing process of the rocker arm in the tilt angle processing module 30 is as follows: With frequency Read the output value of rocker arm tilt sensor 12 at time t The real-time cutting height of the cutting drum center is calculated using the following formula. : ; in, It is the vertical distance from the center of the rocker arm's rotational hinge to the reference surface of the coal seam floor in the roadway where the coal mining machine is located; It is the straight-line distance from the center of the rocker arm's rotational hinge to the center of the cutting drum; It is the installation offset angle, used to compensate for mechanical deviations during the installation of tilt sensor 12; The cutting height variation rate is calculated by differentiating the real-time sampling height at the center of the cutting drum using the following formula. : ; in, This is the 12-sampling time interval of the tilt sensor. ; The output tilt angle feature sequence is { ,}

[0047] Specifically, the spatial orientation of the cutting drum of the coal mining machine directly affects the cutting load. The drum's cutting height (i.e., the cutting height) determines the vertical range of the cutting section; the higher the cutting height, the larger the cutting section and the greater the load. The rate of change of cutting height reflects the drum's lifting speed and is closely related to the dynamic impact of the cutting process. The tilt sensor 12 is installed on the cutting rocker arm to measure the tilt angle of the rocker arm relative to the horizontal reference plane. The real-time cutting height at the center of the drum can be calculated using trigonometric relationships.

[0048] Specifically, the tilt sensor 12 operates at a frequency f b The output value θ(t) of the rocker arm tilt sensor 12 at time t is read at 50Hz. In this embodiment, the tilt sensor 12 can be a high-precision MEMS tilt sensor 12 with a measurement range of ±90°, an accuracy of 0.01°, and a resolution of 0.001°, which can meet the accuracy requirements of height calculation. The tilt sensor 12 is installed on the rocker arm near the center of the rotation hinge, and a rigid connection ensures that the measured value accurately reflects the rocker arm attitude.

[0049] The geometric principle behind the formula for calculating the real-time extraction height of the cutting drum center is as follows: The rocker arm rotates around the pivot hinge center O, and the distance from the drum center P to O is L. arm The angle between OP and the horizontal plane is θ(t) + θ'. The projection distance of the drum center P in the vertical direction is the mining height H(i). When the rocker arm is horizontal (θ(t) + θ' = 0°), sin(0°) = 0, and the mining height H(i) = H0; when the rocker arm swings upward, the sin value is positive, and the mining height increases; when the rocker arm swings downward, the sin value is negative, and the mining height decreases.

[0050] Furthermore, the rate of change V is increased. h (i) means: when V h When (i)>0, the roller rises, the cutting cross-section decreases, and the load tends to decrease; when V h (i) When < 0, the roller descends, the cutting cross-section increases, and the load tends to increase; |V h (i) When the value is large, it indicates that the drum is moving up and down rapidly, the relative speed between the cutting teeth and the coal and rock increases, and dynamic impact load may be generated.

[0051] According to one embodiment of the present invention, the processing procedure for the coal mining machine's cut video information in the video processing module 40 is as follows: (1) Gaussian denoising: for the stereo camera 13 at frame rate The acquired video stream is frame-by-frame extracted at fixed intervals ΔT to obtain the left image of the synchronized frame. And the right image Using standard deviation as A 3×3 Gaussian kernel is used for convolutional denoising, with the center of the Gaussian kernel at the origin. The coordinates of the left and right images of the synchronized frame are substituted into the following two-dimensional Gaussian function to calculate the weight value at each position in the Gaussian kernel: ; The following formula is used to calculate and output each pixel in each frame of the extracted image using convolution: ; in, It is the pixel value at the corresponding position in the extracted frame image; Is it a Gaussian convolution kernel in The weight value of the location; This yields the denoised left image. And the right image .

[0052] It's important to note that Gaussian denoising works by using a two-dimensional Gaussian function as the convolution kernel to perform a weighted average on each pixel and its neighborhood in the image. The weights are determined by the Gaussian function, with the central pixel having the highest weight and pixels further away from the center having progressively lower weights. This weighted averaging effectively suppresses random noise while preserving the image's edge features. The choice of a standard deviation σ = 1.0 balances denoising effectiveness and edge preservation: too small a σ results in poor denoising, while too large a σ leads to blurred edges, affecting the accuracy of subsequent stereo matching.

[0053] (2) Semi-global stereo matching: Semi-global stereo matching is performed on the left and right images of the denoised synchronization frame. The disparity map D(u,v) is calculated by minimizing the energy function E(D). The energy function is: ; in, It represents the raw video pixels of a certain frame in the cropping process, where q is the neighborhood N of pixel p. p The disparity of point q, which is an inner neighbor of another point, is D. q ;C(p,D p ) indicates that the disparity corresponding to pixel p is D p The matching cost function is P1, which represents the penalty factor for smoothness constraints, and P2, which represents the penalty factor for edge constraints; T[·] is an indicator function, which takes the value 1 when the condition is true and 0 otherwise. The disparity map with minimum E(D) is obtained by dynamic programming optimization. .

[0054] According to one embodiment of the present invention, in semi-global stereo matching, the range of the smoothness constraint penalty factor P1 is 5≤P1≤20, and the range of the edge constraint penalty factor P2 is 50≤P2≤200.

[0055] It's important to note that the core idea of ​​the semi-global stereo matching (SGM) algorithm is to solve a one-dimensional energy minimization problem using dynamic programming in multiple directions (typically 8 or 16 directions), and then aggregate the results from each direction to obtain a globally optimal disparity map. Compared to local matching algorithms, SGM possesses global optimality and can obtain more accurate disparity estimates in regions with missing or repeated textures, making it particularly suitable for scenarios with complex and uneven textures, such as coal-rock interfaces.

[0056] (3) 3D point cloud reconstruction: based on camera parameters focal length f, baseline B and optical center Convert the disparity map D(u,v) into 3D point cloud coordinates P(X,Y,Z): ; in, , It converts the pixel coordinate system of the disparity map into a coordinate system with the optical center as the origin.

[0057] (4) Point cloud segmentation of coal and rock cross section: Set the region of interest (ROI) relative to the theoretical cutting surface of the cutting drum, and segment the point cloud cluster Q(t) belonging to the coal and rock cross section from the ROI according to the Random Sampling Consistency (RANSAC) principle.

[0058] It should be noted that the principle of RANSAC segmentation is that the coal and rock cross-section approximates a plane (or a gently curved surface), while the background point cloud (such as hydraulic supports, other areas of the coal wall) does not conform to this planar model. The RANSAC algorithm iteratively samples three points randomly to fit a plane, counts the number of interior points, and finally selects the planar model with the most interior points. The point cloud that conforms to this model is taken as the coal and rock cross-section point cloud cluster Q(t). This segmentation method is robust to outliers and can effectively eliminate interference from complex underground backgrounds.

[0059] (5) Section feature extraction: Calculate the average height of the point cloud cluster. 3D axial bounding box volume V box Point cloud density The coal drop volume is obtained by multiplying the average projected area of ​​two consecutive frames by the cutting advance distance d. .

[0060] Specifically, a Region of Interest (ROI) is defined relative to the theoretical cutting surface of the cutting drum. Based on the principle of random sampling consistency, point cloud clusters Q(t) belonging to the coal and rock cross-section are segmented from this ROI, and the cross-sectional point cloud features are calculated: The average height is calculated using the following formula. : ; in, It is the number of points in the point cloud cluster; The three-dimensional axial bounding box volume V of the point cloud cluster is calculated using the following formula. box The bounding box is determined by the minimum and maximum values ​​of the point cloud on the three coordinate axes. Let the minimum value of the point cloud on the X-axis be X. min The maximum value is X max Similarly, Y min and Y max Z min and Z max : ; Point cloud density is calculated using the following formula. : ; Project each point in the point cloud clusters Q(t) and Q(t+1) of two consecutive frames onto the YZ plane, calculate the corresponding frame projection areas S(t) and S(t+1), and let the truncation and advance distance be d. Then the average cross-sectional area of ​​the two consecutive frames is... ; The volume of coal dropped is calculated using the following formula. : ; The output visual feature sequence is .

[0061] According to one embodiment of the present invention, in the truncated load estimation module 50, the data fusion and load estimation process is as follows: (1) Calculate the weighted fusion eigenvalues : Establish a unified timestamp for the characteristic sequence of the cutting current. Inclination feature sequence { } and visual feature sequences Perform synchronous interpolation to generate aligned, discretized multi-source feature vectors. ; Right now ; Adaptive weights are assigned to each feature using the following formula, calculated based on the inverse of the variance of each feature within the sliding time window; features with smaller variances have larger weights: ; in, yes The dynamic weight of the m-th feature. ; It is the variance of the m-th feature within a single sliding time window; The weighted fusion eigenvalue is calculated using the following formula. And used as input for the time series model: ; (2) Calculate the hidden state output vector : LSTM networks in processing fused feature sequences During the process, the hidden state output vector generated at each time step is calculated using the following formula. : ; in, It is the output gate vector; It represents the element-wise multiplication of two vectors of the same dimension; It is the cell state vector updated at the current time step; It is an activation function; (3) Calculate the correlation score between the current state and each historical state. : Calculated using the following formula : ; in, It is the hidden state at the current cut time n; It is the hidden state of the truncated historical moment t (t≤n); It is the trained weight matrix, used to... and Projected onto the same feature space; It is the transpose of the trained weight vector, used to compress the projected combined features into a scalar score; (4) Use the Softmax function to normalize the scores of all historical time t according to the following formula to obtain the weight distribution: ; in, This represents the weight of the cutoff information at historical time t in predicting the cutoff load at time n; (5) Using attention weights The context vector for the LSTM attention mechanism is calculated by weighted summation of all historical hidden states using the following formula. : ; (6) Transfer the context vector Mapped to the final cut-off load estimate : ; in, It is the weight matrix of the LSTM output layer; It is the bias term of the output layer; , All of these are learned from data during the model training process.

[0062] Therefore, the adaptive weighted fusion in this embodiment dynamically adjusts the weights based on the real-time stability of each feature, giving stable and reliable features a greater influence and improving the accuracy of fusion. When a sensor malfunctions or data is abnormal, the variance of the corresponding feature increases, and the weight automatically decreases. The system can automatically "mask" abnormal data, improving fault tolerance. The calculation of the inverse variance is simple and efficient, meeting real-time requirements. Normalization constraints ensure the numerical stability of the fusion results. The LSTM network in this embodiment can effectively capture the long-term dependencies of the cutting process, solving the problem that shallow models such as BP networks cannot model temporal dynamic characteristics. Secondly, the attention mechanism enables the model to dynamically focus on historical key cutting moments (such as when encountering geological anomalies such as interbedded rock, faults, and coal wall undulations), improving the model's adaptability to complex geological conditions. Thirdly, the context vector cn integrates the weighted aggregation of historical information, providing more comprehensive information and higher estimation accuracy compared to using only the last hidden state hn. Fourthly, the visualization of attention weights helps operators understand the model's decision-making basis, improving the system's interpretability.

[0063] According to one embodiment of the present invention, the LSTM model training and online update process in the truncated load estimation module 50 is as follows: (1) Collect labeled historical datasets ,in The actual load value is fed into the constructed LSTM model, and the parameters of each mechanism gate and each layer are automatically calculated from the training set. (2) After the system obtains a new set of real-time data, it first calculates the prediction error of the model on that data and calculates the loss value using the following formula. : ; in, It is the number of new data samples used to update the model; These are the theoretical load calculation values ​​from the calibrated cut physical model; (3) Calculation parameter update: The output layer parameters are respectively calculated using the coefficient formula. and Perform gradient calculation: ; ; in, It is the loss function applied to the output layer weight matrix. The gradient matrix; It is the loss function with respect to the output layer bias vector The gradient; It is the first The context vector obtained after a sample passes through the attention layer; (4) Based on the calculated gradient values, the LSTM model parameters are iteratively updated using the Adam optimization algorithm, as shown in the following formula: ; ; ; ; ; in, Represents updatable training parameters ; The first moment estimate of the gradient at time step The updated value, For second-order moment estimation at time step The updated value is the internal state maintained by the Adam algorithm; It is the exponential decay rate estimated by the matrix, taking... ; Yes Deviation correction, Yes Deviation correction is used to compensate for the initial time step; It is the base learning rate, taken as... ; It is the equilibrium constant, take ; These are the updated parameter values.

[0064] Therefore, the online model update in this embodiment enables the model to adapt to changes in coal seam hardness, cutter wear, equipment aging, and other operating conditions, maintaining long-term estimation accuracy. The adaptive learning rate characteristic of the Adam optimization algorithm makes the update process adaptive to parameters at different gradient scales and converges stably. The "partial update" strategy of updating only the output layer parameters ensures adaptability while avoiding catastrophic forgetting and retaining the general temporal features learned by the LSTM hidden layers. The online update has low computational cost, meets real-time requirements, and the time cost of one update is less than 10ms, which does not affect the real-time operation of the system.

[0065] Corresponding to the above embodiments, the present invention also proposes a real-time estimation method for coal mining machine cutting load based on multi-sensor fusion.

[0066] The real-time estimation method for coal mining machine cutting load based on multi-sensor fusion in this invention adopts the real-time estimation system for coal mining machine cutting load based on multi-sensor fusion described in the above embodiment, and includes the following steps: Acquire three-phase current signals, rocker arm tilt angle signals, and video data of the coal-rock interface during the coal cutting process; The three-phase current signal is subjected to feature calculation and noise reduction to obtain the cut-off current feature sequence; The real-time mining height and mining height change rate of the cutting drum are calculated based on the rocker arm tilt angle signal to obtain the tilt angle characteristic sequence; Denoising and feature extraction were performed on the video data to obtain a visual feature sequence containing the geometric features of the coal and rock cross section; Spatiotemporal alignment and fusion calculations are performed on each feature sequence, and a long short-term memory network with attention mechanism is used for temporal modeling to output the truncation load estimate in real time. The output layer parameters of the Long Short-Term Memory network are updated online using the Adam optimization algorithm based on the newly acquired data.

[0067] It should be noted that for details not disclosed in the real-time estimation method for coal mining machine cutting load based on multi-sensor fusion in the embodiments of the present invention, please refer to the details disclosed in the real-time estimation system for coal mining machine cutting load based on multi-sensor fusion in the embodiments of the present invention, which will not be repeated here.

[0068] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0069] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0070] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0071] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A real-time estimation system for coal mining machine cutting load based on multi-sensor fusion, characterized in that, include: The cutting status information acquisition module includes a current sensor, a tilt sensor, and a binocular camera; the current sensor is connected to the frequency converter of the cutting motor and is used to acquire the three-phase current signal of the cutting motor; the tilt sensor is installed on the cutting rocker arm of the coal mining machine and is used to acquire the spatial attitude angle signal of the cutting drum; the binocular camera is installed on the body of the coal mining machine, facing the cutting drum, and is used to acquire video data of the coal-rock interface. The current processing module performs feature calculation and noise reduction on the three-phase current signal output by the current sensor, and outputs the cutting current feature sequence; the tilt angle processing module calculates the real-time mining height and mining height change rate of the cutting drum based on the spatial attitude angle signal output by the tilt angle sensor, and outputs the tilt angle feature sequence; the video processing module performs noise reduction and feature extraction on the video data output by the binocular camera, and outputs a visual feature sequence containing the geometric features of the coal and rock cross section. The truncation load estimation module is used to perform spatiotemporal alignment and fusion calculation on the feature sequences output by each processing module, and to perform temporal modeling using a long short-term memory network with attention mechanism to output truncation load estimates; and to update the output layer parameters of the long short-term memory network online using the Adam optimization algorithm based on newly acquired data.

2. The real-time estimation system for coal mining machine cutting load based on multi-sensor fusion according to claim 1, characterized in that, In the current processing module, the process for processing the cut-off current is as follows: The current sensor is connected to a frequency Obtain the three-phase current signal of the cutting motor , , The current processing module performs a Clarke transform on the three-phase current signal using the following formula to obtain the current components in the two-phase stationary coordinate system. and : ； The magnitude of the instantaneous current composite vector is calculated using the following formula. : ； right Perform a moving average filter with a window length of L, and obtain the denoised cutoff current characteristic sequence using the following formula: }; ； Where i is the sequence number of the corresponding time, k is the summation index variable, and L is the moving average window length.

3. The real-time estimation system for coal mining machine cutting load based on multi-sensor fusion according to claim 1, characterized in that, In the tilt angle processing module, the rocker arm tilt angle processing process is as follows: With frequency Read the output value of the rocker arm tilt sensor at time t The real-time cutting height of the cutting drum center is calculated using the following formula. : ； in, It is the vertical distance from the center of the rocker arm's rotational hinge to the reference surface of the coal seam floor in the roadway where the coal mining machine is located; It is the straight-line distance from the center of the rocker arm's rotational hinge to the center of the cutting drum; It is the installation offset angle; The cutting height variation rate is calculated by differentiating the real-time sampling height at the center of the cutting drum using the following formula. : ； in, It is the sampling time interval of the tilt sensor. ; The output tilt angle feature sequence is { , } 4. The real-time estimation system for coal mining machine cutting load based on multi-sensor fusion according to claim 1, characterized in that, In the video processing module, the processing procedure for the coal mining machine's cutting video information is as follows: (1) Gaussian denoising: The binocular camera is denoised at a frame rate The acquired video stream is frame-by-frame extracted at fixed intervals ΔT to obtain the left image of the synchronized frame. And the right image Using standard deviation as A 3×3 Gaussian kernel is used for convolutional denoising, with the center of the Gaussian kernel at the origin. The coordinates of the left and right images of the synchronized frame are substituted into the following two-dimensional Gaussian function to calculate the weight value at each position in the Gaussian kernel: ； The following formula is used to calculate and output each pixel in each frame of the extracted image using convolution: ； in, It is the pixel value at the corresponding position in the extracted frame image; Is it a Gaussian convolution kernel in The weight value of the location; This yields the denoised left image. And the right image ; (2) Semi-global stereo matching: Semi-global stereo matching is performed on the left and right images of the denoised synchronization frame. The disparity map D(u,v) is calculated by minimizing the energy function E(D), where the energy function is: ； in, It represents the raw video pixels of a certain frame in the cropping process, where q is the neighborhood N of pixel p. p The disparity of point q, which is an inner neighbor of another point, is D. q ;C(p,D p ) indicates that the disparity corresponding to pixel p is D p The matching cost function is P1, which represents the penalty factor for smoothness constraints, and P2, which represents the penalty factor for edge constraints; T[·] is an indicator function, which takes the value 1 when the condition is true and 0 otherwise. The disparity map with minimum E(D) is obtained by dynamic programming optimization. ; (3) 3D point cloud reconstruction: based on camera parameters focal length f, baseline B and optical center The disparity map D(u,v) is converted into three-dimensional point cloud coordinates P(X,Y,Z): ； in, , It converts the pixel coordinate system of the disparity map into a coordinate system with the optical center as the origin; (4) Point cloud segmentation of coal and rock cross section: Set the region of interest (ROI) relative to the theoretical cutting surface of the cutting drum, and segment the point cloud cluster Q(t) belonging to the coal and rock cross section from the ROI according to the Random Sampling Consistency (RANSAC) principle. (5) Cross-sectional feature extraction: Calculate the average height of the point cloud cluster. 3D axial bounding box volume V box Point cloud density The coal drop volume is obtained by multiplying the average projected area of ​​two consecutive frames by the cutting advance distance d. .

5. The real-time estimation system for coal mining machine cutting load based on multi-sensor fusion according to claim 4, characterized in that, In the semi-global stereo matching, the smoothness constraint penalty factor P1 has a value range of 5 ≤ P1 ≤ 20, and the edge constraint penalty factor P2 has a value range of 50 ≤ P2 ≤ 200.

6. The real-time estimation system for coal mining machine cutting load based on multi-sensor fusion according to claim 4, characterized in that, In the truncated load estimation module, the data fusion and load estimation process is as follows: (1) Calculate the weighted fusion eigenvalues : Establish a unified timestamp for the characteristic sequence of the cutting current. Inclination feature sequence { } and visual feature sequences Perform synchronous interpolation to generate aligned, discretized multi-source feature vectors. ; Right now ; Adaptive weights are assigned to each feature using the following formula, calculated based on the inverse of the variance of each feature within the sliding time window; features with smaller variances have larger weights: ； in, yes The dynamic weight of the m-th feature. ; It is the variance of the m-th feature within a single sliding time window; The weighted fusion eigenvalue is calculated using the following formula. And used as input for the time series model: ； (2) Calculate the hidden state output vector : LSTM networks in processing fused feature sequences During the process, the hidden state output vector generated at each time step is calculated using the following formula. : ； in, It is the output gate vector; It represents the element-wise multiplication of two vectors of the same dimension; It is the cell state vector updated at the current time step; It is an activation function; (3) Calculate the correlation score between the current state and each historical state. : Calculated using the following formula : ； in, It is the hidden state at the current cut time n; It is the hidden state of the truncated historical moment t (t≤n); It is the trained weight matrix, used to... and Projected onto the same feature space; It is the transpose of the trained weight vector, used to compress the projected combined features into a scalar score; (4) Use the Softmax function to normalize the scores of all historical time t according to the following formula to obtain the weight distribution: ； in, This represents the weight of the cutoff information at historical time t in predicting the cutoff load at time n; (5) Using attention weights The context vector for the LSTM attention mechanism is calculated by weighted summation of all historical hidden states using the following formula. : ； (6) Transfer the context vector Mapped to the final cut-off load estimate : ； in, It is the weight matrix of the LSTM output layer; It is the bias term of the output layer; , All of these are learned from data during the model training process.

7. The real-time estimation system for coal mining machine cutting load based on multi-sensor fusion according to claim 6, characterized in that, In the truncated load estimation module, the LSTM model training and online update process is as follows: (1) Collect labeled historical datasets ,in The actual load value is fed into the constructed LSTM model, and the parameters of each mechanism gate and each layer are automatically calculated from the training set. (2) After the system obtains a new set of real-time data, it first calculates the prediction error of the model on that data and calculates the loss value using the following formula. : ； in, It is the number of new data samples used to update the model; These are the theoretical load calculation values ​​from the calibrated cut physical model; (3) Calculation parameter update: The output layer parameters are respectively calculated using the coefficient formula. and Perform gradient calculation: ； ； in, It is the loss function applied to the output layer weight matrix. The gradient matrix; It is the loss function with respect to the output layer bias vector The gradient; It is the first The context vector obtained after a sample passes through the attention layer; (4) Based on the calculated gradient values, the LSTM model parameters are iteratively updated using the Adam optimization algorithm, as shown in the following formula: ； ； ； ； ； in, Represents updatable training parameters ; The first moment estimate of the gradient at time step The updated value, For second-order moment estimation at time step The updated value is the internal state maintained by the Adam algorithm; It is the exponential decay rate estimated by the matrix, taking... ; Yes Deviation correction, Yes Deviation correction is used to compensate for the initial time step; It is the base learning rate, taken as... ; It is the equilibrium constant, take ; These are the updated parameter values.

8. A method for real-time estimation of cutting load of a coal mining machine based on multi-sensor fusion, characterized in that, The real-time estimation system for coal mining machine cutting load based on multi-sensor fusion as described in any one of claims 1 to 7 includes the following steps: Acquire three-phase current signals, rocker arm tilt angle signals, and video data of the coal-rock interface during the coal cutting process; The three-phase current signals are subjected to feature calculation and noise reduction processing to obtain the cut-off current feature sequence; The real-time mining height and mining height change rate of the cutting drum are calculated based on the rocker arm tilt angle signal to obtain the tilt angle characteristic sequence; The video data is denoised and features are extracted to obtain a visual feature sequence containing the geometric features of the coal and rock cross section; Spatiotemporal alignment and fusion calculations are performed on each feature sequence, and a long short-term memory network with attention mechanism is used for temporal modeling to output the truncation load estimate in real time. The output layer parameters of the Long Short-Term Memory network are updated online using the Adam optimization algorithm based on the newly acquired data.

Images