Multi-mode monitoring and anomaly detection method, system and equipment of belt conveying system and medium

Abstract

The invention discloses a multi-modal monitoring and anomaly detection method, system and device for a belt conveying system and a medium, and belongs to the technical field of anomaly detection.The method comprises the steps that a plurality of multi-modal sensors are arranged on the whole path along the path of the belt conveying system, and original signals in the running process of the belt conveying system are collected; the original signals are preprocessed; performing environment compensation calculation to generate engineering data; mapping the engineering data to the same space coordinate system of the belt conveying system, and completing feature extraction to obtain feature vectors; multi-modal fusion analysis is carried out on the feature vectors, abnormal state identification and fault level classification are realized, and an early warning signal and a risk level are output; and executing a matched response control strategy according to the early warning signal, the risk level and the corresponding space coordinate system position. According to the method, the abnormal state can be dynamically recognized under the complex working condition, differential response control is achieved, and the recognition capability and processing precision of early-stage, multi-source and dynamic anomalies are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of anomaly detection, and in particular to a multimodal monitoring and anomaly detection method, system, equipment and medium for a belt conveyor system. Background Art

[0002] Belt conveyor systems are key equipment for material transportation in coal mine production. They are widely used for the continuous conveying of bulk materials such as coal and ore, and feature long conveying distances, large transport volumes, and a high degree of automation. The system primarily consists of conveyor belts, rollers, a drive mechanism, a tensioning device, a cleaning mechanism, and protective devices. A motor drives the rollers, driving the belts to continuously transport materials from their starting point to their destination. This type of system is not only suitable for mining scenarios but is also commonly used in complex environments such as docks and factories where efficient material handling is required. It is a crucial component in ensuring efficient production operations in coal mines.

[0003] However, the existing coal mine belt conveyor system has many problems in long-term operation, which seriously affect its safety, reliability and intelligence level, mainly reflected in the following aspects: Inadequate monitoring coverage and limited recognition capabilities: Traditional systems often use sensors and cameras located in key locations, such as drive ends and transfer points, failing to cover the middle or edges of long conveyor belts. This creates numerous blind spots, preventing early detection of some faults. Furthermore, due to interference factors such as strong dust, low light levels, and high humidity often found in mine environments, some sensors lack sensitivity and have poor anti-interference capabilities. This makes false alarms or missed alarms more likely in low-visibility environments, resulting in weak overall system perception.

[0004] Single response mechanism, difficult to dynamically adjust: Existing systems typically use fixed thresholds for alarms or shutdowns, failing to account for dynamic factors such as equipment aging and changes in the operating environment, making them prone to misjudgments or missed detections. For example, while no intervention is required in the early stages of equipment operation, more frequent control intervention may be required as rollers age. Furthermore, various monitoring devices in existing systems (such as temperature and vibration sensors, deviation sensors, and video surveillance) often operate independently, lacking linkage and collaborative analysis mechanisms, making it impossible to make comprehensive judgments based on the correlation between multiple abnormal signals.

[0005] Extensive control strategies and low automation: Current belt conveyor systems often shut down the system in the event of an anomaly. Although the failure may be localized, the response often disrupts the entire conveyor line, causing production interruptions and economic losses. Furthermore, the system lacks automatic closed-loop control capabilities, requiring manual inspection and intervention after an anomaly occurs. This results in delayed response and low efficiency. Furthermore, manual intervention is even more difficult in mine environments characterized by high temperature, high humidity, and high dust levels.

[0006] Solving these problems presents numerous technical challenges. First, the complex mine environment exposes sensors and visual equipment to dust, moisture, and vibration, making data acquisition stability difficult to ensure. Second, the high density of sensors deployed in long-distance transportation scenarios necessitates technological breakthroughs in achieving high-precision data synchronization and stable communication. Furthermore, the system's response to abnormal conditions requires dynamic threshold adjustment and intelligent judgment based on multiple factors (such as equipment status, historical trends, and environmental parameters). Building a rules engine and multi-factor fusion algorithm also presents challenges. Furthermore, implementing location-based regional intervention control and automated response strategies requires high-precision positioning and hardware and software collaborative design. Summary of the Invention

[0007] In view of the above-mentioned problems, the present invention is proposed.

[0008] Therefore, the technical problem solved by the present invention is: how to solve the complex mine environment, where sensors and visual equipment are exposed to dust, water vapor and vibration for a long time, and the stability of data acquisition is difficult to ensure; in long-distance transportation scenarios, the density of sensors is high, and how to achieve high-precision data synchronization and stable communication still requires technical breakthroughs.

[0009] To solve the above technical problems, the present invention provides the following technical solutions: a multimodal monitoring and anomaly detection method for a belt conveyor system, comprising: deploying a number of multimodal sensors along the entire path of the belt conveyor system, collecting original signals during the operation of the belt conveyor system, and preprocessing the original signals; performing environmental compensation calculations based on the preprocessed original signals to generate engineering data; mapping the compensated engineering data to the same spatial coordinate system of the belt conveyor system, and performing feature extraction to obtain feature vectors; performing multimodal fusion analysis on the feature vectors to realize abnormal state identification and fault level classification, and outputting warning signals and risk levels; executing a matching response control strategy based on the warning signal, risk level and corresponding spatial coordinate system position, and verifying the anomaly identification results through multimodal sensor feedback to realize closed-loop control.

[0010] As a preferred solution of the multimodal monitoring and abnormality detection method of a belt conveyor system described in the present invention, the multimodal sensors include a laser displacement sensor, a temperature and vibration sensor, an infrared thermal imager, a soundprint sensor, a piezoelectric film sensor, a rotary encoder, a temperature sensor and a tension sensor; the engineering data includes lateral offset, vibration acceleration, temperature distribution, proportion of high-frequency energy of the soundprint, belt surface pressure and belt running speed.

[0011] As a preferred solution of the multimodal monitoring and abnormality detection method of a belt conveyor system described in the present invention, wherein: the lateral offset includes recording the output voltage of the laser displacement sensor at the center position of the belt as the reference zero point voltage when the belt is initially installed; determining the displacement change corresponding to each volt of voltage based on the technical parameters of the laser displacement sensor; obtaining the current ambient temperature in real time through the temperature sensor to determine the thermal expansion coefficient of the belt material; performing machine-selected correction of the displacement value, and outputting the compensated displacement value; the vibration acceleration includes converting the charge signal of the temperature vibration sensor into a voltage through a charge amplifier, calculating the vibration acceleration according to a preset sensitivity coefficient, converting the resistance value into a real-time temperature based on the resistance-temperature relationship of the resistance material, and correcting the acceleration value through the temperature drift curve; the temperature distribution includes converting the grayscale value matrix collected by the infrared thermal imager into an initial temperature distribution through a calibrated quadratic polynomial, monitoring the dust concentration in real time, correcting the temperature value using an exponential compensation formula, and generating a true The actual temperature distribution is obtained by a blackbody furnace calibration experiment, using a blackbody radiation source with a known temperature, recording the average grayscale value corresponding to each temperature point, and then using the least squares method to fit the temperature-grayscale relationship; the high-frequency energy proportion of the soundprint includes performing a short-time Fourier transform on the time domain current signal collected by the soundprint sensor, extracting the high-frequency band energy, dynamically adjusting the signal analysis window length according to the belt running speed, using a long window to improve the frequency resolution at low speeds, and using a short window to enhance the time resolution at high speeds, and calculating the percentage of high-frequency energy in the full frequency band energy; the belt surface pressure includes calculating the original belt surface pressure based on the product of the resistance change obtained by the piezoelectric film sensor and the calibration coefficient, eliminating the influence of ambient temperature on the piezoelectric effect through the temperature sensor, and outputting the real belt surface pressure; the belt running speed includes counting the number of pulses per unit time of the rotary encoder, calculating the original belt running speed in combination with the circumference of the roller, detecting the belt slip rate based on the tension sensor data, and correcting the belt running speed proportionally.

[0012] As a preferred solution of the multimodal monitoring and anomaly detection method of a belt conveyor system described in the present invention, wherein: the engineering data obtained after compensation is mapped to the same spatial coordinate system of the belt conveyor system, including realizing microsecond time synchronization of all sensors, and mapping the engineering data to the same spatial coordinate system for normalization processing according to the belt running speed; establishing a unified three-dimensional belt conveyor spatial coordinate system, wherein the first direction is the belt running direction X-axis, the second direction is the belt transverse width direction Y-axis, and the third direction is the direction perpendicular to the belt surface Z-axis; the lateral offset is linearly projected according to the installation position of the laser displacement sensor in the transverse direction of the belt and the current belt forward distance, and mapped to the transverse cross-sectional position of the belt, i.e., the Y-axis; the vibration acceleration data is based on the installation point of the roller of the belt conveyor system The position, sampling time and belt running distance are projected point by point and mapped to the spatial node where the roller is located; the temperature distribution is mapped in two dimensions based on the imaging time, the equipment viewing angle coverage and the belt running distance to generate the image temperature coverage area; the belt surface pressure is projected in a plane based on the attachment surface position of the piezoelectric film sensor and the belt running distance and mapped to the local area of ​​the belt surface; the belt transportation system path is divided into several spatial perception units, and the mapped multi-source engineering data are gathered in each unit. The multi-dimensional feature vector is constructed by combining the lateral offset, vibration acceleration, temperature distribution and belt surface pressure parameters. The belt running speed and the high-frequency energy proportion of the soundprint that are not involved in the spatial mapping are introduced, and the weighted fusion is performed based on the consistency of the data acquisition time series to output the final fused multi-dimensional feature vector, which is expressed as: , in, To fuse multi-dimensional feature vectors, is the multidimensional feature vector of a specific spatial position, is a time-related weight function, T is the current data collection time, is the product.

[0013] As a preferred solution of the multimodal monitoring and anomaly detection method for a belt conveyor system described in the present invention, the time-related weight function is expressed as: , in, is the speed weight, is the voiceprint weight; the speed weight is expressed as: , in, is the current belt running speed; Design reference speed for the belt; when When the weight increases, set the maximum value of speed weight ,limit , to prevent radical judgment; the voiceprint weight is expressed as: , in, The current high-frequency energy ratio of the voiceprint; is the rate of change, indicating a sudden change in abnormal sound; is the adjustment coefficient to control the amplification effect of consolidation.

[0014] As a preferred solution of the multimodal monitoring and anomaly detection method for a belt conveyor system described in the present invention, the method for realizing abnormal state identification and fault level classification includes constructing a feature distribution mean and covariance matrix based on historical normal operating condition data, and using Mahalanobis distance to calculate the structural deviation degree of the current fused multidimensional feature vector; setting a dynamic judgment threshold based on historical fluctuation characteristics to determine whether the current state is abnormal; after determining the abnormality, calculating the standardized deviation contribution of each feature dimension of the fused multidimensional feature vector, and extracting the set of dimensions that dominate the abnormality; the method for calculating the structural deviation degree of the current fused multidimensional feature vector using Mahalanobis distance is expressed as: , in, is the degree of structural deviation, is the mean of the characteristic distribution constructed based on historical normal operating condition data, is the covariance matrix constructed based on historical normal operating condition data, and A is the transpose. The mean and standard deviation of the Mahalanobis distance in the time window are introduced to calculate the dynamic judgment threshold, which is expressed as: , in, is the sensitivity adjustment parameter, is the dynamic decision threshold, is the mean of the Mahalanobis distance, is the standard deviation of the Mahalanobis distance; the standardized offset contribution is expressed as: , in, For the The current offset in the dimension, In the history window The standard deviation of the dimension, The contribution of the standardized deviation is calculated; a contribution threshold is set, and the dimensions whose standardized deviation contribution exceeds the contribution threshold are selected from all dimensions to construct a set of abnormal dominant source dimensions.

[0015] As a preferred solution of the multimodal monitoring and abnormality detection method of a belt conveyor system described in the present invention, the verification of the abnormality identification result through multimodal sensor feedback includes: when the abnormality dominant source dimension set simultaneously includes vibration acceleration parameters, temperature distribution parameters and surface pressure parameters, the abnormality type is determined to be roller jamming; when the abnormality dominant source dimension set includes lateral offset parameters and surface pressure parameters, and does not include vibration acceleration parameters and temperature distribution parameters, the abnormality type is determined to be belt deviation; when the abnormality dominant source dimension set includes running speed parameters, and surface pressure parameters are not included, and the proportion of high-frequency energy of the soundprint does not constitute a dominant source, the abnormality type is determined to be belt slippage.

[0016] The invention provides a multimodal monitoring and abnormality detection system for a belt conveyor system.

[0017] To solve the above technical problems, the present invention provides the following technical solutions: a multimodal monitoring and anomaly detection system for a belt conveyor system, comprising: an original signal acquisition module, an environmental compensation module, a spatial mapping module, a fusion recognition module and a response control module; the original signal acquisition module is used to deploy a number of multimodal sensors along the entire path of the belt conveyor system to collect the original signals during the operation of the belt conveyor system and pre-process the original signals; the environmental compensation module is used to perform environmental compensation calculations based on the pre-processed original signals to generate engineering data; the spatial mapping module is used to map the compensated engineering data to the same spatial coordinate system of the belt conveyor system, and complete feature extraction to obtain a feature vector; the fusion recognition module is used to perform multimodal fusion analysis on the feature vector to realize abnormal state identification and fault level classification, and output a warning signal and risk level; the response control module is used to execute a matching response control strategy according to the warning signal, risk level and corresponding spatial coordinate system position, and verify the anomaly identification result through multimodal sensor feedback to realize closed-loop control.

[0018] The present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the multimodal monitoring and abnormality detection method of a belt conveyor system are implemented.

[0019] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the multimodal monitoring and abnormality detection method of a belt conveyor system are implemented.

[0020] The present invention's beneficial effects: By introducing a multimodal fusion perception mechanism, characteristic vector structure offset calculation, and anomaly type identification method based on dominant dimension combinations, the present invention addresses the existing problems of belt conveyor systems, such as reliance on a single sensor for anomaly monitoring, inability to interpret judgment logic, and difficulty identifying multi-parameter coupling faults. The present invention's solution enables dynamic identification of abnormal states under complex operating conditions, and combines the characteristic offset structure to determine the specific anomaly type and dominant cause, enabling differentiated response control. This improves the belt conveyor system's ability to identify and accurately handle early-stage, multi-source, and dynamic anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 This is an overall flow chart of a multimodal monitoring and anomaly detection method for a belt conveyor system provided by one embodiment of the present invention.

[0023] Figure 2 A module diagram of a multimodal monitoring and anomaly detection system for a belt conveyor system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0024] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0025] Example 1, with reference to Figure 1 , is an embodiment of the present invention, which provides a multimodal monitoring and anomaly detection method for a belt conveyor system, comprising: S1. Several multimodal sensors are deployed along the entire path of the belt conveyor system to collect the original signals during the operation of the belt conveyor system and pre-process the original signals.

[0026] S2. Perform environmental compensation calculation based on the preprocessed original signal to generate engineering data.

[0027] S3. Map the engineering data obtained after compensation to the same spatial coordinate system of the belt conveyor system, and complete feature extraction to obtain a feature vector.

[0028] S4. Perform multimodal fusion analysis on the feature vector to realize abnormal state recognition and fault level classification, and output warning signals and risk levels.

[0029] S5. Execute the matching response control strategy based on the warning signal, risk level, and corresponding spatial coordinate system position, and verify the abnormality identification results through multimodal sensor feedback to achieve closed-loop control.

[0030] It should be noted that, according to the aforementioned steps, the present invention addresses the existing problems in belt conveyor systems, such as reliance on a single sensor for abnormality monitoring, unexplainable judgment logic, and difficulty identifying multi-parameter coupling faults, by introducing a multimodal fusion perception mechanism, feature vector structure offset calculation, and anomaly type identification method based on dominant dimension combinations. The present invention's solution enables dynamic identification of abnormal states under complex operating conditions, and combines the feature offset structure to determine the specific anomaly type and dominant cause, enabling differentiated response control. This improves the belt conveyor system's ability to identify and accurately handle early-stage, multi-source, and dynamic anomalies.

[0031] Example 2 is an embodiment of the present invention, which provides a multimodal monitoring and abnormality detection method for a belt conveyor system based on the previous embodiment, including: In the embodiment of the present application, in step S1, a plurality of multimodal sensors are arranged along the entire path of the belt conveyor system to collect the original signals during the operation of the belt conveyor system, and the original signals are pre-processed, including: By deploying multiple multimodal sensors along the entire path of the belt conveyor system, including laser displacement sensors, temperature and vibration sensors, infrared thermal imagers, voiceprint sensors, piezoelectric film sensors, and rotary encoders, the corresponding original physical signals are collected, and signals such as voltage, current, charge, grayscale, and pulse are preprocessed.

[0032] Traditional methods rely on single-point sensors or cameras, which only cover critical areas and fail to achieve real-time monitoring of the entire conveyor belt path. Sensor equipment performance degrades in complex environments (such as coal dust, high humidity, and low light), leading to false alarms and missed detections.

[0033] The present invention deploys various types of multimodal sensors throughout the entire path of the belt conveyor system, including laser displacement sensors, temperature and vibration sensors, infrared thermal imagers, voiceprint sensors, piezoelectric film sensors, and rotary encoders, etc., to collect corresponding original physical signals as the data basis for subsequent feature extraction and abnormality judgment.

[0034] A laser displacement sensor is used to collect voltage signals, which will be used to convert the lateral offset distance of the belt in subsequent processing.

[0035] A temperature-vibration sensor is used to collect charge signals and resistance values ​​for further calculation of vibration acceleration and real-time temperature.

[0036] An infrared thermal imager is used to collect the grayscale matrix of the digital image, and the grayscale value will be used to restore the belt surface temperature distribution map.

[0037] Use a voiceprint sensor to collect current and voltage signals, and analyze their spectral characteristics to identify the proportion of high-frequency abnormal sound energy.

[0038] A rotary encoder is used to collect pulse signals and calculate the belt running speed and cumulative running distance.

[0039] A piezoelectric film sensor is used to collect resistance changes or charge amounts, providing input basis for subsequent conversion of belt surface pressure.

[0040] The signals output by the above sensors mainly include voltage, current, charge, grayscale value and pulse count, etc., which are raw data in analog or digital form. They are directly collected data and need to be preprocessed to eliminate interference and abnormal noise to form a stable and usable primary signal stream.

[0041] Furthermore, voltage, current, and charge are the raw physical signals output by sensors. They are essentially continuously changing analog quantities (for example, the 0-10V voltage fluctuations output by a laser sensor). These signals themselves do not carry physical meaning and must be converted into engineering data using calibration formulas. For example, the charge signal from a temperature and vibration sensor requires a formula to convert it into acceleration. In this case, the charge is the signal, and the acceleration is the data.

[0042] In the embodiment of the present application, performing environmental compensation calculation based on the pre-processed original signal in step S2 to generate engineering data includes: The original signal is compensated based on environmental factors, and the sensor calibration formula is used to convert the signal into engineering data with physical significance, including lateral offset, vibration acceleration, temperature distribution, high-frequency energy ratio of soundprint, belt running speed and belt surface pressure.

[0043] The laser displacement value is calculated through temperature compensation; the vibration acceleration is calculated through temperature drift compensation; the infrared temperature value is calculated through dust compensation; the high-frequency energy ratio of the soundprint is calculated through dynamic STFT analysis; the true belt speed is calculated through slip rate correction; and the true pressure value of the piezoelectric film is calculated through temperature compensation.

[0044] The displacement value of the laser displacement sensor, i.e. the method for generating the lateral offset, is as follows: The directly collected data is preprocessed to obtain subsequent indirect data for coupling, completing the dynamic threshold adjustment of multimodal data collaborative verification.

[0045] Direct data refers to the directly collected voltage, current, charge, grayscale value and pulse count data, while indirect data refers to engineering data. Compensation is taken into account in the calculation, which is the dynamic threshold adjustment of the present invention.

[0046] When the belt is initially installed, record the output voltage of the sensor at the center of the belt as the reference zero voltage. Based on the sensor technical parameters, determine the displacement change corresponding to each volt of voltage and express it as: Original displacement = (current voltage - reference zero voltage) / displacement change per volt Temperature also affects belt displacement, so temperature compensation correction should be considered.

[0047] The current ambient temperature is obtained in real time through the temperature sensor, the thermal expansion coefficient of the belt material is determined, and the displacement value is corrected by the machine. The output lateral offset is expressed as: Lateral offset = original displacement × (1 + thermal expansion coefficient × temperature difference) The specific steps of obtaining vibration acceleration by the temperature vibration sensor include: The charge signal is converted into voltage through a charge amplifier, and the vibration acceleration is calculated according to the preset sensitivity coefficient. Based on the resistance-temperature relationship of the platinum resistor, the resistance value is converted into real-time temperature, and the acceleration value is corrected by the temperature drift curve. Platinum has extremely high chemical stability and can still maintain a linear relationship between resistance and temperature in high temperature, high humidity, and corrosive environments. It is suitable for complex working conditions such as coal mines. Compared with materials such as copper and nickel, platinum is not easily oxidized at high temperatures and has strong long-term stability. The resistance of the platinum thermal resistor PT1000 is 1000Ω at 0℃. For every 1℃ change in temperature, the resistance changes by about 3.85Ω. The sensitivity is 10 times that of the platinum thermal resistor PT100, making it more suitable for accurate detection of slight temperature changes. The standard operating range of platinum metal is -200℃~+850℃, covering the needs of coal mine belt monitoring, expressed as: Original vibration acceleration = charge amount / charge amplifier sensitivity Vibration acceleration = original acceleration × [1-temperature drift coefficient × (current temperature-calibration temperature)] The method for generating the temperature distribution of an infrared thermal imager includes the following steps: The grayscale matrix is ​​converted into an initial temperature distribution using a calibrated quadratic polynomial. Dust concentration is monitored in real time, and the temperature values ​​are corrected using an exponential compensation formula to generate a true temperature distribution map. The thermal imager quantifies the intensity of the received infrared radiation into grayscale values ​​on a scale of 0-255, with high grayscale values ​​in high-temperature areas and low grayscale values ​​in low-temperature areas.

[0048] Through the blackbody furnace calibration experiment, a blackbody radiation source with a known temperature is used to record the average grayscale value corresponding to each temperature point. Then, the temperature-grayscale relationship is obtained by fitting using the least squares method: Temperature = a × grayscale value 2 +b×grayscale value+c Among them, a, b, and c are values ​​calibrated through experiments on the blackbody radiation law.

[0049] Compensated surface temperature = temperature × e 0.015×粉尘浓度 The method for generating the high-frequency energy ratio of a voiceprint of a voiceprint sensor includes the following steps: Perform short-time Fourier transform (STFT) on the time domain current signal to extract the energy in the 80-120kHz frequency band.

[0050] Abnormal belt noise (such as bearing jamming) manifests as a transient impact signal. Traditional Fast Fourier Transform (FFT) cannot capture changes in the time dimension. STFT can provide both time and frequency resolution.

[0051] The signal analysis window length is dynamically adjusted according to the belt running speed. A long window is used to improve the frequency resolution at low speed, and a short window is used to enhance the time resolution at high speed. The percentage of high-frequency energy in the entire frequency band energy is calculated as the basis for determining mechanical faults.

[0052] The method for generating the belt running speed of the rotary encoder includes the following steps: The number of pulses per unit time of the rotary encoder is counted and the original speed is calculated based on the circumference of the drum.

[0053] The belt slip rate is detected based on the tension sensor data, and the proportionally corrected speed value is expressed as: Original speed = number of pulses / (number of pulses per revolution of encoder × time interval) × circumference of roller Compensated belt running speed = original speed × (1-slip rate) The method for generating a belt surface pressure value of a piezoelectric film sensor comprises the following steps: The original pressure is calculated based on the product of the resistance change and the calibration coefficient.

[0054] The auxiliary temperature sensor is used to eliminate the influence of ambient temperature on the piezoelectric effect and output the true pressure value, which is expressed as: Original pressure = calibration coefficient × resistance change Compensated surface pressure = original pressure - (temperature compensation coefficient × temperature change) Indirect data is obtained by preprocessing direct data.

[0055] In the embodiment of the present application, in step S3, the engineering data obtained after compensation is mapped to the same spatial coordinate system of the belt conveyor system, and feature extraction is performed to obtain a feature vector including: Due to the wide variety of sensors, their dispersed installations, and varying signal acquisition methods, the engineering parameters calculated in step S2 are distributed across different locations, times, and dimensions, making them unsuitable for direct use in comprehensive anomaly identification. Therefore, step S3 requires the establishment of a spatial coordinate system and data mapping to project all data onto a common reference framework, thereby constructing a spatiotemporal feature structure. This is achieved by spatially aggregating the mappable parameters of belt lateral offset, vibration acceleration, surface temperature, and belt surface pressure, while also incorporating the non-spatial parameters of operating speed and voiceprint as dynamic weighting factors. This achieves the coupling of spatial state and temporal dynamics, ultimately forming a fused feature vector.

[0056] The compensated engineering data is uniformly mapped to the three-dimensional spatial coordinate system of the belt operation. Dynamic position projection is performed based on the sensor installation position, timestamp and belt running distance. Multimodal feature extraction is performed under unified coordinates to form a multidimensional feature vector.

[0057] Traditional solutions use a fixed thermal expansion coefficient (e.g., a standard value of ±0.0001 / °C for metal materials). However, the expansion coefficient of belt materials (such as rubber and composite materials) can fluctuate due to aging, humidity, and other factors, leading to compensation errors. Coal mines experience significant day-night temperature fluctuations (e.g., -20°C to 60°C). Long-term use of static coefficients can result in displacement measurement errors exceeding ±1mm. The thermal expansion coefficient is automatically calibrated every 30 minutes (using a heating device to locally increase temperature and measure deformation to infer the α value). The compensation coefficient is updated in real time, reducing the error to ±0.2mm. For high-humidity environments (e.g., underground coal mine humidity >90%), the α value algorithm for rubber materials has been optimized (from the standard 0.0005 / °C to 0.00048 / °C).

[0058] The method of the present invention achieves microsecond-level time synchronization of all sensors through the Precision Time Protocol (PTP) protocol, and maps the engineering data to the same spatial coordinate system for normalization processing based on the belt running speed (rotary encoder data).

[0059] A unified three-dimensional coordinate system for belt conveyor is established, wherein the first direction is the running direction of the belt, the second direction is the transverse width direction of the belt, and the third direction is the direction perpendicular to the belt surface.

[0060] The lateral offset data is linearly projected according to the sensor's installation position in the lateral direction of the belt and the current belt advance distance, and mapped to the lateral cross-sectional position of the belt; the vibration acceleration data is point-projected according to the roller installation point position, sampling time and belt running distance, and mapped to the spatial node where the roller is located; the infrared thermal imaging temperature data is two-dimensionally mapped according to the imaging time, the equipment viewing angle coverage range and the belt running distance to generate the image temperature coverage area; the belt surface pressure data is planarly projected according to the attachment surface position of the piezoelectric film and the belt running trajectory, and mapped to the local surface area of ​​the belt surface.

[0061] It should be noted that each of the aforementioned engineering parameters, such as lateral offset, vibration acceleration, temperature distribution, and surface pressure, is characteristic data obtained after physical quantity calculation and environmental compensation in step S2. These parameters essentially constitute the components of the subsequent characteristic vectors. Therefore, the spatial mapping and aggregation operations performed in step S3 aim to structure these already physically meaningful parameters according to a unified coordinate standard. This does not generate new parameters, but rather spatially organizes and integrates information from existing results.

[0062] The belt conveyor path is divided into several spatial perception units. The mapped multi-source engineering data are gathered in each unit, and a multidimensional feature vector is constructed by combining the lateral offset, vibration acceleration, temperature distribution, and belt surface pressure. The belt running speed and the high-frequency energy ratio of the soundprint that are not involved in the spatial mapping are introduced, and weighted fusion is performed based on the consistency of data acquisition time series to output the final multidimensional feature vector.

[0063] Considering that the aforementioned parameters come from different sources and have different dimensions (e.g., offset in millimeters, acceleration in m / s², temperature in degrees Celsius, and pressure in Pascals), direct fusion may introduce dimensional imbalances and weight imbalances. Therefore, in this step, after completing the spatial mapping of each parameter and before feature aggregation, a unified normalization mechanism is introduced. Using methods such as linear scaling, Z-score, or min-max normalization, all eigenvalues ​​are normalized to the range of 0 and 1. This ensures that the fused feature vector has a unified dimensional scale when constructed, facilitating weight allocation and model analysis.

[0064] In order to achieve unified spatial representation and fusion analysis of multimodal data, the present invention establishes a three-dimensional spatial coordinate system corresponding to the belt conveyor path: the first direction is the belt running direction (X-axis), the second direction is the belt transverse width direction (Y-axis), and the third direction is the direction perpendicular to the belt surface (Z-axis). All types of engineering data need to be mapped to this coordinate system.

[0065] Given the different installation methods, sensing ranges, and data attributes of various sensors, their spatial mapping logic is as follows: The lateral offset output by the laser displacement sensor is determined based on the Y coordinate value of its installation position on the belt cross section, combined with the running distance of the belt during the sampling time (calculated by the encoder), to determine its mapping position in the X-axis direction, forming a linear projection on the XY plane: , The above calculation The theoretical integral formula is used for continuous signal conditions to obtain accurate cumulative displacement, while in actual sampling, discrete integral approximation is used for: , Discrete integral approximation Used for speed integral accumulation under discrete sampling conditions such as encoders, using discrete integral approximation The final laser displacement data is mapped to coordinate points: , in, It is the real-time position of the laser displacement data in the horizontal direction; The installation reference position of the laser displacement sensor on the belt cross section (second direction), in meters; The lateral deviation of the belt collected at the current moment is converted from the laser voltage value and is in meters. The current data collection time, in seconds; For belts at the moment The running speed is in meters per second; is the initial belt running position, set to 0; For belts at the moment The cumulative running distance; For the The speed value of the sampling point, is the total number of sampling points; is the sampling interval; The coordinate points mapped to the final laser displacement data.

[0066] The vibration acceleration data output by the temperature vibration sensor is used to calculate its XYZ point mapping in the coordinate system based on the YZ coordinates of the roller installation point and the forward displacement of the belt; The installation position of the temperature and vibration sensor in the transverse direction of the belt; is the fixed height of the temperature vibration sensor in the vertical direction (third direction); is the current data collection moment; is the distance the belt runs at that moment.

[0067] but: , in, is the three-dimensional coordinate position of the sensor when the belt passes through the roller.

[0068] Infrared thermal imaging data, by analyzing the imaging device's field of view and image frame time, combined with the belt advance distance, determines the dynamic projection area of ​​the image temperature coverage area in the XY plane; The vertical installation height from the thermal imager to the belt surface; is the current data collection moment; the image is an N×M pixel matrix.

[0069] The horizontal coverage is: , in, 、 are the left and right boundaries of the infrared image in the horizontal projection of the belt, It is the horizontal coordinate position corresponding to the vertical projection of the center of the thermal imager lens onto the belt surface. is the vertical installation height from the thermal imager to the belt surface, is the horizontal field of view of the infrared thermal imager.

[0070] Longitudinal (belt direction) coverage (assuming image acquisition cycle is ): , in, 、 They are the starting position and ending position of the image coverage area in the belt running direction, is the cumulative running distance of the belt at time, The belt speed during image acquisition.

[0071] Rule No. The pixel coordinates are: , in, is the number of pixel rows of the infrared image in the belt running direction (X axis), is the number of pixel columns of the infrared image in the transverse direction of the belt (Y axis), is the index of the current pixel in the row direction of the image, It is the index of the current pixel in the column direction of the image.

[0072] The coordinates of the image are discrete (integer pixel numbers), but the physical space is continuous, so the integer index , Scaled to the interval value of [0, 1] for interpolation calculation, 、 Indicates the relative position of the current pixel in the entire image range (normalized position scale).

[0073] Each pixel in the final image can be uniquely located to a point on the two-dimensional coordinate plane of the belt.

[0074] The piezoelectric film pressure data is mapped to an XY surface block of a certain area on the belt surface based on the position of the sensor attachment area on the belt surface and the current running distance of the belt. The width direction (lateral direction) of the piezoelectric film; is the length direction (longitudinal); is the horizontal coordinate of the attachment starting point; is the current data collection moment; The current belt advance distance.

[0075] The four boundaries of the mapping domain are: ,

[0076] The region is defined in the spatial coordinate system as: , in, is the set of piezoelectric film mapping areas, represented as rectangular areas in the belt's two-dimensional coordinate system; is the coordinate of any point in the area in the belt running direction (first direction, X axis); is the coordinate of any point in the area in the transverse width direction of the belt (the second direction, the Y axis); The starting projection position of the piezoelectric film in the X direction (towards the front end); The end projection position of the piezoelectric film in the X direction (towards the rear end); is the starting position of the piezoelectric film in the Y direction (towards the left end); The end position of the piezoelectric film in the Y direction (towards the right end).

[0077] Through the above method, multimodal perception data of different dimensions are uniformly mapped to specific points, line segments or areas in the three-dimensional spatial coordinate system, and then divided into spatial units at fixed intervals (such as an X-direction grid every 1 meter), and data aggregation is completed within each unit.

[0078] Based on data aggregation, the system extracts the following main features: lateral offset, vibration acceleration value, compensated surface temperature, and compensated surface pressure, forming a high-dimensional operating status feature vector for each spatial unit.

[0079] At the same time, considering that some data (such as belt running speed and the proportion of high-frequency energy of soundprint) do not have a clear spatial mapping relationship, but are highly sensitive to the overall operating status, they are added to the feature vector as time series weighting factors to form a "main spatial feature * auxiliary dynamic factor" structure to provide input support for the subsequent anomaly recognition model.

[0080] The four-dimensional spatial features are fused with weights to obtain a fused feature vector with enhanced dynamic sensitivity.

[0081] Construct a fusion feature vector composed of a weighted combination of multiple spatially mappable parameters and temporal dynamic factors, expressed as: , in, To fuse multi-dimensional feature vectors, is the multidimensional feature vector of a specific spatial position, is a time-related weight function, T is the current data collection time, is the product, multiplication operation.

[0082] Local space eigenvector: , in, is the normalized lateral offset, is the normalized vibration acceleration value, is the normalized surface temperature, is the normalized surface pressure, corresponding to a specific coordinate point; It is a time-dependent weight function used to control the contribution of non-spatial data to the results (can be generated by neural networks, empirical rules, or model logic).

[0083] The weight adjustment is shown in Table 1.

[0084] Table 1 Weight adjustment table , The time-dependent weight is expressed as: , in, is the speed weight, is the voiceprint weight.

[0085] current 、 These are two dynamic adjustment factors rather than the input features themselves, speed Voiceprint change rate In the calculation, these dimensions do not directly participate in the anomaly detection algorithm as feature vector dimensions. Instead, they act as response adjustment factors to influence the weighted strength of the overall features. Their function is to increase sensitivity under certain operating conditions, such as amplifying the weight of abnormal signals during high-speed operation or sudden changes in the voiceprint. Therefore, no normalization is performed.

[0086] The fused multidimensional feature vector is expressed as: , The weight design of belt speed, linear amplification and saturation suppression type, is expressed as: , in, is the current belt running speed, Design reference speed for the belt. , the weight increases, which means the faster the belt runs, the lower the abnormal tolerance is, and the maximum speed weight is set. ,limit , to prevent radical judgments; when running at high speed, even a slight offset or temperature rise is more likely to quickly evolve into a fault, so the reaction weight must be increased.

[0087] The weight design of the voiceprint change rate and the change rate driven mutation function are expressed as: , in, is the current voiceprint high-frequency energy ratio, is the rate of change, indicating a sudden change in the abnormal sound. To adjust the coefficient and control the amplification effect of the solid change. Rapid changes in the intensity of abnormal noise often indicate that the roller or support structure is rapidly deteriorating, and the sensitivity should be quickly triggered to increase.

[0088] The final fusion weight function is: , The meanings of the characters are shown in Table 2.

[0089] Table 2 Character meaning table , In the embodiment of the present application, in step S4, multimodal fusion analysis is performed on the feature vector to realize abnormal state identification and fault level classification, and the output of warning signals and risk levels includes: Perform multimodal fusion analysis on the feature vectors, and realize abnormal state identification and fault level classification through dynamic threshold judgment, time series trend modeling or machine learning algorithms, and output warning signals and risk levels.

[0090] The unmappable dynamic factor parameters are weighted by a weight function and then multiplied and fused with the mappable parameters to generate a fused feature vector. The feature distribution mean and covariance matrix are constructed based on historical normal operating condition data, and the Mahalanobis distance is used to calculate the structural deviation of the current feature vector. A dynamic judgment threshold based on historical fluctuation characteristics is set to determine whether the current state is abnormal. After determining the anomaly, the standardized deviation contribution of each feature dimension is calculated to extract the set of dominant source dimensions of the anomaly. The preset abnormal structure type map is matched according to the combination of dominant source dimensions to determine the current anomaly type. The spatial anomaly response map is generated by combining the abnormal location coordinates, type, and risk level, and outputs hierarchical control instructions.

[0091] The dynamic judgment threshold calculates the trend change rate based on the second-order time derivative of the eigenvector and corrects the current threshold through the trend prediction model to improve the response capability to rapid parameter offsets.

[0092] The generation of response instructions is based on the spatial response map constructed by the triplets of abnormality type, risk level and spatial coordinates, and differentiated control strategies such as speed limit, prompt or shutdown are implemented according to the abnormal conditions in different areas.

[0093] Construct a historical window of nearly N moments The feature set within is expressed as: , in, is the feature set, A historical window of nearly N moments The moment within, for The fused multidimensional feature vector at each moment.

[0094] Calculate the expectation (mean) and covariance of the historical sample: , in, is a collection of historical normal operating conditions All eigenvectors in Calculate the dimension-by-dimension average, the result is a vector of length 4, called the historical mean vector of the fused feature vector, is a collection of historical normal operating conditions All eigenvectors in , calculate the covariance matrix to measure the joint change relationship between each feature. The result is a 4×4 symmetric matrix for subsequent Mahalanobis distance determination. is the average value vector of the fused feature vector in the historical window, is the covariance matrix of the fused eigenvectors in the history window.

[0095] Define the Mahalanobis distance function to characterize the structural deviation between the current vector and the normal distribution: , in, is the degree of structural deviation, is the mean of the characteristic distribution constructed based on historical normal operating condition data, is the covariance matrix constructed based on historical normal operating condition data, and A is the transpose.

[0096] Introducing the mean and standard deviation of the Mahalanobis distance within the time window , build dynamic anomaly threshold: , in, It is a sensitivity adjustment parameter, which can be adjusted according to risk preference. is the dynamic judgment threshold, reflecting the adaptive abnormal tolerance of the system under the current working conditions. is the mean of the Mahalanobis distance, is the standard deviation of the Mahalanobis distance.

[0097] For example, if the Mahalanobis distance is calculated every second or minute for a period of time (e.g., the last 10 time points), a Mahalanobis distance dataset will exist, and the mean and standard deviation can be calculated. The subscript D indicates a statistic calculated for the Mahalanobis distance: the mean and standard deviation of the Mahalanobis distance. As the dynamic decision threshold, the Mahalanobis distance function is compared with the result. > Otherwise it is normal.

[0098] When the system determines that an anomaly exists, it is necessary to further identify which parameters dominate the abnormal behavior, that is, to clarify the source of the deviation.

[0099] The constructed offset vector is expressed as: ,

[0100] in, is the offset vector, Represents the offset value of the i-th feature dimension.

[0101] Local spatial eigenvector .

[0102] Here It represents the engineering quantity value of the i-th characteristic dimension collected or calculated by the system at time T. is the lateral offset distance d, is the vibration acceleration a, is the temperature t, is the pressure p. is the historical mean of the i-th feature dimension. The larger the offset, the further the current state deviates from the normal state in terms of the i-th feature. Combined with the historical fluctuation of this dimension , calculate its standardized contribution This parameter is the contribution strength to the overall deviation of the anomaly.

[0103] The Mahalanobis distance is used to calculate the structural deviation of the current feature vector, and the abnormal contribution of each dimension is defined as: , in, is the current offset of the i-th dimension, is the standard deviation of the dimension in the historical window, It is the standardized contribution, indicating the relative impact of the dimension on the anomaly.

[0104] Setting contribution thresholds , identify the main dominant factors: , The current i is the dimension number, which is 1, 2, 3, or 4. It is the contribution threshold of the dominant factor, generally ranging from 0.2 to 0.3.

[0105] That is, among all dimensions, the dimension whose contribution exceeds the contribution threshold is selected and regarded as the dominant source of abnormality. It is the contribution of each feature dimension to the current anomaly.

[0106] In the embodiment of the present application, in step S5, a matching response control strategy is executed according to the warning signal, risk level and corresponding spatial coordinate system position, and the abnormality identification result is verified through multimodal sensor feedback to achieve closed-loop control, including: Differentiated control strategies such as speed limit, prompt or shutdown are implemented according to abnormal conditions in different areas.

[0107] When the dominant source dimension set contains acceleration parameters, temperature parameters, and pressure parameters at the same time, the abnormality type is determined to be roller jamming; When the dominant source dimension set includes distance parameters and pressure parameters, but does not include acceleration parameters and temperature parameters, the abnormality type is determined to be belt deviation; When the dominant source dimension set includes the speed parameter and the pressure parameter is not included, and the voiceprint signal does not constitute a dominant source, the abnormality type is determined to be belt slip.

[0108] Example 3, reference Figure 2, is an embodiment of the present invention, which provides a multimodal monitoring and anomaly detection system for a belt conveyor system, including an original signal acquisition module, an environmental compensation module, a spatial mapping module, a fusion recognition module and a response control module.

[0109] The original signal acquisition module is used to deploy a number of multimodal sensors along the entire path of the belt conveyor system, collect the original signals during the operation of the belt conveyor system, and pre-process the original signals.

[0110] The environmental compensation module is used to perform environmental compensation calculations based on the preprocessed original signal and generate engineering data.

[0111] The spatial mapping module is used to map the compensated engineering data to the same spatial coordinate system of the belt conveyor system and complete feature extraction to obtain feature vectors.

[0112] The fusion recognition module is used to perform multimodal fusion analysis on feature vectors, realize abnormal state recognition and fault level classification, and output warning signals and risk levels.

[0113] The response control module is used to execute the matching response control strategy according to the warning signal, risk level and the corresponding spatial coordinate system position, and verify the abnormality identification results through multimodal sensor feedback to achieve closed-loop control.

[0114] This embodiment also provides an electronic device suitable for a multimodal monitoring and anomaly detection method for a belt conveyor system, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement a multimodal monitoring and anomaly detection method for a belt conveyor system as proposed in the above embodiment.

[0115] This embodiment further provides a storage medium storing a computer program, which, when executed by a processor, implements a multimodal monitoring and abnormality detection method for a belt conveyor system as proposed in the above embodiment.

[0116] The storage medium proposed in this embodiment and the multimodal monitoring and abnormality detection method for a belt conveyor system proposed in the above embodiment belong to the same inventive concept. Technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented with the help of software and necessary general-purpose hardware. Of course, it can also be implemented with hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0118] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A multimodal monitoring and anomaly detection method for a belt conveyor system, characterized by: include, Several multimodal sensors are deployed along the entire path of the belt conveyor system to collect the original signals during the operation of the belt conveyor system and pre-process the original signals; Perform environmental compensation calculation based on the pre-processed original signal to generate engineering data; The engineering data obtained after compensation is mapped to the same spatial coordinate system of the belt conveyor system, and feature extraction is completed to obtain feature vectors; Perform multimodal fusion analysis on feature vectors to identify abnormal states and classify fault levels, and output warning signals and risk levels; According to the warning signal, risk level and the corresponding spatial coordinate system position, the matching response control strategy is executed, and the abnormality identification results are verified through multimodal sensor feedback to achieve closed-loop control.

2. The multimodal monitoring and anomaly detection method for a belt conveyor system according to claim 1, wherein: The plurality of multimodal sensors include a laser displacement sensor, a temperature vibration sensor, an infrared thermal imager, a voiceprint sensor, a piezoelectric film sensor, a rotary encoder, a temperature sensor and a tension sensor; The engineering data includes lateral offset, vibration acceleration, temperature distribution, high-frequency energy ratio of sound pattern, belt surface pressure and belt running speed.

3. The multimodal monitoring and anomaly detection method for a belt conveyor system according to claim 2, wherein: The lateral offset includes recording the output voltage of the laser displacement sensor at the center of the belt as the reference zero point voltage when the belt is initially installed; According to the technical parameters of the laser displacement sensor, determine the displacement change corresponding to each volt of voltage; The current ambient temperature is obtained in real time through the temperature sensor to determine the thermal expansion coefficient of the belt material; Perform machine-selected correction of displacement value and output the compensated displacement value; The vibration acceleration includes converting the charge signal of the temperature vibration sensor into a voltage through a charge amplifier, calculating the vibration acceleration according to a preset sensitivity coefficient, converting the resistance value into a real-time temperature based on the resistance-temperature relationship of the resistor material, and correcting the acceleration value through a temperature drift curve; The temperature distribution includes converting the grayscale value matrix collected by the infrared thermal imager into an initial temperature distribution through a calibrated quadratic polynomial, monitoring the dust concentration in real time, correcting the temperature value using an exponential compensation formula, generating a true temperature distribution, performing a blackbody furnace calibration experiment, using a blackbody radiation source of known temperature, recording the average grayscale value corresponding to each temperature point, and then using the least squares method to fit the temperature-grayscale relationship; The proportion of high-frequency energy of the voiceprint includes performing a short-time Fourier transform on the time-domain current signal collected by the voiceprint sensor to extract the high-frequency energy, dynamically adjusting the signal analysis window length according to the belt running speed, using a long window to improve the frequency resolution at low speed and a short window to enhance the time resolution at high speed, and calculating the percentage of high-frequency energy in the total frequency band energy; The belt surface pressure includes calculating the original belt surface pressure based on the product of the resistance change obtained by the piezoelectric film sensor and the calibration coefficient, eliminating the influence of the ambient temperature on the piezoelectric effect through the temperature sensor, and outputting the real belt surface pressure; The belt running speed includes counting the number of pulses per unit time of the rotary encoder, calculating the original belt running speed in combination with the circumference of the roller, detecting the belt slip rate based on the tension sensor data, and correcting the belt running speed proportionally.

4. The multimodal monitoring and anomaly detection method for a belt conveyor system according to claim 3, wherein: The engineering data obtained after compensation is mapped to the same spatial coordinate system of the belt conveyor system, including: Achieve microsecond-level time synchronization of all sensors and map engineering data to the same spatial coordinate system for normalization based on belt running speed; Establish a unified three-dimensional coordinate system for the belt conveyor, where the first direction is the X-axis, the second direction is the Y-axis, and the third direction is the Z-axis perpendicular to the belt surface. The lateral offset is linearly projected based on the installation position of the laser displacement sensor in the lateral direction of the belt and the current belt advance distance, and mapped to the lateral cross-sectional position of the belt, i.e., the Y-axis. The vibration acceleration data is projected based on the installation point position, sampling time and belt running distance of the belt conveyor system's rollers and mapped to the spatial node where the rollers are located. The temperature distribution is mapped in two dimensions based on the imaging time, the device's viewing angle coverage, and the belt's running distance to generate an image temperature coverage area. The belt surface pressure is projected onto a local area of ​​the belt surface based on the position of the piezoelectric film sensor's attachment surface and the belt's running distance. The belt conveyor system path is divided into several spatial perception units. The mapped multi-source engineering data are gathered in each unit. The multidimensional feature vector is constructed by combining the lateral offset, vibration acceleration, temperature distribution, and belt surface pressure parameters. The belt running speed and the high-frequency energy ratio of the soundprint that are not involved in the spatial mapping are introduced. The weighted fusion is performed based on the consistency of the data acquisition time series, and the final fused multidimensional feature vector is output, which is expressed as: ， in, To fuse multi-dimensional feature vectors, is the multidimensional feature vector of a specific spatial position, is a time-related weight function, T is the current data collection time, is the product.

5. The multimodal monitoring and anomaly detection method for a belt conveyor system according to claim 4, wherein: The time-dependent weight function is expressed as, ， in, is the speed weight, is the voiceprint weight; The speed weight is expressed as, ， in, is the current belt running speed; Design reference speed for the belt; when When the weight increases, set the maximum value of speed weight ,limit , to prevent radical judgments; The voiceprint weight is expressed as, ， in, The current high-frequency energy ratio of the voiceprint; is the rate of change, indicating a sudden change in abnormal sound; is the adjustment coefficient to control the amplification effect of consolidation.

6. The multimodal monitoring and anomaly detection method for a belt conveyor system according to claim 5, wherein: The realization of abnormal state identification and fault level classification includes constructing a feature distribution mean and covariance matrix based on historical normal operating condition data, and using Mahalanobis distance to calculate the structural deviation degree of the current fused multi-dimensional feature vector; Set dynamic judgment thresholds based on historical fluctuation characteristics to determine whether the current state is abnormal; After determining the anomaly, the standardized offset contribution of each feature dimension of the fused multidimensional feature vector is calculated to extract the set of dimensions that dominate the anomaly. The structural deviation degree of the current fused multi-dimensional feature vector calculated by using Mahalanobis distance is expressed as: ， in, is the degree of structural deviation, is the mean of the characteristic distribution constructed based on historical normal operating condition data, is the covariance matrix constructed based on historical normal operating condition data, and A is the transpose; The mean and standard deviation of the Mahalanobis distance in the time window are introduced to calculate the dynamic decision threshold, which is expressed as: ， in, is the sensitivity adjustment parameter, is the dynamic decision threshold, is the mean of the Mahalanobis distance, is the standard deviation of the Mahalanobis distance; The normalized offset contribution is expressed as, ， in, For the The current offset in the dimension, In the history window The standard deviation of the dimension, is the normalized offset contribution; A contribution threshold is set, and the dimensions whose standardized deviation contribution exceeds the contribution threshold are selected from all dimensions to construct a set of dimensions that are the dominant source of anomalies.

7. The multimodal monitoring and anomaly detection method for a belt conveyor system according to claim 6, wherein: Verifying the abnormality recognition result through multimodal sensor feedback includes: When the dominant source dimension set of the anomaly includes vibration acceleration parameters, temperature distribution parameters, and surface pressure parameters, the anomaly type is determined to be roller jamming; When the dominant source dimension set of the anomaly includes the lateral offset parameter and the surface pressure parameter, but does not include the vibration acceleration parameter and the temperature distribution parameter, the anomaly type is determined to be belt deviation; When the running speed parameter is included in the set of dimensions of the dominant source of the anomaly, the surface pressure parameter is not included, and the high-frequency energy proportion of the soundprint does not constitute the dominant source, the anomaly type is determined to be belt slippage.

8. A multimodal monitoring and anomaly detection system for a belt conveyor system, using a multimodal monitoring and anomaly detection method for a belt conveyor system according to any one of claims 1 to 7, characterized in that: include: Original signal acquisition module, environmental compensation module, space mapping module, fusion recognition module and response control module; The original signal acquisition module is used to deploy a number of multimodal sensors along the entire path of the belt conveyor system, collect the original signals during the operation of the belt conveyor system, and pre-process the original signals; The environmental compensation module is used to perform environmental compensation calculation based on the pre-processed original signal to generate engineering data; The spatial mapping module is used to map the engineering data obtained after compensation to the same spatial coordinate system of the belt conveyor system, and complete feature extraction to obtain feature vectors; The fusion recognition module is used to perform multimodal fusion analysis on the feature vectors, realize abnormal state recognition and fault level classification, and output warning signals and risk levels; The response control module is used to execute the matching response control strategy according to the early warning signal, risk level and corresponding spatial coordinate system position, and verify the abnormality identification result through multimodal sensor feedback to achieve closed-loop control.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the multimodal monitoring and abnormality detection method for a belt conveyor system according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a multimodal monitoring and abnormality detection method for a belt conveyor system according to any one of claims 1 to 7 are implemented.

Images