Conveyor state monitoring device and method based on machine vision

Abstract

The invention provides a conveyor state monitoring device and method based on machine vision, and the device comprises a monitoring cabin which can autonomously reciprocate along a rigid guide rail at one side of a conveyor, automatic charging piles are arranged at the two ends of the guide rail, and the monitoring cabin is integrated with an infrared thermal imager, a visual camera and a sound sensor. A vibration sensor and an incremental photoelectric encoder are arranged along the line. A processor is arranged in the monitoring cabin, and time alignment, feature extraction, fusion of a multi-scale feature fusion model and three-level decision judgment are carried out on infrared signals, visual signals, sound signals and vibration signals on the basis of encoder timestamps. According to the method, fault diagnosis and false alarm filtering are achieved by controlling inspection of a monitoring cabin, multi-source data preprocessing, confrontation feature fusion network processing and three-level progressive judgment, and accurate positioning is achieved based on the mapping relation between encoder pulses and support numbers. According to the invention, the monitoring coverage range is expanded, the endurance continuity is improved, the data transmission stability is improved, and the false alarm rate is reduced.

Description

Technical Field

[0001] This invention relates to a conveyor condition monitoring device and method based on machine vision, belonging to the field of conveyor condition monitoring technology. Background Technology

[0002] Belt conveyors are core transportation equipment in industrial sectors such as mining and ports. Their health depends primarily on two key components: the idler roller assembly, which serves as the support structure, and the conveyor belt, which bears the load. The idler rollers are numerous and prone to bearing wear and jamming; the conveyor belt, subjected to constant tension and friction, is susceptible to serious problems such as misalignment and internal tearing. Existing monitoring technologies have the following main shortcomings:

[0003] (1) The monitoring method is passive and has blind spots: Most solutions use fixed-point sensor installation or manual inspection, which cannot cover the entire conveyor line and has monitoring blind spots.

[0004] (2) The problem of battery life and power supply for self-driven inspection: A few mobile inspection solutions are limited by battery technology and lack reliable automatic charging and power management logic, making it difficult to guarantee long-term uninterrupted operation.

[0005] (3) Unreliable signal transmission: The data connection between the mobile inspection device and the fixed node is easily interrupted during operation, resulting in the loss of critical data.

[0006] (4) Single function and inability to make comprehensive diagnosis: Existing solutions mostly only detect one of the idlers or the conveyor belt, lacking an integrated platform that can analyze the status of both, and the diagnostic logic is mostly a single judgment with poor fault tolerance.

[0007] (5) Inaccurate positioning and maintenance guidance: Fault alarms can usually only be located to a certain section, and cannot be accurate to the specific location of the idler or conveyor belt, resulting in time-consuming and labor-intensive on-site troubleshooting.

[0008] Therefore, there is an urgent need for an integrated intelligent monitoring solution that can operate autonomously, has reliable endurance, stable data transmission, and can simultaneously perform high-precision detection and positioning of conveyor rollers and conveyor belts. Summary of the Invention

[0009] The present invention provides a conveyor condition monitoring device and method based on machine vision to solve the problems existing in the prior art.

[0010] The technical solutions adopted in this invention are as follows:

[0011] A machine vision-based conveyor condition monitoring device, the conveyor including a conveyor belt, idlers, conveyor line supports, and a drive unit, comprising:

[0012] The monitoring cabin can reciprocate autonomously along a guide rail fixed to one side of the conveyor. Charging piles for charging the monitoring cabin are provided at both ends of the guide rail.

[0013] The multimodal data acquisition module includes an infrared thermal imager, a visual camera and a sound sensor integrated into the monitoring cabin, vibration sensors wirelessly arranged in stages along the support of the conveyor, and an incremental photoelectric encoder installed in the drive unit.

[0014] The data processing and fusion module, deployed in the monitoring cabin, is used to perform time alignment, feature extraction, fusion based on a multi-scale feature fusion model, and three-level decision-making on infrared images, visual images, sound signals, and vibration signals based on the timestamps output by the incremental photoelectric encoder.

[0015] Furthermore, the monitoring cabin includes a multi-degree-of-freedom gimbal, and the visual camera and infrared thermal imager are jointly mounted on the multi-degree-of-freedom gimbal, which are respectively aimed at the surface of the idler roller or the conveyor belt by adjusting the pitch angle.

[0016] Furthermore, position limit sensors are provided at both ends of the guide rail, and a power management unit is provided in the monitoring cabin; when the monitoring cabin runs to the end of the guide rail and triggers the position limit sensor, the power management unit controls the monitoring cabin to connect with the charging pile for charging or run in reverse according to the current power status.

[0017] Furthermore, the monitoring cabin is equipped with a wireless communication module that has signal strength monitoring and automatic retransmission mechanism; during the operation of the monitoring cabin, the data transmission connection with the two vibration sensors is automatically switched with the midpoint between the positions of the two adjacent vibration sensors as the dividing point.

[0018] Furthermore, the three-level decision-making process includes, sequentially, a first-level single-feature threshold determination based on a health baseline, a second-level fault suspicion determination based on a fusion score, and a third-level confirmation determination based on standardized deviation statistics.

[0019] Furthermore, the multi-scale feature fusion model is a pre-trained adversarial feature fusion generation network, including a generator and a discriminator, deployed in the processor of the monitoring cabin.

[0020] Furthermore, the output pulses of the incremental photoelectric encoder are mapped to the support numbers along the conveyor line; the support numbers are sequentially encoded along the conveying direction starting from the drive end, and a reference number is set for each preset distance. The supports between adjacent reference numbers are encoded with decimal places according to the actual distance from the drive end in meters.

[0021] The present invention also discloses a monitoring method for the above-mentioned machine vision-based conveyor condition monitoring device, comprising the following steps:

[0022] Step S1: Control the monitoring cabin to run along the guide rail, adjust the angle of the infrared thermal imager and the visual camera through the multi-degree-of-freedom gimbal, and collect infrared images of the idler roller and visual images of the conveyor belt respectively; simultaneously collect sound signals and vibration signals, and obtain timestamps and position information through incremental photoelectric encoders; during the operation of the monitoring cabin, the data transmission link is automatically switched with the midpoint of the adjacent vibration sensors as the boundary;

[0023] Step S2: Using the photoelectric encoder timestamp as a reference and the vibration signal as a benchmark, a dynamic time warping algorithm is used to achieve nonlinear time alignment of multimodal data; noise reduction, outlier correction and targeted preprocessing are performed on each modal data, and features related to temperature, texture, morphology, impact characteristics and energy entropy are extracted, and the features are normalized to the [0,1] interval according to the signal type.

[0024] Step S3: Input the normalized features into the pre-trained multi-scale feature fusion model and output the initial anomaly score; calculate the dynamic weights based on the data quality of each modality to obtain the weighted fusion score; sequentially perform the first-level health baseline threshold initial screening, the second-level fusion score fine judgment based on the preset threshold, and the third-level confirmation judgment based on the standardized deviation statistic. When a fault is confirmed, identify the main cause modality. If the threshold is not reached, mark it as a false alarm.

[0025] Step S4: Based on the mapping relationship between the photoelectric encoder position information and the bracket number, locate the fault to the specific bracket; infer the cause of the fault by associating with the fault database, and generate and output a structured report containing the bracket number, fault type, possible causes and characteristic data.

[0026] Furthermore, the dynamic time warping algorithm specifically involves: constructing a sequence of feature vectors for each mode, calculating the Euclidean distance matrix between frame pairs in the sequence, obtaining the minimum cumulative distance by solving the cumulative distance matrix through dynamic programming, and obtaining the optimal bending path by backtracking from the endpoint to the starting point, thereby achieving time synchronization of multi-source signals based on vibration signals.

[0027] Furthermore, the multi-scale feature fusion model is an adversarial feature fusion generation network, including a generator and a discriminator; the generator uses an encoder-decoder structure to map multimodal inputs into latent space feature vectors and then reconstructs them into fused features; the discriminator outputs the fault category probability distribution; and the discriminator is trained until the discriminator loss converges through an alternating optimization strategy.

[0028] Furthermore, in step S3, the healthy baseline range is determined based on the historical statistical mean and standard deviation of each feature under normal operating conditions; the preset threshold used in the secondary fusion scoring is set based on the mean and standard deviation of the primary abnormality score and the fusion score of the normal sample; the confirmation threshold for the tertiary judgment is 3, and the fault is confirmed when the absolute value of the standardized deviation is greater than 3.

[0029] Furthermore, the dynamic weight calculation method is as follows: the infrared image weight is determined based on the image occlusion ratio, the sound signal weight is determined based on the signal-to-noise ratio, and the vibration signal weight is determined based on the signal standard deviation; each weight is used for weighted fusion after normalization processing.

[0030] Furthermore, the identification conditions for false alarms are: the primary anomaly score or fusion score in the secondary judgment exceeds a preset threshold, but the absolute value of the standardized deviation of all features in the tertiary judgment is less than 3.

[0031] The present invention has the following beneficial effects:

[0032] (1) The monitoring cabin runs autonomously along the rigid guide rail and can perform reciprocating inspections of the entire conveyor line. Compared with the fixed-point sensor layout, it reduces the monitoring blind spots, and effectively detects the middle section of the long-distance conveyor.

[0033] (2) By configuring charging piles at both ends of the guide rail and combining the logic judgment of the power management unit of the monitoring cabin, automatic charging and battery life under low power conditions are realized, reducing the frequency of manual charging or battery replacement, which helps to ensure uninterrupted operation for a longer period of time.

[0034] (3) A wireless communication module with signal strength monitoring and automatic retransmission mechanism is adopted, and a data link switching strategy based on the sensor's geographical location is designed. This can maintain the continuity of data transmission during the movement of the monitoring cabin and reduce the risk of data loss due to connection switching to a certain extent.

[0035] (4) The attitude adjustment of infrared thermal imaging and visual camera is achieved by using a multi-degree-of-freedom gimbal, so that the monitoring cabin can collect information on the temperature distribution of the idler roller and the surface condition of the conveyor belt in time, integrating the two types of detection functions into a single device, reducing the repeated investment in equipment.

[0036] (5) The three-level progressive decision-making mechanism filters transient interference and data fluctuations layer by layer through initial screening with single features, fine judgment with fusion scoring and statistical confirmation with standardized deviation. Compared with the single threshold judgment method, it can reduce the false alarm rate and verify and confirm suspected faults.

[0037] (6) Based on the mapping relationship between incremental photoelectric encoder pulses and bracket numbers, the fault location can be located to the specific bracket number, reducing the troubleshooting scope from the traditional segmented approach to a point-based approach, providing a clearer location for on-site maintenance and helping to shorten the troubleshooting time.

[0038] (7) The dynamic weighting calculation mechanism can adjust the fusion strategy in real time according to data quality indicators such as infrared occlusion ratio, sound signal-to-noise ratio, and vibration signal stability, and has a certain adaptive capability in environments such as light changes and dust interference. Attached Figure Description

[0039] Figure 1 This is a structural diagram of the monitoring device of the present invention.

[0040] Figure 2 This is a structural diagram of the monitoring cabin.

[0041] Figure 3 This is a schematic diagram showing how a multi-degree-of-freedom gimbal tilts upwards to make the visual camera tilt upwards.

[0042] Figure 4 This is a schematic diagram showing how a multi-degree-of-freedom gimbal tilts downwards, causing the visual camera to point downwards.

[0043] Figure 5 This is a flowchart of the monitoring method of the present invention.

[0044] Figure 6 The flowchart shows the algorithm for the three-level decision-making mechanism.

[0045] in:

[0046] 1. Conveyor belt; 2. Drive unit; 3. Monitoring cabin; 4. Charging pile; 5. Guide rail; 30. Roller; 31. Multi-degree-of-freedom gimbal; 32. Visual camera; 33. Infrared thermal imager. Detailed Implementation

[0047] The invention will now be further described with reference to the accompanying drawings.

[0048] like Figure 1 First, the guide rail 5 on the side of the conveyor is installed. The guide rail 5 is made of rigid material and is laid along the entire conveyor direction. Its length is adapted to the length of the conveyor body, ensuring that the monitoring cabin 3 can cover the entire inspection range of the conveyor line. Charging piles 4 and position limit sensors are fixedly installed at both ends of the guide rail 5 (for ease of drawing). Figure 1 (Only one side of the charging pile 4 is shown). The charging interface of the charging pile 4 is matched with the charging end of the monitoring cabin 3. The position limit sensor is used to accurately detect the position of the running end point of the monitoring cabin 3, and provide a trigger signal for subsequent reverse operation or charging control.

[0049] The monitoring cabin 3 integrates a drive motor, rollers, a high-performance processor, a battery module, a power management unit, and a wireless communication module. The drive motor is connected to the rollers 30 via a transmission connection (e.g., ...). Figure 2 The monitoring cabin 3 is equipped with a motor-driven roller 30 that reciprocates along the guide rail 5. The battery module provides power to all electrical components of the monitoring cabin 3. The power management unit collects the remaining power data of the battery module in real time and compares it with a preset power threshold, which is calculated based on the minimum power required for a single full inspection and return to the nearest charging pile 4.

[0050] When the monitoring cabin 3 moves to the end of the guide rail 5 and triggers the position limit sensor, the power management unit immediately obtains the current power status: if the remaining power is higher than the preset threshold, the control unit instructs the drive motor to run in reverse and continue to perform the inspection task; if the remaining power is lower than the preset threshold, the control unit controls the monitoring cabin 3 to move to the charging pile 4, so that the charging interface is connected to the charging pile 4, and starts the automatic charging process. After charging to the preset target power (or full charge), the control unit instructs the monitoring cabin 3 to run in reverse and resume the inspection.

[0051] A multi-degree-of-freedom gimbal 31 is installed at the front of the monitoring cabin 3. A visual camera 32 and an infrared thermal imager 33 are both fixed to the multi-degree-of-freedom gimbal 31. A preset program controls the pitch angle adjustment of the multi-degree-of-freedom gimbal 31 to switch the detection target: when the multi-degree-of-freedom gimbal 31 pitches downwards, the infrared thermal imager 33 is aimed at the idler roller area of ​​the conveyor to collect infrared temperature images of the idler rollers (e.g., ...). Figure 4 When the multi-degree-of-freedom gimbal 31 tilts upwards, the vision camera 32 is aimed at the surface of the conveyor belt 1 (e.g., ...). Figure 3 The device is used to acquire visual images of the conveyor belt 1, thereby integrating two types of detection functions through the same multi-degree-of-freedom gimbal 31 to achieve time-sharing monitoring of the idler roller and the conveyor belt 1.

[0052] Sound sensors are installed at the bottom of monitoring compartment 3 to collect sound signals generated during the operation of the idler roller and the surrounding environment in real time.

[0053] Triaxial vibration sensors are wirelessly arranged at preset intervals on the supports along the conveyor line to collect vibration signals during the operation of the idler rollers.

[0054] An incremental photoelectric encoder is installed on the drive unit 2. This encoder is connected to the transmission components of the conveyor and can output pulse signals that reflect the operating status of the conveyor in real time, thereby obtaining accurate timestamps and position information.

[0055] The wireless communication module employs a wireless protocol with signal strength monitoring and automatic retransmission mechanisms to ensure reliable data transmission. As the monitoring cabin 3 moves along the guide rail 5, the midpoint between the positions of two adjacent vibration sensors serves as the data link switching boundary. Before reaching this boundary, the monitoring cabin 3 establishes a data transmission connection with the preceding vibration sensor. After crossing the boundary, it automatically switches to establishing a connection with the following vibration sensor. This location-based dynamic switching strategy ensures the continuity of vibration signal acquisition and avoids data transmission interruptions caused by the movement of the monitoring cabin 3.

[0056] All supports along the conveyor line are numbered according to a unified rule. The numbering starts from the drive end and proceeds sequentially along the conveying direction. A baseline number is set every 50 meters. Supports located between two baseline numbers are numbered with decimal places based on their actual distance from the drive end. For example, a support 12.3 meters from the drive end is numbered 1.123, and a support 65.7 meters from the drive end is numbered 2.157. A one-to-one mapping relationship is established between the pulse signals output by the incremental photoelectric encoder and the support number. By counting the pulses, the current support position of monitoring compartment 3 can be determined, providing a precise location reference for fault location.

[0057] Combination Figure 5 At the software algorithm execution level, the multi-scale feature fusion model is first pre-trained and solidified. Multimodal historical data of the conveyor under normal operating conditions and six types of fault states are collected. The six types of faults include bearing inner ring fault, bearing outer ring fault, rolling element fault, cage fault, conveyor belt misalignment fault, and conveyor belt tearing fault.

[0058] Continuous wavelet transform is performed on the collected sound and vibration signals to construct a time-frequency diagram. The transform formula is as follows:

[0059] ,

[0060] The time-frequency image is stacked together with the infrared image and the visual image to form a multimodal input tensor. An adversarial feature fusion generative network is constructed, which includes a generator and a discriminator. The generator adopts an encoder-decoder structure to map the multimodal input X into latent space feature vectors. Post-reconstruction into fused feature representation The generator introduces a feature-level transformation layer, which enhances feature diversity by adding Gaussian noise. Its output is:

[0061] ,

[0062] in, , .

[0063] The discriminator is a multi-classification network that receives real samples. Or generate samples Output the probability distribution of the corresponding fault category. .

[0064] Define the generator loss function as follows:

[0065] ,

[0066] in, , ;

[0067] The discriminator loss function is: .

[0068] Total loss introduces dynamic adversarial factor The dynamic adversarial factor update formula is:

[0069] ,

[0070] in, and They are respectively global and local sub-domains. distance.

[0071] The total loss formula is: .

[0072] An alternating optimization strategy was adopted, using the Adam optimizer with an initial learning rate of 0.001 and a batch size of 128. The network was trained until the discriminator loss was below the threshold of 0.001 for 100 consecutive rounds, at which point training was stopped.

[0073] After training, the test set is input into the generator to obtain the fused feature vector, which is then passed through a lightweight classifier to output a primary anomaly score. The threshold for primary anomaly scoring is determined based on statistics from normal samples. Threshold for weighted fusion scoring ,in , for The mean and standard deviation, , For weighted fusion scoring The mean and standard deviation of the trained generator and threshold are used to determine the optimal values. , It is embedded in a high-performance processor within the monitoring cabin. Simultaneously, it uses the historical average values ​​of various characteristics under normal operating conditions. with standard deviation The baseline health range for primary screening is determined as follows: .

[0074] During online monitoring, the data acquisition steps are performed first: the monitoring cabin 3 is controlled to run along the guide rail 5 at a preset speed, and the angles of the visual camera 32 and the infrared thermal imager 33 are adjusted in real time by the multi-degree-of-freedom gimbal 31 to simultaneously acquire infrared images of the idler rollers and visual images of the conveyor belt. The sound sensor acquires sound signals, the vibration sensor acquires vibration signals, and the incremental photoelectric encoder outputs timestamps and position information simultaneously. During operation, the monitoring cabin 3 automatically switches the data transmission link with the vibration sensor at preset boundary points to ensure continuous data acquisition.

[0075] After data acquisition is completed, the process enters the preprocessing stage: using the timestamp sequence output by the incremental photoelectric encoder. As a global reference time axis, the data of each modality are divided into the same detection window. Construct feature vector sequences for each modality. Let the two feature sequences to be aligned be... and ,in and Let be the lengths of the two sequences, respectively. Calculate the Euclidean distance matrix between the frame pairs of the sequences. Matrix elements:

[0076] ,

[0077] in, is the dimension of the feature vector.

[0078] Solving the cumulative distance matrix using dynamic programming Its elements Let represent the minimum cumulative distance from the starting point (1,1) to the point (i,j), initialized using the following formula:

[0079] ,

[0080] , ,

[0081] The state transition equation is:

[0082] ( and );

[0083] The optimal curved path is obtained by tracing back from the endpoint (M,N) to the starting point (1,1). ,in This achieves nonlinear time alignment of multi-source signals based on vibration signals. This path specifies the sequence... The Frames and Sequences The The correspondence between frames is established, thereby achieving non-linear alignment of two signal sequences on the time axis.

[0084] Apply time series features The rule removes transient outliers, if It is then identified as an outlier and replaced with the median of its neighborhood, where The mean of the signal. This represents the standard deviation of the signal.

[0085] The visual image is first subjected to Gaussian filtering to suppress noise, then converted to the HSV color space, and the saturation S component is extracted to reduce the impact of uneven illumination. The S component image is then adaptively binarized using the Otsu algorithm to segment the conveyor belt region. The edge point set E of the binarized image is extracted using the Canny edge detection algorithm. For the infrared image, median filtering is used for noise reduction, and the grayscale matrix is ​​converted using a first-order polynomial linear mapping. Convert to temperature matrix The mapping formula is ,in The slope coefficient, The intercept coefficient was determined through a standard temperature calibration experiment, and the idler roller region was extracted as the region of interest using a template matching method.

[0086] Zero-drift removal is achieved by subtracting the global mean from both the sound and vibration signals. After downsampling, wavelet packet decomposition is performed, yielding eight sub-vectors for each signal. The cross-correlation between each sub-vector of the sound signal and each sub-vector of the vibration signal is calculated. Signals in frequency bands exceeding a preset threshold are selected for reconstruction based on their cross-correlation values. The reconstructed signals are then subjected to variational mode decomposition. Decomposed into K eigenmode functions Its solution is obtained by minimizing a variational problem:

[0087] ,

[0088] in, for One modal component; The center frequency of each component; The original signal, For time variables, Yes The partial derivative function, Here, * represents the impulse function, and * represents the convolution operator. This represents the constraint conditions. The objective function is used to select the optimal intrinsic mode function and determine... value, Parameters such as these.

[0089] Features are extracted from the preprocessed modal data: the edge slope k and edge tilt angle are obtained by least-squares linear fitting of the edge point set E of the visual image. The average tilt angle at the same test point under normal operating conditions Define deviation characteristics as the baseline. Simultaneously construct a gray-level co-occurrence matrix. Extract contrast With entropy As a texture feature, where L is the number of gray levels in the image.

[0090] Extract the highest temperature in the region of interest of the idler roller. Minimum temperature Average temperature and temperature gradient The statistical measure, the temperature gradient, is obtained by applying the Sobel operator to the temperature matrix. Edge detection is performed to obtain the results.

[0091] The envelope is obtained by performing a Hilbert transform on each mode obtained from variational mode decomposition. Calculate kurtosis Root mean square Simultaneously, the energy of each mode with high cross-correlation in the sub-bands under wavelet packet decomposition is calculated. The energy probability distribution is obtained. Then the energy entropy of the i-th mode is All extracted features are normalized to the [0,1] interval according to signal type. The normalization formula is as follows: ,in This represents the maximum value of the characteristic of this type of signal. This is the minimum value of the characteristics of this type of signal.

[0092] like Figure 6 After feature processing is completed, the three-level decision-making process begins:

[0093] In the first-level judgment stage, the individual features of each modality are compared with the healthy baseline range. If any feature value exceeds the baseline range, the detection point is marked as initially suspicious and enters the second-level judgment stage; if all feature values ​​are within the baseline range, it is judged as normal.

[0094] In the secondary judgment stage, the normalized multimodal feature vectors are input into the pre-trained multi-scale feature fusion model to output a preliminary anomaly score. Simultaneously, dynamic weights are calculated based on the data quality of each modality, including infrared image weights. Sound signal weights Vibration signal weights ,in , , To preset the confidence constant, , , The scores for each modality are weighted and fused using the aforementioned weights to obtain a weighted fused score. ;like or If the test result is positive, the test point is considered to be suspected of having a fault and proceeds to Level 3; otherwise, it is considered normal.

[0095] In the third-level judgment stage, the standardized deviation Z-score of all features of the suspected fault detection point and the historical healthy baseline is calculated. If the absolute Z-score of a feature is greater than 3, the fault is confirmed, and the feature with the largest absolute Z-score and its mode are identified. The fault type is then associated with the fault database to determine the main fault type. If the absolute Z-score of all features is less than 3, it is marked as a false alarm, prompting for further review.

[0096] In actual fault detection scenarios, for bearing inner ring faults, the vibration signal is subjected to variational mode decomposition and Hilbert transform, and the calculated kurtosis K=6.2 is far beyond the healthy baseline range under normal working conditions. At the same time, envelope spectrum analysis shows the presence of inner ring fault characteristic frequencies. The system confirms that the fault is a bearing inner ring fault through a three-level decision-making process.

[0097] For conveyor belt misalignment faults, the edge tilt angle deviation Δθ=6° extracted from the visual image exceeds the healthy baseline range of the detection point, while the texture features show no significant abnormalities. The system, combined with a three-level decision-making mechanism, determines that it is a conveyor belt misalignment fault.

[0098] After fault confirmation, the system, combining the precise pulse signal provided by the incremental photoelectric encoder, and based on the pre-established mapping relationship between pulse signals and bracket numbers, accurately locates the fault point to a specific bracket number, such as bracket number 32 or bracket number 2.157. A structured fault report is then automatically generated and sent to the maintenance terminal via wireless network. An example report format is shown below:

[0099] Fault number: F-202501;

[0100] Stent number: No. 32;

[0101] Fault type: Idler roller bearing outer ring failure;

[0102] Possible causes: Insufficient lubrication or bearing wear;

[0103] Characteristic data: vibration kurtosis = 5.8, envelope spectrum peak appears at the outer fault characteristic frequency;

[0104] Maintenance recommendation: Immediately check the bearing of the support roller, add grease or replace the bearing.

[0105] The system will trigger tiered warnings based on the severity of the fault. For serious faults that may lead to equipment downtime or safety risks, a shutdown prompt can be sent directly to the control center so that maintenance personnel can take timely emergency repair measures to prevent the fault from escalating.

[0106] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements without departing from the principle of the present invention, and these improvements should also be considered within the scope of protection of the present invention.

Claims

1. A machine vision-based conveyor status monitoring device, the conveyor comprising a conveyor belt (1), idlers, conveyor line supports, and a drive unit (2), characterized in that: include: The monitoring cabin (3) can move back and forth autonomously along the guide rail (5) fixed on one side of the conveyor. The guide rail is equipped with charging piles (4) for charging the monitoring cabin at both ends. The multimodal data acquisition module includes an infrared thermal imager, a visual camera and a sound sensor integrated into the monitoring cabin, vibration sensors wirelessly arranged in stages along the support of the conveyor, and an incremental photoelectric encoder installed in the drive unit. The data processing and fusion module, deployed in the monitoring cabin, is used to perform time alignment, feature extraction, fusion based on a multi-scale feature fusion model, and three-level decision-making on infrared images, visual images, sound signals, and vibration signals based on the timestamps output by the incremental photoelectric encoder.

2. The conveyor status monitoring device based on machine vision as described in claim 1, characterized in that: The monitoring cabin includes a multi-degree-of-freedom gimbal (31), and the visual camera (32) and infrared thermal imager (33) are mounted together on the multi-degree-of-freedom gimbal, and are respectively aimed at the surface of the idler roller or the conveyor belt by adjusting the pitch angle.

3. The conveyor status monitoring device based on machine vision as described in claim 1, characterized in that: The guide rail (5) is equipped with position limit sensors at both ends, and the monitoring cabin is equipped with a power management unit. When the monitoring cabin runs to the end of the guide rail and triggers the position limit sensor, the power management unit controls the monitoring cabin to connect with the charging pile for charging or to run in reverse according to the current power status.

4. The machine vision-based conveyor status monitoring device as described in claim 1, characterized in that: The monitoring cabin is equipped with a wireless communication module that has signal strength monitoring and automatic retransmission mechanism; during the operation of the monitoring cabin, the data transmission connection with the two vibration sensors is automatically switched with the midpoint between the positions of the two adjacent vibration sensors as the dividing point.

5. The conveyor status monitoring device based on machine vision as described in claim 1, characterized in that: The three-level decision-making process includes, sequentially, a first-level single-feature threshold determination based on a health baseline, a second-level fault suspicion determination based on a fusion score, and a third-level confirmation determination based on standardized deviation statistics.

6. The conveyor status monitoring device based on machine vision as described in claim 1, characterized in that: The multi-scale feature fusion model is a pre-trained adversarial feature fusion generation network, which includes a generator and a discriminator, and is deployed in the processor in the monitoring cabin.

7. The conveyor status monitoring device based on machine vision as described in claim 1, characterized in that: The output pulses of the incremental photoelectric encoder are mapped to the support numbers along the conveyor line. The support numbers are sequentially encoded along the conveying direction starting from the drive end. A reference number is set for each preset distance, and the supports between adjacent reference numbers are encoded with decimal places according to the actual distance from the drive end in meters.

8. A monitoring method for a conveyor condition monitoring device based on machine vision as described in any one of claims 1-7, characterized in that: Includes the following steps: Step S1: Control the monitoring cabin to run along the guide rail, adjust the angle of the infrared thermal imager and the visual camera through the multi-degree-of-freedom gimbal, and collect infrared images of the idler roller and visual images of the conveyor belt respectively; simultaneously collect sound signals and vibration signals, and obtain timestamps and position information through incremental photoelectric encoders; during the operation of the monitoring cabin, the data transmission link is automatically switched with the midpoint of the adjacent vibration sensors as the boundary; Step S2: Using the photoelectric encoder timestamp as a reference and the vibration signal as a benchmark, a dynamic time warping algorithm is used to achieve nonlinear time alignment of multimodal data; Noise reduction, outlier correction and targeted preprocessing are performed on each modal data. Temperature, texture, morphology, impact characteristics and energy entropy related features are extracted and normalized to the [0,1] interval according to signal type. Step S3: Input the normalized features into the pre-trained multi-scale feature fusion model and output the initial anomaly score; calculate the dynamic weights based on the data quality of each modality to obtain the weighted fusion score; sequentially perform the first-level health baseline threshold initial screening, the second-level fusion score fine judgment based on the preset threshold, and the third-level confirmation judgment based on the standardized deviation statistic. When a fault is confirmed, identify the main cause modality. If the threshold is not reached, mark it as a false alarm. Step S4: Based on the mapping relationship between the photoelectric encoder position information and the bracket number, locate the fault to the specific bracket; The fault database is used to infer the cause of the fault, generate and output a structured report containing the stent number, fault type, possible causes and characteristic data.

9. The monitoring method of the conveyor status monitoring device based on machine vision as described in claim 8, characterized in that: The dynamic time warping algorithm is as follows: construct a sequence of feature vectors for each mode, calculate the Euclidean distance matrix between frame pairs in the sequence, obtain the minimum cumulative distance by solving the cumulative distance matrix through dynamic programming, and obtain the optimal bending path by backtracking from the endpoint to the starting point, thereby achieving time synchronization of multi-source signals based on vibration signals.

10. The monitoring method of the conveyor status monitoring device based on machine vision as described in claim 8, characterized in that: The multi-scale feature fusion model is an adversarial feature fusion generation network, which includes a generator and a discriminator. The generator uses an encoder-decoder structure to map the multimodal input into latent space feature vectors and then reconstruct them into fused features. The discriminator outputs a probability distribution of fault categories; it is trained until the discriminator loss converges through an alternating optimization strategy.

11. The monitoring method of the conveyor status monitoring device based on machine vision as described in claim 8, characterized in that: In step S3, the healthy baseline range is determined based on the historical statistical mean and standard deviation of each feature under normal operating conditions; the preset threshold used in the secondary fusion scoring is set based on the mean and standard deviation of the primary abnormality score and the fusion score of the normal sample; the confirmation threshold for the tertiary judgment is 3, and the fault is confirmed when the absolute value of the standardized deviation is greater than 3.

12. The monitoring method of the conveyor status monitoring device based on machine vision as described in claim 8, characterized in that: The dynamic weight calculation method is as follows: the infrared image weight is determined based on the image occlusion ratio, the sound signal weight is determined based on the signal-to-noise ratio, and the vibration signal weight is determined based on the signal standard deviation; each weight is used for weighted fusion after normalization.

13. The monitoring method of the conveyor status monitoring device based on machine vision as described in claim 11, characterized in that: The criteria for identifying false alarms are: the primary anomaly score or fusion score in the secondary judgment exceeds the preset threshold, but the absolute value of the standardized deviation of all features in the tertiary judgment is less than 3.

Images