A power plant coal conveying system operation monitoring method based on an internet of things sensing system

Abstract

The present application relates to the technical field of visual detection, in particular to a power plant coal conveying system operation monitoring method based on an Internet of Things sensing system, comprising the following steps: based on the surface image of the conveying belt collected by a multispectral camera and the idler image collected by the camera, fusing the near-infrared waveband of the surface image of the conveying belt, extracting the end face profile of the idler image, and establishing equipment operation state representation data. The present application collects the conveying belt and idler image through a multispectral camera, fuses the near-infrared waveband image, and finely extracts the idler end face profile data, thereby realizing accurate representation of the conveying system operation state; on the basis of the state representation data, the conveying belt short-time change information atlas is obtained through the difference operation of continuous time frames, and the dynamic threshold is set according to the statistical law of the pixel value in the atlas, thereby segmenting and positioning the potential longitudinal cracks.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and in particular to a method for monitoring the operation of a power plant coal conveying system based on an Internet of Things (IoT) sensing system. Background Technology

[0002] The field of visual inspection technology is a technical field that uses visual perception devices (such as multispectral cameras, CCD cameras, etc.) to acquire images of the target or region to be tested, and then uses image processing technology, including edge extraction, image enhancement, feature extraction, pattern recognition and other algorithmic methods, to detect and judge the state, quality or abnormality of the target object.

[0003] Existing technologies struggle to identify early-stage microcracks and subtle changes in operational status; furthermore, the lack of equipment spatial mapping methods obscures fault location, increasing subsequent troubleshooting costs. For instance, in actual coal conveying systems, the initial stages of microcracks are difficult to detect due to their faint image feature changes, allowing abnormal cracks to develop until they cause significant equipment failure, increasing maintenance costs and safety hazards. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method for monitoring the operation of a power plant coal conveying system based on an Internet of Things (IoT) sensor system.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a method for monitoring the operation of a power plant coal conveying system based on an Internet of Things (IoT) sensing system, comprising the following steps:

[0006] Based on the surface images of the conveyor belt acquired by the multispectral camera and the images of the idler rollers acquired by the camera, the surface images of the conveyor belt are fused in the near-infrared band, and the end face contours of the idler roller images are extracted to establish equipment operating status characterization data.

[0007] Based on the equipment operation status characterization data, differential operation is performed on the conveyor belt characterization data of continuous time frames to obtain a short-time change map of the conveyor belt surface. Based on the short-time change map of the conveyor belt surface, a dynamic threshold is set according to the statistical distribution of pixel values ​​in the map for segmentation processing to identify potential longitudinal crack growth points.

[0008] Based on the location of the potential longitudinal crack growth point and the equipment operating status characterization data, the cumulative edge gradient value of the growth point along the conveyor belt running direction is calculated, and the angular displacement of the idler roller profile in continuous time frames is calculated to obtain the key feature value of equipment failure. Based on the key feature value of equipment failure, it is compared with the dynamic threshold set according to the current running speed of the conveyor belt to generate a quantitative assessment result of failure mode.

[0009] Based on the failure mode quantitative evaluation results of continuous time frames, it is determined whether there is a monotonically increasing trend, and a continuous equipment abnormal state is established. Based on the continuous equipment abnormal state, the conveyor belt encoder position and idler camera identifier corresponding to the occurrence of the locked state, the fault location result of the conveyor system is obtained.

[0010] Preferably, the steps for obtaining the device operating status characterization data are as follows:

[0011] Based on the conveyor belt surface image acquired by the multispectral camera and the idler roller image acquired by the camera, the frame number, image number and timestamp of the two types of images are recorded respectively. For the time difference between the recorded image number and the adjacent frames in the timestamp list, the image frames with time errors less than the preset tolerance range are selected to obtain timestamp aligned image frame pairs.

[0012] Based on the timestamp-aligned image frame pairs, the image layer channel information corresponding to the near-infrared band in the conveyor belt surface image is extracted. The channel information is then processed by grayscale normalization according to the pixel position number and then superimposed. During the fusion process, the superimposed grayscale value is bound to the image coordinates to generate a fused image, thus obtaining a near-infrared band fused image.

[0013] Based on the near-infrared band fused image, and combined with the corresponding timestamp-aligned image frame to center the idler image, edge extraction is performed on the idler image to extract the edge of the idler end face. The curvature value, closure and continuous change characteristics of the main axis direction of the end face edge line are used to construct a structural feature vector associated with the idler state. The structural feature vector is combined with the pixel average brightness value and slope gradient value in the near-infrared band fused image according to the frame index number to form a multi-dimensional vector set, generating equipment operation status characterization data.

[0014] Preferably, the step of obtaining the short-time change pattern of the conveyor belt surface is as follows:

[0015] Based on the equipment operation status characterization data, the conveyor belt image sequence within the range of unified idler camera identification and synchronization timestamp is extracted. The image sequence is arranged in ascending order according to the image frame number. The pixel grayscale matrix information of each frame image is read and the shooting time difference of adjacent frame images and the conveyor belt running speed at the corresponding time point are recorded to form a continuous time frame data group.

[0016] Based on the continuous time frame data group, calculate the grayscale change index per unit movement distance between adjacent frames;

[0017] Based on the grayscale change index per unit movement distance, and combined with the pixel grayscale difference image matrix of the corresponding frame, the grayscale difference image of each frame and the corresponding grayscale change index per unit movement distance are mapped in two layers on the time axis. The grayscale image matrix is ​​expressed in a thermal mapping manner, and a time-series mapping atlas is generated using the grayscale change index sequence per unit movement distance as the index of change amplitude, forming a short-time change map of the conveyor belt surface.

[0018] Preferably, the step of obtaining the potential longitudinal crack growth point is as follows:

[0019] Based on the short-time change map of the conveyor belt surface, the gray values ​​of all pixels in the map are extracted frame by frame and a gray value histogram is constructed. The frequency of gray values ​​in each interval is counted, the probability distribution of the frequency in each interval is calculated, and the gray value probability distribution is sorted to obtain the cumulative probability curve of gray values.

[0020] Based on the gray value cumulative probability curve, the gray value corresponding to the first time the cumulative probability of gray value exceeds the 95th percentile is selected as the dynamic threshold. The dynamic threshold is applied to the short-time change map of the conveyor belt surface to perform pixel gray-level binarization frame by frame. The pixel areas above the dynamic threshold are marked as potential abnormal areas to form a potential abnormal area mask map.

[0021] Based on the mask map of potential abnormal regions, the geometric features of each region, including aspect ratio and area, are extracted. Pixel regions with aspect ratio greater than a set value, area exceeding the minimum area limit, and continuous boundaries are selected and identified as potential longitudinal crack growth points.

[0022] Preferably, the steps for obtaining the key feature values ​​of the equipment failure are as follows:

[0023] Based on the location data of the potential longitudinal crack growth points, the edge pixel values ​​in the corresponding conveyor belt surface image are extracted along the conveyor belt running direction. The difference between the gray value of each pixel and the gray value of the adjacent pixel is calculated and the absolute value is taken. The difference is accumulated point by point to form the edge gradient accumulation value of the longitudinal crack growth point location.

[0024] Based on the cumulative edge gradient value of the longitudinal crack growth point location, combined with the corresponding continuous time frame of the idler end face contour image data, the boundary point coordinates of the idler end face contour are extracted frame by frame. By fitting the ellipse center of the boundary point of the idler end face contour, the rotation angle change value of each idler end face contour center point between adjacent frames is calculated. The rotation angle change value of multiple consecutive frames is accumulated to form the angular displacement of the idler end face contour.

[0025] Based on the angular displacement of the roller end face profile and the cumulative edge gradient value of the longitudinal crack growth point location, the numerical sequence of the two parameters is combined to obtain the key characteristic value of equipment failure.

[0026] Preferably, the steps for obtaining the failure mode quantification assessment results are as follows:

[0027] Based on the key feature values ​​of equipment failure, the values ​​of each feature index in the key feature value sequence of equipment failure are extracted, and the statistics of each index value are calculated, including the maximum value, average value and variance of the feature value sequence. The feature index name corresponding to the statistics is recorded to form the statistical results of equipment failure features.

[0028] Based on the statistical results of the equipment failure characteristics, the current operating speed data of the conveyor belt is called. According to the current operating speed value of the conveyor belt, the safe value range of the historical characteristic statistical results that match the speed range is found. The limit threshold that is suitable for the current operating speed is determined and compared with the statistical quantities of each indicator of the equipment failure characteristics statistical results one by one. The statistical indicators that exceed the limit threshold are marked to form the threshold comparison judgment result.

[0029] Based on the threshold comparison results, the number of statistical indicators and the names of characteristic indicators that exceed the limit threshold are recorded. According to the number of statistical indicators and the names of characteristic indicators that exceed the dynamic threshold, the corresponding failure mode category and quantification level are determined and matched in combination with the predefined failure mode risk mapping rules to form a failure mode quantification assessment result.

[0030] Preferably, the step of obtaining the persistent abnormal state of the equipment is as follows:

[0031] Based on the failure mode quantification assessment results, a sequence of conveyor belt image frames from the past 30 consecutive minutes was selected. One image was extracted per minute to obtain a total of 30 images and timestamp information. At the same time, the failure mode quantification assessment results and the average edge intensity of each image were extracted to form a sequence dataset.

[0032] Based on the sequence dataset, calculate the failure trend enhancement response factor;

[0033] Based on the failure trend enhancement response factor, the time window consisting of 30 frames is continuously slid, and the failure trend enhancement response factor in adjacent windows is compared. If the failure trend enhancement response factor is continuously positive and exceeds the preset threshold in three consecutive windows, it is considered to form an upward trend, and the corresponding time period is marked as a continuous equipment abnormal state.

[0034] Preferably, the steps for obtaining the fault location result of the conveying system are as follows:

[0035] Based on the abnormal state period determined by the continuous abnormal state of the equipment, the conveyor belt image frame index corresponding to the abnormal state period is called, the frame acquisition timestamp recorded in the image frame index is extracted, and the real-time position information of the corresponding conveyor belt encoder is queried according to the timestamp to form the conveyor belt encoder position sequence of the abnormal state.

[0036] Based on the conveyor encoder position sequence in the abnormal state, the image frame index corresponding to the position sequence is matched one by one, the camera identification code corresponding to each image frame index is called, the camera identification code is classified and statistically analyzed according to the conveyor encoder position, the camera identification code with the highest frequency of occurrence is selected as the key camera identification in the abnormal area, and the key camera identification classification result is generated.

[0037] Based on the key camera identifiers identified in the key camera identifier classification results, and combined with the physical installation location information of the corresponding cameras, a mapping relationship between the encoder position and the camera installation position is established, the spatial location of the conveyor belt corresponding to the abnormality area is locked, and the fault location result of the conveyor system is obtained.

[0038] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0039] This invention acquires images of the conveyor belt and idlers using a multispectral camera, fuses near-infrared images, and precisely extracts the idler end-face contour data to accurately characterize the operating status of the conveyor system. Based on the state characterization data, a short-term change information spectrum of the conveyor belt is obtained through differential calculation of continuous time frames. A dynamic threshold is set according to the statistical regularity of pixel values ​​within the spectrum to segment and locate potential longitudinal cracks. Furthermore, based on the crack growth point location and state data, the cumulative gradient value of the crack edge along the conveyor belt direction is calculated, and the angular displacement change of the idler contour is identified, forming a quantitative result of key features for equipment failure. A dynamic threshold is set in conjunction with the conveyor belt running speed for judgment, capturing equipment abnormal states with a continuously rising trend. Finally, by locking the conveyor belt encoder position and idler camera marker at the time of the abnormal state, spatial location of the conveyor system fault is achieved. In summary, this invention can improve the sensitivity of conveyor equipment anomaly identification, reduce the lag in equipment anomaly detection, and enhance the accuracy of conveyor system operation monitoring. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the steps of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0042] Please see Figure 1 This invention provides a technical solution: a method for monitoring the operation of a power plant coal conveying system based on an Internet of Things (IoT) sensor system, comprising the following steps:

[0043] Based on the surface images of the conveyor belt acquired by the multispectral camera and the images of the idler rollers acquired by the camera, the surface images of the conveyor belt are fused in the near-infrared band, and the end face contours of the idler roller images are extracted to establish equipment operating status characterization data.

[0044] Based on the equipment operation status characterization data, differential operation is performed on the conveyor belt characterization data of continuous time frames to obtain the short-time change map of the conveyor belt surface. Based on the short-time change map of the conveyor belt surface, a dynamic threshold is set according to the statistical distribution of pixel values ​​in the map for segmentation processing to identify potential longitudinal crack growth points.

[0045] Based on the location of potential longitudinal crack growth points and the equipment operating status characterization data, the cumulative edge gradient value of the growth points along the conveyor belt running direction is calculated, and the angular displacement of the idler roller profile in continuous time frames is calculated to obtain the key feature value of equipment failure. Based on the key feature value of equipment failure, it is compared with the dynamic threshold set according to the current running speed of the conveyor belt to generate the quantitative assessment result of failure mode.

[0046] Based on the failure mode quantitative assessment results of continuous time frames, it is determined whether there is a monotonically increasing trend, and a continuous abnormal equipment state is established. Based on the continuous abnormal equipment state, the corresponding conveyor belt encoder position and idler camera mark when the lock state occurs, the fault location result of the conveyor system is obtained.

[0047] The steps for obtaining equipment operating status characterization data are as follows:

[0048] Based on the conveyor belt surface image acquired by the multispectral camera and the idler roller image acquired by the camera, the frame number, image number and timestamp of the two types of images are recorded respectively. For the time difference between the recorded image number and the adjacent frames in the timestamp list, the image frames with time errors less than the preset tolerance range are selected to obtain timestamp aligned image frame pairs.

[0049] Based on timestamp-aligned image frame pairs, image layer channel information corresponding to the near-infrared band in the conveyor belt surface image is extracted. The channel information is then processed by grayscale normalization according to pixel bit number and then superimposed. During the fusion process, the superimposed grayscale value is bound to the image coordinates to generate a fused image, thus obtaining a near-infrared band fused image.

[0050] Based on near-infrared band fused images, and combined with corresponding timestamp-aligned image frames to center the idler roller images, edge extraction is performed on the idler roller images. The edge of the idler roller end face is extracted, and the curvature value, closure, and continuous change characteristics of the main axis direction of the end face edge line are used to construct a structural feature vector associated with the idler roller state. The structural feature vector is then merged and arranged with the pixel average brightness value and slope gradient value in the near-infrared band fused image according to the frame index number into a multi-dimensional vector set to generate equipment operation status characterization data.

[0051] Specifically, based on the conveyor belt surface image acquired by the multispectral camera and the idler roller image acquired by the camera, the system automatically records the frame number, image number, and acquisition timestamp for each frame of the conveyor belt surface image. Similarly, it also records the frame number, image number, and acquisition timestamp for each frame of the idler roller image. Then, in order to match conveyor belt images and idler roller images acquired at similar times, the system iterates through the timestamp list of one type of image (e.g., the conveyor belt surface image), and for each timestamp in the list... Check if a timestamp exists in the timestamp list of another type of image (idler image). Make the absolute value of the time difference between the two Less than a preset time tolerance This time tolerance The settings need to consider the difference in sampling frequencies between the two cameras and the data transmission latency within the system. For example, if the conveyor belt surface image camera has a frame rate of 50 frames per second (i.e., each frame interval is 20 milliseconds) and the idler roller image camera has a frame rate of 25 frames per second (i.e., each frame interval is 40 milliseconds), considering the maximum possible sampling asynchrony and the approximately 5 millisecond latency introduced by data buffering and network transmission, the settings can be adjusted accordingly. The value is set to 15 milliseconds, and the calculation method is as follows: For simplicity, 15ms is used here. The filtering criteria are adjusted to: if the timestamp of the conveyor belt surface image is... Then look for the timestamp in the idler roller image. If multiple images are found within the interval, the idler image with the smallest time difference is selected. If no image is found, the conveyor belt surface image frame will not participate in subsequent pairing. All conveyor belt image frames that meet this time difference condition are combined with the corresponding idler image frames to form a timestamp-aligned image frame pair containing the conveyor belt image, the idler image, and their respective original timestamps and frame numbers.

[0052] Based on timestamp-aligned image frame pairs, image data of a specific near-infrared band is extracted from the multispectral image of the conveyor belt surface in each pair. For example, the image layer channel information corresponding to the near-infrared band with a center wavelength of 850 nm and a bandwidth of 40 nm is selected. This band is chosen because it is more sensitive to defects such as aging and early cracks in the conveyor belt material. Then, for each pixel of the extracted single-band near-infrared image, grayscale normalization is performed according to its position index (i.e., row and column coordinates) in the image. This normalization process linearly maps the pixel grayscale value of the near-infrared channel in the current frame from its original range (e.g., 0 to 4095 for a 12-bit image) to the standard 8-bit grayscale range of 0 to 255. The calculation process is as follows: first, all pixels of the near-infrared channel in the current frame are traversed to find its minimum grayscale value. and maximum grayscale value Then for any pixel The original grayscale value Its normalized gray value Through formula Calculations show that if Then the normalized grayscale value of all pixels is set to 0 or 127, and the normalized grayscale value of each pixel after processing is... Its corresponding image coordinates The two components are directly bound together to form a new single-channel grayscale image. This normalized single-band near-infrared image is the near-infrared band fused image.

[0053] Based on near-infrared band fusion images, and combined with the corresponding idler roller images in each timestamp-aligned image frame pair, edge detection is first performed on the idler roller images to extract the end face contours of the idler rollers. Specifically, the Canny edge detection algorithm can be used, with its low threshold... and high threshold Based on experience, settings can be adjusted, for example, if the overall contrast of the idler roller image is high, the settings can be adjusted accordingly. and Alternatively, an optimal global threshold can be dynamically calculated by applying the OTSU (Otsu method) to the image's grayscale histogram, and then the high and low thresholds can be set as a multiple of this optimal threshold, for example... To be the optimal threshold, The threshold is set to 0.5 times the optimal threshold. After edge detection, an approximately circular or elliptical closed contour representing the idler roller end face is identified and extracted from the edge pixels using Hough circle transform or ellipse fitting algorithms. For the extracted idler roller end face edge contour, its geometric feature parameters are calculated, including the average and maximum curvature of each point on the contour, the closure of the contour (e.g., by calculating the distance between the start and end points of the contour; if it is less than a preset number of pixels, such as 5 pixels, it is considered closed, and the closure parameter is assigned a value of 1; otherwise, it is 0; or the ratio of the contour perimeter to a certain covered area is calculated), and the angle of the principal axis direction of the contour determined by principal component analysis. These calculations are then used to obtain... The curvature value, closure measure, and main axis direction angle are combined into a numerical vector to form a structural feature vector associated with the current idler roller state. At the same time, the global pixel average brightness value is calculated from the corresponding near-infrared band fused image, and the average gradient magnitude of the image is calculated using the Sobel operator as the slope gradient value. Finally, the aforementioned idler roller structural feature vector containing curvature, closure, and main axis direction is merged with the pixel average brightness value and slope gradient value extracted from the near-infrared band fused image according to the unified index number of the current image frame to form a multi-dimensional feature vector. The set of multi-dimensional feature vectors of all frames is the equipment operating status characterization data.

[0054] The steps for obtaining the short-time variation spectrum of the conveyor belt surface are as follows:

[0055] Based on the equipment operation status characterization data, the conveyor belt image sequence within the range of unified idler camera identification and synchronization timestamp is extracted. The image sequence is arranged in ascending order according to the image frame number. The pixel grayscale matrix information of each frame image is read and the shooting time difference of adjacent frame images and the corresponding conveyor belt running speed are recorded to form a continuous time frame data group.

[0056] Based on the continuous time frame data group, the grayscale change index per unit movement distance between adjacent frames is calculated using the following formula:

[0057] ;

[0058] in, For the first The grayscale change index per unit movement distance corresponding to frame grouping (unit: grayscale value / meter). The number of pixels in the vertical direction of each frame of the image. The number of pixels in the horizontal direction of each frame of the image. For the first The first frame of the image Line number The pixel grayscale value of the column. For the first Frame and the The time interval between frames (in seconds). For the first The conveyor belt speed (in meters per second) corresponding to the frame image. It is a very small constant, used to avoid when When the denominator is zero (unit: meters);

[0059] Based on the grayscale change index per unit movement distance, and combined with the pixel grayscale difference image matrix of the corresponding frame, the grayscale difference image of each frame and the corresponding grayscale change index per unit movement distance are mapped in two layers on the time axis. The grayscale image matrix is ​​expressed in the form of thermal mapping, and a time-series mapping atlas is generated with the grayscale change index sequence per unit movement distance as the index of change amplitude, forming a short-time change map of the conveyor belt surface.

[0060] Specifically, based on equipment operation status characterization data, which includes frame numbers, image numbers, and millisecond-accurate capture timestamps for each frame of conveyor belt surface image (i.e., near-infrared band fused image) and idler roller image, as well as idler roller structural feature vectors and average pixel brightness and slope gradient values ​​of the conveyor belt surface image obtained through image analysis, all data collected by a specific camera are first filtered out according to the "unified idler roller camera identifier." Then, within this data, the conveyor belt surface images are further filtered according to the "synchronization timestamp range." The synchronization timestamp range here is dynamically set; for example, the system can set a 5-second sliding time window, or define an observation segment based on the distance the conveyor belt has traveled (e.g., every 10 meters) fed back by the conveyor belt encoder, ensuring that the selected image sequence corresponds to a continuous physical segment on the conveyor belt within the camera's field of view. The system processes the complete or partial transit process, arranging the selected conveyor belt surface images (i.e., near-infrared band fused images) in ascending order according to their original "image frame numbers" to ensure the temporal continuity of the images. For each frame in this ordered image sequence, the system directly reads its complete pixel grayscale matrix information from the equipment operation status characterization data and records the shooting time difference between the frame and its subsequent image. This time difference is obtained by querying and calculating the difference between the shooting timestamps recorded by the two frames. At the same time, the system obtains the instantaneous running speed of the conveyor belt at the shooting time point of each frame from an external conveyor belt speed monitoring system (e.g., a speed encoder connected to the conveyor belt drive motor or driven wheel), and integrates the pixel grayscale matrix of these images, the time difference between adjacent frames, and the corresponding conveyor belt running speed to form a continuous time frame data group.

[0061] formula: The advantage of this formula lies in its ability to provide a quantitative indicator of the severity of surface changes independent of the conveyor belt's operating speed by normalizing the average pixel grayscale variation between image frames to a unit conveying distance. This allows for a more accurate reflection of the true rate of change in the conveyor belt's surface condition, avoiding misjudgments caused by speed fluctuations. For example, even with a constant surface wear rate, the difference between frames will be greater in a short period when the speed is high, and vice versa when the speed is low. This formula addresses this by dividing by... (i.e., the inter-frame conveyor belt travel distance) eliminates this effect, making The value can directly compare surface changes at different speeds; the parameter The introduction ensures that when the conveyor belt is stationary ( In rare cases where the calculated distance traveled is zero due to extremely low speeds, the denominator will not be zero, thus ensuring the robustness of the calculation.

[0062] The steps to obtain the parameters are as follows:

[0063] This represents the number of pixels in the vertical direction of each frame of the image. This value is determined by the hardware specifications and configuration of the multispectral camera used to acquire the images. During system initialization or camera parameter configuration, the camera's image sensor resolution is set. For example, if an area scan camera is selected, its sensor has 1024 vertical pixels. The value is 1024, which is fixed during image acquisition unless the camera configuration changes. It is obtained by directly reading the metadata of the camera's output image or the resolution parameter in the preset camera configuration file. For example, for an image with a resolution of 1280x1024... Pixel.

[0064] The steps to obtain the parameters are as follows:

[0065] This represents the number of pixels in the horizontal direction of each frame of the image, and... Similarly, this value is also determined by the hardware specifications of the multispectral camera that acquires the images and its current configuration; for example, if the camera sensor has 1280 horizontal pixels, then... The value is 1280, and this value remains fixed during image acquisition. It can be obtained by consulting the camera specifications, reading the metadata of the camera's output image, or the camera resolution parameters set in the system configuration file. For example, for an image with a resolution of 1280x1024... Pixel.

[0066] The steps to obtain the parameters are as follows:

[0067] Representing the In the frame image, located at the first line, number The grayscale values ​​of the pixels in the column are derived from the "near-infrared band fused image" in the "continuous time frame data group" generated in the previous step. Their values ​​typically range from 0 to 255 after normalization (for an 8-bit grayscale image). The grayscale value of each pixel is directly obtained from the image data according to coordinates. Read, for example, in the first In the frame image, the pixel at coordinates (100, 200) has a grayscale value of 150. .

[0068] The steps to obtain the parameters are as follows:

[0069] Indicates the first Frame image and the first The time interval between frames, in seconds, is obtained from the "Continuous Time Frame Data Group," specifically by querying the [number]th frame. The frame's capture timestamp minus the first frame The capture timestamp of each frame is calculated. The capture timestamp is recorded by the multispectral camera when acquiring the image, accurate to milliseconds or higher. For example, if the frame's timestamp is... The frame's capture timestamp is "10:00:01.200", the first... If the frame's capture timestamp is "10:00:01.240", then... .

[0070] The steps to obtain the parameters are as follows:

[0071] Indicates that during the filming of the first In the frame image, the instantaneous operating speed of the conveyor belt, in meters per second, is measured. This data originates from the real-time readings of a speed sensor integrated with the conveyor system (such as a rotary encoder mounted on the drive roller or tensioner). The speed sensor converts the detected pulse count into a speed value and provides it to the monitoring system via a data interface (such as a PLC or industrial bus). The frame image's capture timestamp is used to query the corresponding conveyor belt speed record at that moment. For example, by querying the speed record closest to the timestamp "10:00:01.200", the following can be obtained: meters per second.

[0072] The steps to obtain the parameters are as follows:

[0073] It is a very small positive constant, for example, rice.

[0074] Calculation process:

[0075] Given two consecutive images with the following parameters:

[0076] Pixel;

[0077] Pixel;

[0078] Consider only one Let's illustrate with a small region. For example, the grayscale values ​​of these two images in this small region are as follows:

[0079] No. Pixel grayscale values ​​of a frame image (Value range 0-255):

[0080] ;

[0081] No. Pixel grayscale values ​​of a frame image (Value range 0-255):

[0082] ;

[0083] Time interval between adjacent frames Seconds (corresponding to a sampling rate of 25 frames per second);

[0084] No. Conveyor belt running speed corresponding to the frame image meters per second;

[0085] Minimal constant rice;

[0086] Calculation process:

[0087] Calculate the sum of the absolute values ​​of the differences between corresponding pixel grayscale values ​​in two frames of images. :

[0088] For the example area:

[0089] ;

[0090] ;

[0091] ;

[0092] ;

[0093] Sum of absolute differences (for (Region) = .

[0094] For example, for the entire The image, the sum of the absolute differences in pixel gray levels calculated is .

[0095] Calculate the average absolute difference in grayscale values ​​per pixel:

[0096] Average difference = ;

[0097] Average difference = (Grayscale unit);

[0098] Calculate the denominator :;

[0099] Denominator = ;

[0100] Denominator = ;

[0101] Calculate the grayscale change index per unit movement distance :

[0102] ;

[0103] This result indicates that in the first Frame (i.e., the first) Frame and the Between frames, for every meter the conveyor belt surface moves, the average pixel grayscale value of the image changes by approximately 12.5 units. The value is an important basis for the subsequent generation of the "short-term change map of the conveyor belt surface".

[0104] Based on the grayscale change index sequence per unit movement distance calculated in the previous step, and the pixel grayscale difference image matrix corresponding to each frame (or frame pair) generated synchronously during the calculation of this index (each pixel value in this matrix represents the absolute change in grayscale of the later frame relative to the previous frame at the same spatial location), the system begins to construct a short-time change map of the conveyor belt surface. Specifically, for each time point... (representing the first) Frame and the Between frames, the pixel grayscale difference image matrix corresponding to the time point is visualized by converting it into a heatmap. When generating the heatmap, the magnitude of the pixel grayscale difference is mapped to different colors; for example, areas with small differences are displayed as cool tones (such as blue), and areas with large differences are displayed as warm tones (such as red). The color transition range is dynamically adjusted based on the minimum and maximum differences in the current batch of images (such as images from the past minute), or a fixed mapping range, such as 0 to 50 grayscale level differences, is used to ensure the comparability of different time-series images. Simultaneously, the grayscale change index per unit movement distance associated with the frame pair is also included. The numerical value serves as a key attribute or label for the heatmap on the timeline. Each heatmap in this series, arranged chronologically, reflects the "spatial distribution" of changes on the conveyor belt surface at a specific moment, and is associated with... The value sequences then constitute a "time series of magnitude" of change. These heatmap sequences are then compared with... Combining value sequences, for example, The value series is overlaid as a line chart below or next to the heatmap series, or using... The value is used to adjust the overall brightness or contrast of the corresponding heatmap (e.g., The larger the value, the higher the color saturation or the stronger the contrast of the heat map, ultimately forming a comprehensive visualization result that can simultaneously show the spatial location and temporal intensity trend of changes on the conveyor belt surface, namely, the short-term change map of the conveyor belt surface.

[0105] The steps for obtaining potential longitudinal crack growth points are as follows:

[0106] Based on the short-time change map of the conveyor belt surface, the gray values ​​of all pixels in the map are extracted frame by frame and a gray value histogram is constructed. The frequency of gray values ​​in each interval is counted, the probability distribution of the frequency in each interval is calculated, and the gray value probability distribution is sorted to obtain the cumulative probability curve of gray values.

[0107] Based on the cumulative probability curve of gray values, the gray value corresponding to the first time the cumulative probability of gray values ​​exceeds the 95th percentile is selected as the dynamic threshold. The dynamic threshold is applied to the short-time change map of the conveyor belt surface to perform pixel gray-level binarization frame by frame, and the pixel areas above the dynamic threshold are marked as potential abnormal areas to form a mask map of potential abnormal areas.

[0108] Based on the mask map of potential abnormal regions, the geometric features of each region, including aspect ratio and area, are extracted. Pixel regions with aspect ratio greater than a set value, area exceeding the minimum area limit, and continuous boundaries are selected as potential longitudinal crack growth points.

[0109] Specifically, based on the short-time variation map of the conveyor belt surface, this map consists of a series of heat maps arranged in chronological order. Each heat map represents a pixel gray-level difference image matrix at a given time point and is associated with a corresponding gray-level change index per unit distance traveled. The system processes these heatmaps frame by frame. For each heatmap frame (the underlying grayscale difference image matrix data, where pixel values ​​represent the magnitude of grayscale changes), it first iterates through all pixels in the image, extracting the grayscale value of each pixel (here, grayscale value refers to the pixel value of the difference image, ranging from 0 to 255, representing the degree of change). Then, it constructs a grayscale histogram for that frame, with the horizontal axis representing the grayscale value and the vertical axis representing the number of pixels with that grayscale value. The grayscale value intervals are set according to the actual situation; for example, for a grayscale range of 0-255, it can be divided into 256 intervals, each... The interval width is 1, or it can be divided into fewer intervals, such as 32 intervals, each with a width of 8. The frequency of pixel occurrence (i.e., the number of pixels) in each preset grayscale value interval is counted. Then, the frequency of each interval is divided by the total number of pixels in the frame image to obtain the probability of grayscale value occurrence in that interval, forming a probability distribution of the frequency of each interval. Finally, the probabilities of these grayscale value intervals are accumulated in ascending order of grayscale value to calculate the cumulative probability corresponding to each grayscale value. These cumulative probability values ​​and the corresponding grayscale values ​​are plotted as a curve, which is the cumulative probability curve of grayscale value of the frame image.

[0110] Based on the cumulative probability curve of grayscale values ​​for each frame obtained in the previous paragraph, a dynamic threshold is determined for each frame. Specifically, on the cumulative probability curve of the grayscale values ​​for that frame, starting from the low end of the grayscale value range, the first grayscale value whose cumulative probability exceeds or equals 95% is found, and this grayscale value is set as the dynamic segmentation threshold for the current frame. For example, if the cumulative probability corresponding to grayscale value 180 is 94.8%, and the cumulative probability corresponding to grayscale value 181 is 95.1%, then the dynamic threshold is set to 181. This 95th percentile selection is based on experience and aims to exclude the vast majority of background or insignificantly changing pixels in the image, while retaining those grayscale values. For a few pixels with more drastic changes, the calculated dynamic threshold is then applied to the original grayscale difference image (i.e., the original data of the heatmap) in the short-term change spectrum of the conveyor belt surface corresponding to that frame. Each pixel in the image is binarized: if the grayscale value (difference) of a pixel is greater than this dynamic threshold, the pixel is marked as 1 (or white), indicating that the pixel belongs to a potential abnormal region; if the grayscale value of a pixel is less than or equal to this dynamic threshold, the pixel is marked as 0 (or black), indicating that the pixel belongs to a normal region. After this processing, the original grayscale difference image is converted into a binary image, in which the area marked as 1 constitutes the potential abnormal region mask map.

[0111] Based on the frame-by-frame potential anomaly region mask generated in the previous step, this mask marks regions where pixel grayscale changes exceed the dynamic threshold. Next, geometric feature analysis is performed on each connected potential anomaly region (i.e., a connected component consisting of pixels with a value of 1) in these masks. First, a connected component labeling algorithm is used to identify all independent white regions in the image. For each identified independent region, its minimum bounding rectangle is calculated, and the aspect ratio of the region is calculated based on the rectangle's length and width. Simultaneously, the total number of pixels contained in the region is calculated as its area. Next, these extracted geometric features are filtered; the aspect ratio setting needs to be based on the typical shape of longitudinal cracks on actual conveyor belts. The aspect ratio is determined by various factors. For example, analysis of historical crack images reveals that longitudinal cracks are typically elongated, with their length significantly greater than their width. Therefore, the aspect ratio can be set to a value greater than 5, such as 8. This means that only areas with an aspect ratio greater than 8 are retained. The minimum area limit is set to filter out excessively small areas caused by noise or minor irrelevant variations. This value can be estimated based on the conveyor belt width and camera resolution. For instance, if the minimum physical length of the crack to be detected is 5 cm, and each pixel in the image represents 0.1 cm, then the minimum length corresponds to 50 pixels. Considering that the crack width is at least a few pixels, the minimum area limit can be set to... Square pixels, and boundary continuity is determined by checking whether the boundary pixels of the region form one or a few closed or nearly closed contours. The connectivity within the region is usually guaranteed by the connected component labeling algorithm itself. For the screening conditions, a region must simultaneously meet the following: its aspect ratio is greater than the set aspect ratio threshold (e.g., 8), its area exceeds the set minimum area limit (e.g., 150 square pixels), and its boundary shows a certain continuity on the image (excluding overly fragmented or scattered point sets). Pixel regions that meet all these conditions are finally identified as potential longitudinal crack growth points.

[0112] The steps for obtaining key characteristic values ​​of equipment failure are as follows:

[0113] Based on the location data of potential longitudinal crack growth points, the edge pixel values ​​in the corresponding conveyor belt surface image are extracted along the conveyor belt running direction. The difference between the gray value of each pixel and the gray value of the adjacent pixel is calculated and the absolute value is taken. The difference is accumulated point by point to form the edge gradient accumulation value of the longitudinal crack growth point location.

[0114] Based on the cumulative edge gradient value of the longitudinal crack growth point location, combined with the corresponding continuous time frame of idler end face contour image data, the boundary point coordinates of the idler end face contour are extracted frame by frame. By fitting the ellipse center of the boundary point of the idler end face contour, the rotation angle change value of each idler end face contour center point between adjacent frames is calculated. The rotation angle change value of multiple consecutive frames is accumulated to form the angular displacement of the idler end face contour.

[0115] Based on the angular displacement of the roller end face profile and the cumulative edge gradient of the longitudinal crack growth point location, the numerical sequence of the two parameters is combined to obtain the key characteristic values ​​of equipment failure.

[0116] Specifically, based on the location data of potential longitudinal crack growth points determined in the previous stage, which typically includes the center coordinates of the growth point in the image or its contour pixel set, the system first needs to determine the running direction of the conveyor belt in the image. This can be achieved by analyzing the displacement of feature points in multiple consecutive frames of images or by pre-calibrating according to the geometric relationship of the camera installation. Once the running direction is determined (e.g., the vertical direction of the image is upward, representing the forward movement of the conveyor belt), for each potential longitudinal crack growth point, the system will extract one or more lines along this running direction from the corresponding original conveyor belt surface image (i.e., the "near-infrared band fused image" mentioned above) within the region of the growth point or its vicinity. The grayscale values ​​of pixels on a line segment are calculated. The length of this line segment can be set according to the size of the potential crack growth point, for example, 1.2 times its length. The width can be several pixels (e.g., 3 to 5 pixels) for averaging. For each pixel on this (or these) line segment, the absolute value of the difference between its grayscale value and the grayscale value of its previous neighboring pixel on the line segment is calculated. This difference represents the severity of the local grayscale change, i.e., the edge intensity. The absolute values ​​of all the grayscale differences between all the adjacent pixels calculated along this line segment (or the average of multiple line segments) are summed to obtain a total value. This total value is the cumulative edge gradient value at the location of the potential longitudinal crack growth point.

[0117] Based on the cumulative edge gradient values ​​of each potential longitudinal crack growth point obtained from the aforementioned calculations, and combined with the idler image data of corresponding continuous time frames extracted from the "equipment operation status characterization data" (especially the idler end face contours obtained after edge extraction processing), the dynamic behavior of the idler is analyzed. For each frame of the idler image in the time series, the set of boundary point coordinates constituting the idler end face contour is first extracted from the existing idler end face edge pixel data. Using these boundary point coordinates, an optimal ellipse is fitted using the least squares method to accurately represent the end face contour of the idler, and the coordinates of the center point of the fitted ellipse are calculated. And the angles along the major and minor axes or principal axes of the ellipse, for two consecutive frames (e.g., the first...). Frame and the (frames), respectively obtaining the center points of the two ellipses. and and the two principal axis direction angles and If the idler roller is rotating normally and the camera is fixed, the center of its end-face ellipse should remain relatively stable or only slightly displaced in the image. The principal axis angle will change as the idler roller rotates. Here, we calculate the change in rotation angle of the center point of the idler roller's end-face profile between adjacent frames. This typically involves comparing the angular position changes of specific markers on the roller surface (if visible and stable) in consecutive frames, or, in the absence of obvious markers, analyzing the rotation angle of the entire profile (e.g., if there are asymmetrical wear or deposits on the roller end-face causing profile feature points, the angular displacement of these feature points can be traced, or the overall rotation angle can be estimated using more complex image registration techniques). If the principal axis angle is used, then the calculation... ,this This refers to the rotation angle change value between single frames. Then, the rotation angle change values ​​of these single frames are accumulated and summed over multiple consecutive frames (for example, a preset time window, such as the time from when a point on the conveyor belt enters the camera's field of view to when it leaves the field of view, or a fixed number of frames, such as 10 frames) to obtain the total angular displacement of the roller end face profile within a time period.

[0118] Based on the angular displacement sequence of the idler end face profile calculated in the aforementioned steps (each time point or each small time window corresponds to an angular displacement value), and the sequence of edge gradient accumulation values ​​calculated for the potential longitudinal crack growth point location on the conveyor belt surface within the same time frame or time window (if there are multiple growth points in the same frame, their maximum value, average value, or separate records can be taken), the system combines these two parameter value sequences from different sources but synchronized in time. Specifically, the combination method can be to directly store the two sequences side by side to form a two-dimensional or higher-dimensional feature vector sequence, or to align them by timestamps to form a record set containing the changes of these two parameters over time. Each record in this set contains the idler angular displacement and crack edge gradient accumulation value at a specific moment. This entire sequence of combined parameter value sequences arranged in chronological order constitutes the key characteristic value of equipment failure.

[0119] The steps for obtaining the failure mode quantification assessment results are as follows:

[0120] Based on the key feature values ​​of equipment failure, the values ​​of each feature index in the key feature value sequence of equipment failure are extracted, and the statistics of each index value are calculated, including the maximum value, average value and variance of the feature value sequence. The feature index name corresponding to the statistics is recorded to form the statistical results of equipment failure features.

[0121] Based on the statistical results of equipment failure characteristics, the current operating speed data of the conveyor belt is called. According to the current operating speed value of the conveyor belt, the safe value range of the historical characteristic statistical results that match the speed range is found. The limit threshold that is suitable for the current operating speed is determined and compared with the statistical quantities of each indicator of the equipment failure characteristic statistical results one by one. The statistical indicators that exceed the limit threshold are marked to form the threshold comparison judgment result.

[0122] Based on the threshold comparison results, the number of statistical indicators and the names of characteristic indicators that exceed the limit threshold are recorded. According to the number of statistical indicators and the names of characteristic indicators that exceed the dynamic threshold, the corresponding failure mode category and quantification level are determined and matched in combination with the predefined failure mode risk mapping rules to form the failure mode quantitative assessment result.

[0123] Specifically, based on the key feature values ​​of equipment failure obtained in the previous step, this is a sequence of combined parameter values ​​arranged in chronological order, including "angular displacement of the idler roller end face profile" and "cumulative edge gradient value of potential longitudinal crack growth point location". First, extract the numerical lists of these two feature indicators from this sequence. For example, obtain the time series of an angular displacement value. A time series of marginal gradient accumulation values ,in It is the length of the sequence, corresponding to a specific observation time window. Then, statistical analysis is performed independently on the numerical sequence of each feature indicator to calculate its value within this time window. The maximum value at each observation point (e.g., the maximum angular displacement in the angular displacement sequence). The maximum cumulative gradient value in the sequence of marginal gradient cumulative values ), average (e.g., the average of an angular displacement sequence) The average value of the cumulative edge gradient sequence ), and variance values ​​(e.g., the variance of the angular displacement sequence). Variance of the cumulative marginal gradient sequence The calculated statistics (maximum value, average value, variance) are associated with their corresponding original feature index names ("angular displacement of roller end face profile", "cumulative edge gradient value of potential longitudinal crack growth point location") and recorded to form the statistical results of equipment failure characteristics.

[0124] Based on the equipment failure characteristic statistics obtained in the previous paragraph, which include the maximum, average, and variance values ​​of, for example, the "angular displacement of the idler roller end face profile," and the maximum, average, and variance values ​​of the "cumulative edge gradient value of potential longitudinal crack growth point location," the system next retrieves the current operating speed data of the conveyor belt from the conveyor belt speed monitoring system in real time, for example, if the current speed is 2.8 m / s. Then, the system accesses a pre-established historical characteristic statistical database or rule base. This database stores the numerical ranges of various equipment failure characteristic statistical indicators under normal or safe operating conditions in different conveyor belt operating speed ranges. These historical safe numerical ranges are obtained by collecting a large amount of key equipment failure characteristic value data through long-term monitoring of the conveyor system under normal operating conditions, and performing segmented statistical analysis according to conveyor belt speed. For example, for the speed range of 2.5 to 3.0 m / s, historical data shows that the normal average value of the "angular displacement of the idler roller end face profile" is usually between 0.5 and 1.5 degrees / second, with a maximum value not exceeding 3 degrees / second and a variance not exceeding 0.25 degrees / second. The average value of the "cumulative edge gradient at potential longitudinal crack growth point locations" should be below 5000, the maximum value below 10000, and the variance below [missing value]. The system finds the corresponding speed range (2.5 to 3.0 m / s) based on the current conveyor belt running speed (2.8 m / s) and extracts the upper or lower limit of the safe value range of all relevant characteristic statistical indicators within this range as the current judgment threshold. For example, the threshold for the average value of "angular displacement of the roller end face profile" is 1.5 degrees / second. The system compares each statistical indicator in the currently calculated equipment failure characteristic statistical results (such as the current average angular displacement of 1.8 degrees / second) with these threshold values ​​determined from historical data and adapted to the current running speed. If the value of a certain statistical indicator exceeds its corresponding threshold range (for example, 1.8 degrees / second > 1.5 degrees / second), the statistical indicator is marked as abnormal, and all marked statistical indicators and their exceeding conditions are recorded to form the threshold comparison judgment result.

[0125] Based on the threshold comparison results generated in the previous step, the system clearly identifies which equipment failure characteristic statistical indicators have exceeded the current threshold set according to the current conveyor belt operating speed. The system first summarizes and records the number of all statistical indicators exceeding the threshold, along with their specific names. For example, if three statistical indicators are found to be exceeding the limit: the average value of the "angular displacement of the idler roller end face profile," the maximum value of the "cumulative edge gradient value at the potential longitudinal crack growth point location," and the average value of the "angular displacement of the idler roller end face profile," then the system queries a predefined failure mode risk mapping rule table based on the number of these exceeding statistical indicators and which specific indicators are exceeding the limit. This rule table is established based on expert experience, equipment maintenance manuals, and historical fault data analysis. It associates different combinations of exceeding indicators with specific equipment failure modes and their risk levels. For example, the rule table defines that if only the average value of the "cumulative edge gradient value at the potential longitudinal crack growth point location" and the maximum value of the "angular displacement of the idler roller end face profile" exceed the threshold, then the system will fail to achieve the desired failure mode risk level. If the maximum value exceeds the limit, it may correspond to "early wear or minor cracks on the conveyor belt surface", with a risk level of "low". If the average value and variance of "angular displacement of the idler end face profile" both exceed the limit, and the relevant indicators of "cumulative value of edge gradient at potential longitudinal crack growth point location" also exceed the limit, it may correspond to "idler jamming or damage causing severe scratches on the conveyor belt or a serious risk of crack propagation", with a risk level of "high". The system searches for and matches the most matching entry in the risk mapping rule table based on the number and name combination of the indicators that actually exceed the limit (i.e., the three indicators and their names recorded previously), thereby determining the most likely failure mode category of the current conveyor system (such as "longitudinal tear risk of conveyor belt" or "belt surface damage caused by idler failure") and the quantitative risk level of the failure mode (such as represented by levels 1-5, where 1 is the lowest risk and 5 is the highest risk, or directly described as "warning", "general", "serious", etc.), and finally forms the quantitative assessment result of the failure mode in this evaluation.

[0126] The steps for obtaining persistent equipment abnormal status are as follows:

[0127] Based on the failure mode quantitative assessment results, a sequence of conveyor belt images from the past 30 consecutive minutes was selected. One image was extracted per minute to obtain a total of 30 images and timestamp information. At the same time, the failure mode quantitative assessment results and the average edge intensity of each image were extracted to form a sequence dataset.

[0128] Based on the sequence dataset, the failure trend enhancement response factor is calculated using the following formula:

[0129] ;

[0130] in, Enhanced response factor for failure trends within 30 data frames. For the first Average edge intensity of the frame image For the first Failure mode quantification evaluation results of frame images, , representing the total number of frames. This represents the timestamp of the first frame of the image in the sequence. This represents the timestamp of the 30th frame of the sequence.

[0131] Based on the failure trend enhancement response factor, the time window consisting of 30 frames is continuously slid, and the failure trend enhancement response factor in adjacent windows is compared. If the failure trend enhancement response factor is continuously positive and exceeds the preset threshold for three consecutive windows, it is considered to form an upward trend, and the corresponding time period is marked as a continuous abnormal equipment state.

[0132] Specifically, based on the failure mode quantification (FM) assessment results generated by the aforementioned steps, the system provides a failure mode category and risk level for each assessed time point (or short window). The system then selects data within a fixed-length time window for trend analysis. Specifically, using the current time point as a baseline, it backtracks and selects a sequence of conveyor belt image frames acquired within the past 30 consecutive minutes. To reduce computational load and maintain the smoothness of the trend analysis, the system samples one frame per minute, selecting 30 frames at equal intervals from this 30-minute image sequence and recording the precise acquisition timestamps of these selected images. For each of these 30 selected frames, the system re-calls or searches for the failure mode quantification assessment result already calculated at that time point (usually a...). A numerical risk level is assigned, such as an integer from 1 to 5, or a continuous risk score. Meanwhile, for each selected original conveyor belt surface image (i.e., a "near-infrared band fused image"), its average edge intensity is calculated. This average edge intensity can be obtained by applying the Sobel operator or other edge detection operators to the entire image to obtain an edge map, and then calculating the average intensity of all pixels in the edge map, or by calculating the proportion of edge pixels to the total number of pixels. Finally, the timestamps of these 30 images, the corresponding failure mode quantization assessment results, and the calculated average edge intensity are organized to form a time series dataset containing 30 records. Each record contains three data items: (timestamp, failure mode quantization assessment result, and average edge intensity).

[0133] formula: The advantage of this formula lies in its ability to quantify the degree of enhancement of the failure trend by calculating the cumulative sum of the products between changes in the average edge intensity of the image and changes in the failure mode quantitative assessment results, and then dividing by the total time span. When changes in the physical characteristics of the image (such as edge intensity) are in the same direction as changes in the risk assessment results (i.e., both increase or decrease simultaneously), the product is positive, indicating that the physical changes and the trends in the risk assessment are consistent. The accumulation of this consistency can reflect whether the failure trend is continuously increasing or decreasing. This ensures that the magnitude of change in edge intensity is included in the calculation, while The term directly reflects the direction and magnitude of changes in the risk assessment results. Multiplying the two emphasizes the synergistic effect between physical changes and changes in risk assessment. Dividing by the total time... The purpose is to obtain an average rate of change so that the response factors of different time windows are comparable, thereby making it more stable to judge the trend;

[0134] The steps to obtain the parameters are as follows:

[0135] The first in the representative sequence dataset The average edge intensity of the frame image, this value is obtained by applying the first... The image is obtained by image processing of the original conveyor belt surface image (i.e., the "near-infrared band fused image"). First, the Sobel operator is applied to the image to calculate the gradient magnitude of each pixel, forming a gradient map. Then, the arithmetic mean of the gradient magnitudes of all pixels in this gradient map is calculated, which is the average edge intensity of the image frame. For example, for an 8-bit grayscale image, the average edge intensity value may fluctuate within a small range between 0 and 255. For example, the first... After the frame image is processed by the Sobel operator, its average edge intensity is calculated. .

[0136] The steps to obtain the parameters are as follows:

[0137] The first in the representative sequence dataset The failure mode quantification assessment result corresponding to the frame image is obtained in the previous steps based on the "key feature values ​​of equipment failure" and the "predefined failure mode risk mapping rules." It is a numerical value that quantifies the risk level of the equipment state reflected in the frame image. For example, the risk level can range from 1 (no risk or very low risk) to 5 (very high risk), or it can be a continuous risk score between 0 and 1. Extracted directly from the "sequence dataset", for example, for the first... If a frame image has a failure mode quantification assessment result of 3 (representing medium risk) recorded in the sequence dataset, then... .

[0138] The steps to obtain the parameters are as follows:

[0139] The total number of frames used in the calculation is calculated by selecting a sequence of conveyor belt images from the past 30 minutes, with one frame extracted per minute. The value is fixed at 30; this parameter is a preset value, for example. .

[0140] The steps to obtain the parameters are as follows:

[0141] This represents the acquisition timestamp of the first frame in the sequence dataset. This timestamp is recorded from the original image frame information when the "sequence dataset" is constructed. For example, if the acquisition time of the first frame in the sequence is "2023-10-26 10:00:00.000", convert it to a Unix timestamp, for example... Second.

[0142] The steps to obtain the parameters are as follows:

[0143] Represents the first in the sequence dataset The acquisition timestamp of the frame (i.e., the last frame, frame 30 in this example) is obtained in the same way as... Similarly, the timestamp of the frame image is read directly from the "sequence dataset". For example, since one frame is extracted per minute, for a total of 30 frames, the 30th frame is approximately 29 minutes after the 1st frame. If the 1st frame is 10:00:00, then the 30th frame is approximately 10:29:00. Its precise timestamp is also converted to seconds, for example... Second.

[0144] Substituting the actual parameters into the formula, we get 0.5. This result shows that within this 30-frame data segment, the average enhancement rate of the failure trend is 0.5. A positive value indicates that, on average, the change in the average edge intensity of the image is in the same direction as the change in the failure mode quantification assessment result, and shows an overall enhancing trend. If this value remains positive and exceeds a certain preset significance threshold, it may indicate that the device condition is deteriorating.

[0145] The failure trend enhancement response factor calculated in the previous paragraph A sliding time window method is used to continuously monitor whether this trend is persistent. Each time window is calculated based on data from the past 30 consecutive minutes (i.e., 30 frames of images). The value, this time window slides forward in fixed steps, for example, every 5 minutes (or less, such as 1 minute, corresponding to the acquisition of a new frame of data and the update of the sequence dataset), is recalculated. This will result in a The system then compares the calculated failure trend enhancement response factor over several consecutive windows (e.g., three consecutive windows) to determine if a sustained upward trend exists, based on a preset threshold value, for example... This threshold value is set based on historical normal operation and the development process of known faults. The values ​​are obtained through statistical analysis of the distribution. For example, one could take 1.5 times the upper limit of the normal fluctuation range plus one standard deviation, or based on the failure cases. The typical value setting before a significant increase, if the failure trend enhancement response factor is calculated through three consecutive sliding windows. All values ​​are greater than 0 (indicating a strengthening trend), and each value exceeds a preset threshold. (For example, , , If a significant and continuous upward trend in equipment condition deterioration has formed, the system will mark the entire time period covered by these three consecutive windows (from the beginning of the first window to the end of the third window) as a continuous abnormal equipment condition.

[0146] The steps for obtaining fault location results for the conveyor system are as follows:

[0147] Based on the abnormal state period determined by the continuous abnormal state of the equipment, the conveyor belt image frame index corresponding to the abnormal state period is called, the frame acquisition timestamp recorded in the image frame index is extracted, and the real-time position information of the corresponding conveyor belt encoder is queried according to the timestamp to form the conveyor belt encoder position sequence of the abnormal state.

[0148] Based on the abnormal conveyor encoder position sequence, the image frame index corresponding to the position sequence is matched one by one, the camera identification code corresponding to each image frame index is called, the camera identification code is classified and statistically analyzed according to the conveyor encoder position, the camera identification code with the highest frequency is selected as the key camera identification in the abnormal area, and the key camera identification classification result is generated.

[0149] Based on the key camera identifiers identified in the key camera identifier classification results, and combined with the physical installation location information of the corresponding cameras, a mapping relationship between the encoder position of the conveyor belt and the camera installation position is established. The spatial location of the conveyor belt corresponding to the area where the anomaly occurred is then locked, and the fault location result of the conveyor system is obtained.

[0150] Specifically, based on the persistent equipment abnormality status and its corresponding occurrence period determined in the aforementioned steps—for example, determining that the period from "2023-10-26 10:15:00" to "2023-10-26 10:45:00" is a persistent equipment abnormality period—the system first retrieves and calls the index information of all conveyor belt image frames acquired during this abnormality period. This image frame index information already contains a unique number for each frame and a frame acquisition timestamp accurate to milliseconds. Then, for this batch of image frames marked as abnormal, the system extracts their frame acquisition timestamps one by one and uses these timestamps to query a synchronously recorded conveyor belt. An encoder database or real-time data stream, where the encoder is typically mounted on the drive or driven wheel of the conveyor belt, provides the physical position information of a point on the conveyor belt relative to a fixed reference point (such as the start of the conveyor belt or a specific sensor position). This information is usually expressed in meters or encoder pulse counts. By matching the acquisition timestamp of each frame of an abnormal image with the timestamp recorded in the encoder database (finding the encoder reading with the closest time), the encoder position value corresponding to the imaged area on the conveyor belt at the time the image was acquired is obtained. All these conveyor belt encoder position values ​​acquired during the abnormal state period are arranged in chronological order to form an abnormal state conveyor belt encoder position sequence.

[0151] Based on the conveyor belt encoder position sequence of the abnormal state formed in the previous step, each position value in this sequence corresponds to an image frame acquired during the period of continuous equipment abnormality. The system then processes each encoder position value in this position sequence one by one and finds the associated original image frame index. From the image frame index information, the unique identifier code of the camera responsible for acquiring the image is extracted (e.g., camera ID "CAM001", camera ID "CAM002", etc., these IDs have been assigned to various cameras installed at different positions along the conveyor belt during system deployment). Then, the system classifies and counts these camera identifier codes according to the conveyor belt encoder position. Specifically, a certain encoder position range can be set (…). For example, intervals can be defined as 5 meters apart, or intervals can be determined based on the camera's field of view. Within each encoder position interval, it can be counted which cameras captured image frames deemed abnormal, and the frequency of each camera appearing in each interval can be calculated. Alternatively, more directly, within the physical segment covered by the encoder position sequence of the entire abnormal state, the total frequency of all participating camera identification codes can be counted, and the camera identification codes that appear most frequently during the entire abnormal period or in the encoder position segment where the abnormality is most concentrated can be selected. These selected camera identification codes are identified as the key camera identifications most relevant to the abnormal area. These key camera identifications and their frequency or correlation scores are then summarized to generate the key camera identification classification results.

[0152] Based on the key camera identifier classification results generated in the previous paragraph, which identified which cameras most frequently captured conveyor belt sections in a persistent equipment malfunction state, the system then combines these identified key camera identifiers with the physical installation location information of each camera pre-stored in the system configuration. This physical installation location information is typically described as absolute coordinates within the conveyor belt system or distances relative to a reference point (e.g., "CAM001" is installed 15 meters behind transfer station 0 of the conveyor belt, covering a section of the conveyor belt from 15 meters to 18 meters in length). The system associates key camera identifiers (such as "CAM001") with their corresponding physical installation locations (15-meter to 18-meter section), and then combines this with the abnormal conveyor belt encoder location sequence obtained in the first step (e.g., this sequence indicates an abnormality). The main focus is on encoder readings between 14,500 and 14,800 (this encoder range corresponds to a distance of 14.5 to 14.8 meters on the conveyor belt, according to calibration). This establishes a precise mapping between the conveyor belt encoder position, camera identification, and the actual physical location of the conveyor belt monitored by the camera. By integrating this information, the system can more accurately pinpoint the specific area of ​​the conveyor belt where the anomaly occurred. For example, if the monitoring range of the key camera "CAM001" highly overlaps with the physical location indicated by the abnormal encoder position sequence, it can be determined that the fault occurred within the conveyor belt section monitored by that camera, ultimately yielding a specific fault location result for the conveyor system, such as "the fault is located on the surface of the conveyor belt approximately 14.5 to 14.8 meters behind transfer station 0, and is under the key monitoring of camera CAM001."

[0153] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for monitoring the operation of a power plant coal conveying system based on an Internet of Things (IoT) sensor system, characterized in that, Includes the following steps: Based on the surface images of the conveyor belt acquired by the multispectral camera and the images of the idler rollers acquired by the camera, the surface images of the conveyor belt are fused in the near-infrared band, and the end face contours of the idler roller images are extracted to establish equipment operating status characterization data. Based on the equipment operation status characterization data, differential operation is performed on the conveyor belt characterization data of continuous time frames to obtain a short-time change map of the conveyor belt surface. Based on the short-time change map of the conveyor belt surface, a dynamic threshold is set according to the statistical distribution of pixel values ​​in the map for segmentation processing to identify potential longitudinal crack growth points. Based on the location of the potential longitudinal crack growth point and the equipment operating status characterization data, the cumulative edge gradient value of the growth point along the conveyor belt running direction is calculated, and the angular displacement of the idler roller profile in continuous time frames is calculated to obtain the key feature value of equipment failure. Based on the key feature value of equipment failure, it is compared with the dynamic threshold set according to the current running speed of the conveyor belt to generate a quantitative assessment result of failure mode. Based on the failure mode quantitative evaluation results of continuous time frames, it is determined whether there is a monotonically increasing trend, a continuous equipment abnormal state is established, and based on the continuous equipment abnormal state, the conveyor belt encoder position and idler camera identifier corresponding to the occurrence of the lock state, the fault location result of the conveyor system is obtained. The steps for obtaining the short-time variation pattern of the conveyor belt surface are as follows: Based on the equipment operation status characterization data, the conveyor belt image sequence within the range of unified idler camera identification and synchronization timestamp is extracted. The image sequence is arranged in ascending order according to the image frame number. The pixel grayscale matrix information of each frame image is read and the shooting time difference of adjacent frame images and the conveyor belt running speed at the corresponding time point are recorded to form a continuous time frame data group. Based on the continuous time frame data group, calculate the grayscale change index per unit movement distance between adjacent frames; Based on the grayscale change index per unit movement distance, and combined with the pixel grayscale difference image matrix of the corresponding frame, the grayscale difference image of each frame and the corresponding grayscale change index per unit movement distance are mapped in two layers on the time axis. The grayscale image matrix is ​​expressed in a thermal mapping manner, and a time-series mapping atlas is generated using the grayscale change index sequence per unit movement distance as the index of change amplitude, forming a short-time change map of the conveyor belt surface.

2. The method for monitoring the operation of a power plant coal conveying system based on an Internet of Things (IoT) sensor system according to claim 1, characterized in that, The steps for obtaining the device operating status characterization data are as follows: Based on the conveyor belt surface image acquired by the multispectral camera and the idler roller image acquired by the camera, the frame number, image number and timestamp of the two types of images are recorded respectively. For the time difference between the recorded image number and the adjacent frames in the timestamp list, the image frames with time errors less than the preset tolerance range are selected to obtain timestamp aligned image frame pairs. Based on the timestamp-aligned image frame pairs, the image layer channel information corresponding to the near-infrared band in the conveyor belt surface image is extracted. The channel information is then processed by grayscale normalization according to the pixel position number and then superimposed. During the fusion process, the superimposed grayscale value is bound to the image coordinates to generate a fused image, thus obtaining a near-infrared band fused image. Based on the near-infrared band fused image, and combined with the corresponding timestamp-aligned image frame to center the idler image, edge extraction is performed on the idler image to extract the edge of the idler end face. The curvature value, closure and continuous change characteristics of the main axis direction of the end face edge line are used to construct a structural feature vector associated with the idler state. The structural feature vector is combined with the pixel average brightness value and slope gradient value in the near-infrared band fused image according to the frame index number to form a multi-dimensional vector set, generating equipment operation status characterization data.

3. The method for monitoring the operation of a power plant coal conveying system based on an Internet of Things (IoT) sensor system according to claim 1, characterized in that, The steps for obtaining the potential longitudinal crack growth points are as follows: Based on the short-time change map of the conveyor belt surface, the gray values ​​of all pixels in the map are extracted frame by frame and a gray value histogram is constructed. The frequency of gray values ​​in each interval is counted, the probability distribution of the frequency in each interval is calculated, and the gray value probability distribution is sorted to obtain the cumulative probability curve of gray values. Based on the gray value cumulative probability curve, the gray value corresponding to the first time the cumulative probability of gray value exceeds the 95th percentile is selected as the dynamic threshold. The dynamic threshold is applied to the short-time change map of the conveyor belt surface to perform pixel gray-level binarization frame by frame. The pixel areas above the dynamic threshold are marked as potential abnormal areas to form a potential abnormal area mask map. Based on the mask map of potential abnormal regions, the geometric features of each region, including aspect ratio and area, are extracted. Pixel regions with aspect ratio greater than a set value, area exceeding the minimum area limit, and continuous boundaries are selected and identified as potential longitudinal crack growth points.

4. The method for monitoring the operation of a power plant coal conveying system based on an Internet of Things (IoT) sensor system according to claim 1, characterized in that, The steps for obtaining the key feature values ​​of the equipment failure are as follows: Based on the location data of the potential longitudinal crack growth points, the edge pixel values ​​in the corresponding conveyor belt surface image are extracted along the conveyor belt running direction. The difference between the gray value of each pixel and the gray value of the adjacent pixel is calculated and the absolute value is taken. The difference is accumulated point by point to form the edge gradient accumulation value of the longitudinal crack growth point location. Based on the cumulative edge gradient value of the longitudinal crack growth point location, combined with the corresponding continuous time frame of the idler end face contour image data, the boundary point coordinates of the idler end face contour are extracted frame by frame. By fitting the ellipse center of the boundary point of the idler end face contour, the rotation angle change value of each idler end face contour center point between adjacent frames is calculated. The rotation angle change value of multiple consecutive frames is accumulated to form the angular displacement of the idler end face contour. Based on the angular displacement of the roller end face profile and the cumulative edge gradient value of the longitudinal crack growth point location, the numerical sequence of the two parameters is combined to obtain the key characteristic value of equipment failure.

5. The method for monitoring the operation of a power plant coal conveying system based on an Internet of Things (IoT) sensor system according to claim 1, characterized in that, The steps for obtaining the failure mode quantitative assessment results are as follows: Based on the key feature values ​​of equipment failure, the values ​​of each feature index in the key feature value sequence of equipment failure are extracted, and the statistics of each index value are calculated, including the maximum value, average value and variance of the feature value sequence. The feature index name corresponding to the statistics is recorded to form the statistical results of equipment failure features. Based on the statistical results of the equipment failure characteristics, the current operating speed data of the conveyor belt is called. According to the current operating speed value of the conveyor belt, the safe value range of the historical characteristic statistical results that match the speed range is found. The limit threshold that is suitable for the current operating speed is determined and compared with the statistical quantities of each indicator of the equipment failure characteristics statistical results one by one. The statistical indicators that exceed the limit threshold are marked to form the threshold comparison judgment result. Based on the threshold comparison results, the number of statistical indicators and the names of characteristic indicators that exceed the limit threshold are recorded. According to the number of statistical indicators and the names of characteristic indicators that exceed the dynamic threshold, the corresponding failure mode category and quantification level are determined and matched in combination with the predefined failure mode risk mapping rules to form a failure mode quantification assessment result.

6. The method for monitoring the operation of a power plant coal conveying system based on an Internet of Things (IoT) sensor system according to claim 1, characterized in that, The steps for obtaining the persistent abnormal state of the equipment are as follows: Based on the failure mode quantification assessment results, a sequence of conveyor belt image frames from the past 30 consecutive minutes was selected. One image was extracted per minute to obtain a total of 30 images and timestamp information. At the same time, the failure mode quantification assessment results and the average edge intensity of each image were extracted to form a sequence dataset. Based on the sequence dataset, calculate the failure trend enhancement response factor; Based on the failure trend enhancement response factor, the time window consisting of 30 frames is continuously slid, and the failure trend enhancement response factor in adjacent windows is compared. If the failure trend enhancement response factor is continuously positive and exceeds the preset threshold in three consecutive windows, it is considered to form an upward trend, and the corresponding time period is marked as a continuous equipment abnormal state.

7. The method for monitoring the operation of a power plant coal conveying system based on an Internet of Things (IoT) sensor system according to claim 1, characterized in that, The steps for obtaining the fault location results of the conveying system are as follows: Based on the abnormal state period determined by the continuous abnormal state of the equipment, the conveyor belt image frame index corresponding to the abnormal state period is called, the frame acquisition timestamp recorded in the image frame index is extracted, and the real-time position information of the corresponding conveyor belt encoder is queried according to the timestamp to form the conveyor belt encoder position sequence of the abnormal state. Based on the conveyor encoder position sequence in the abnormal state, the image frame index corresponding to the position sequence is matched one by one, the camera identification code corresponding to each image frame index is called, the camera identification code is classified and statistically analyzed according to the conveyor encoder position, the camera identification code with the highest frequency of occurrence is selected as the key camera identification in the abnormal area, and the key camera identification classification result is generated. Based on the key camera identifiers identified in the key camera identifier classification results, and combined with the physical installation location information of the corresponding cameras, a mapping relationship between the encoder position and the camera installation position is established, the spatial location of the conveyor belt corresponding to the abnormality area is locked, and the fault location result of the conveyor system is obtained.

Images