Power plant coal conveying system operation monitoring method based on Internet of Things sensing system

Abstract

The invention relates to the technical field of visual inspection, in particular to a power plant coal conveying system operation monitoring method based on an internet-of-things sensing system, which comprises the following steps of: performing near-infrared band fusion on a surface image of a conveying belt based on the surface image of the conveying belt acquired by a multispectral camera and a carrier roller image acquired by the camera; the end face contour of the carrier roller image is extracted, and equipment operation state characterization data is established; according to the method, the images of the conveying belt and the carrier roller are collected through the multispectral camera, the near-infrared band images are fused, the end face contour data of the carrier roller are finely extracted, and accurate characterization of the running state of the conveying system is achieved; on the basis of state characterization data, a short-time change information map of the conveying belt is obtained through differential operation of continuous time frames, a dynamic threshold value is set according to a statistical rule of pixel values in the map, and potential longitudinal cracks are segmented and positioned.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual detection, and particularly relates to a power plant coal conveying system operation monitoring method based on an Internet of Things sensing system. BACKGROUND

[0002] The technical field of visual detection is a technical field of detecting and judging the state, quality or abnormal condition of a target object by collecting an image of a target to be detected or a region through a visual perception device (such as a multispectral camera, a CCD camera or the like), and then utilizing image processing technology, including edge extraction, image enhancement, feature extraction, pattern recognition and the like algorithm means.

[0003] The prior art is difficult to identify early fine cracks and slight changes in the details of the running state, and lacks a device space position mapping means, resulting in fuzzy fault positioning and increasing the subsequent fault troubleshooting cost. For example, in the actual operation process of the coal conveying system, the initial stage of fine cracks is difficult to be detected due to its weak image feature change, resulting in continuous development of abnormal cracks until causing obvious equipment failure, increasing the maintenance cost and safety hazards. Therefore, improvement is needed. SUMMARY

[0004] The purpose of the present application is to solve the shortcomings in the prior art and propose a power plant coal conveying system operation monitoring method based on an Internet of Things sensing system.

[0005] In order to achieve the above-mentioned purpose, the present application adopts the following technical scheme, a power plant coal conveying system operation monitoring method based on an Internet of Things sensing system, comprising the following steps: Based on the conveyor belt surface image collected by the multispectral camera and the camera collected roller image, the surface image of the conveyor belt is fused in the near-infrared wave band, and the end face profile of the roller image is extracted, and the equipment running state representation data is established; Based on the equipment running state representation data, the conveyor belt representation data of the continuous time frame is subjected to difference operation, the conveyor belt surface short-time change graph is obtained, based on the conveyor belt surface short-time change graph, the dynamic threshold is set according to the statistical distribution of the pixel value in the graph for segmentation processing, and the potential longitudinal crack growth point is identified; Based on the position of the potential longitudinal crack growth point and the equipment running state representation data, the edge gradient cumulative value of the growth point along the running direction of the conveyor belt is calculated, and the angular displacement of the roller profile in the continuous time frame is calculated, the equipment failure key feature value is obtained, based on the equipment failure key feature value, the dynamic threshold is compared and judged according to the current running speed of the conveyor belt, and the failure mode quantitative evaluation result is generated; Based on the failure mode quantitative evaluation result of the continuous time frame, it is judged whether there is a monotonic increasing trend, a persistent device abnormal state is established, based on the persistent device abnormal state, the corresponding conveyor belt encoder position and the idler camera identification when the state occurs are locked, and the conveyor system fault positioning result is obtained.

[0006] Preferably, the device operation state representation data acquisition step is: Based on the conveyor belt surface image collected by the multispectral camera and the idler image collected by the camera, the frame number, image number and timestamp of the two types of images are recorded respectively, the time difference of adjacent frames in the recorded image number and timestamp list is screened, the image frames with a time error less than a preset tolerance range are obtained, and a timestamp aligned image frame pair is obtained. Based on the timestamp aligned image frame pair, the image level channel information corresponding to the near-infrared waveband in the conveyor belt surface image is extracted, the channel information is subjected to gray scale normalization processing according to the pixel bit serial number, and then superimposed, the superimposed gray scale value and the image coordinates are bound during the fusion process to generate a fusion image, and a near-infrared waveband fusion image is obtained. Based on the near-infrared waveband fusion image, the edge of the idler image in the corresponding timestamp aligned image frame pair is extracted, the edge of the idler end face is extracted, and the curvature value, closedness and main shaft direction continuous change characteristics of the end face edge line are used to construct a structure feature vector associated with the idler state. The structure feature vector and the pixel average brightness value and slope gradient value in the near-infrared waveband fusion image are combined and arranged as a multi-dimensional vector set according to the frame index number to generate device operation state representation data.

[0007] Preferably, the conveyor belt surface short-term change map acquisition step is: Based on the device operation state representation data, the conveyor belt image sequence within the unified idler camera identification and synchronous timestamp range is extracted, the image sequence is arranged in ascending order according to the image frame number, the pixel gray scale matrix information of each frame image is read, and the shooting time difference of adjacent frame images and the corresponding time point conveyor belt running speed are recorded to form a continuous time frame data group. According to the continuous time frame data group, the unit moving distance gray scale change index between adjacent frames is calculated. According to the unit moving distance gray scale change index, the gray scale difference image of each frame is combined with the corresponding unit moving distance gray scale change index to perform double-layer mapping on the time axis, the gray scale image matrix is expressed in the form of heat mapping, and the unit moving distance gray scale change index sequence is taken as the change amplitude index to generate a time sequence mapping set, forming a conveyor belt surface short-term change map.

[0008] Preferably, the potential longitudinal crack growth point acquisition step is: Based on the short-time change map of the conveyor belt surface, the gray value of all pixels in the map is extracted frame by frame and a gray value histogram is constructed, the occurrence frequency of the gray value in each interval is counted, the probability distribution of the frequency of each interval is calculated, and the gray value cumulative probability curve is obtained by sorting the gray value probability distribution; Based on the gray value cumulative probability curve, the gray value corresponding to the first time when the cumulative probability exceeds 95% is selected as the dynamic threshold, the dynamic threshold is applied to the short-time change map of the conveyor belt surface for pixel gray value binarization frame by frame, and the pixel region higher than the dynamic threshold is marked as a potential abnormal region to form a potential abnormal region mask map; Based on the potential abnormal region mask map, the geometric features of each region, including the aspect ratio and area, are extracted, and the pixel region with an aspect ratio greater than a set value, an area exceeding a minimum area limit, and a continuous boundary is selected as a potential longitudinal crack growth point.

[0009] Preferably, the device failure key feature value acquisition step is: Based on the position data of the potential longitudinal crack growth point, the edge pixel value in the corresponding conveyor belt surface image is extracted along the running direction of the conveyor belt, the difference between the gray value of each pixel point and the gray value of the adjacent pixel point is calculated and the absolute value is taken, the difference is accumulated point by point to form the edge gradient cumulative value of the longitudinal crack growth point position; Based on the edge gradient cumulative value of the longitudinal crack growth point position, the boundary point coordinates of the roller end face profile are extracted frame by frame in combination with the roller end face profile image data of the corresponding continuous time frames, the center of the ellipse fitting the boundary points of the roller end face profile is calculated, the rotation angle change value of each roller end face profile center point between adjacent frames is calculated, and the rotation angle change value of continuous multiple frames is accumulated to form the angular displacement of the roller end face profile. Based on the angular displacement of the roller end face profile and the edge gradient cumulative value of the longitudinal crack growth point position, the numerical sequences of the two parameters are combined to obtain the device failure key feature value.

[0010] Preferably, the failure mode quantitative evaluation result acquisition step is: Based on the device failure key feature value, the numerical value of each feature index in the device failure key feature value sequence is extracted, the statistics of each index value is calculated, including the maximum value, the average value and the variance value of the feature value sequence, and the feature index name corresponding to the statistics is recorded to form the device failure feature statistical result. Based on the device failure feature statistical result, the current running speed data of the conveying belt is called, the safety value range of the historical feature statistical result matched with the speed interval is searched according to the current running speed value of the conveying belt, the limited threshold value adapted to the current running speed is determined, and each index statistical quantity of the device failure feature statistical result is compared one by one, the statistical index exceeding the limited threshold value is marked, and the threshold value comparison judgment result is formed; Based on the threshold value comparison judgment result, the number of statistical indexes exceeding the limited threshold value and the feature index name are recorded, the corresponding failure mode category and quantitative grade are judged and matched according to the number of statistical indexes exceeding the dynamic threshold value and the feature index name, in combination with the pre-defined failure mode risk mapping rule, and the failure mode quantitative evaluation result is formed.

[0011] Preferably, the acquisition step of the persistent device abnormal state is: Based on the failure mode quantitative evaluation result, the image frame sequence of the conveying belt in the past 30 consecutive minutes is selected, 1 frame of image is extracted per minute, a total of 30 frames of images and timestamp information are acquired, and the failure mode quantitative evaluation result and the image average edge intensity of each frame of image are extracted, so as to form a sequence data set; According to the sequence data set, a failure trend enhancement response factor is calculated; Based on the failure trend enhancement response factor, a time window composed of every 30 frames is continuously slid, the failure trend enhancement response factors in adjacent windows are compared, if the failure trend enhancement response factors in three consecutive windows are continuously positive and exceed a preset threshold, it is considered that an upward trend is formed, and the corresponding time period is marked as a persistent device abnormal state.

[0012] Preferably, the acquisition step of the conveying system fault positioning result is: Based on the abnormal state occurrence period determined by the persistent device abnormal state, the image frame index corresponding to the abnormal state period is called, the frame acquisition timestamp recorded in the image frame index is extracted, and the real-time position information of the conveying belt encoder corresponding to the timestamp is queried according to the timestamp, so as to form a conveying belt encoder position sequence of the abnormal state; According to the conveying belt encoder position sequence of the abnormal state, the image frame index corresponding to the position sequence is matched one by one, the camera identification code corresponding to each image frame index is called, the camera identification codes are classified and counted according to the conveying belt encoder position, the camera identification code with the highest occurrence frequency is screened as the key camera identification of the abnormal area, and a key camera identification classification result is generated; According to the key camera identification determined in the key camera identification classification result, in combination with the physical installation position information of the corresponding camera, a mapping relationship between the conveying belt encoder position and the camera installation position is established, the abnormal occurrence area corresponding to the conveying belt space position is locked, and a conveying system fault positioning result is obtained.

[0013] Compared with the prior art, the application has the advantages and positive effects that: The application realizes the accurate characterization of the running state of the conveying system by collecting the images of the conveying belt and the carrier roller through the multispectral camera, fusing the near-infrared band image and finely extracting the end face profile data of the carrier roller. On the basis of the state characterization data, the short-time change information atlas of the conveying belt is obtained through the difference operation of the continuous time frames, and the dynamic threshold is set according to the statistical law of the pixel value in the atlas to segment and locate the potential longitudinal cracks. Further, based on the position of the crack growth point and the state data, the gradient cumulative value of the crack edge along the conveying belt direction is calculated and the angular displacement change of the carrier roller profile is recognized to form the quantitative results of the key features of the equipment failure. The dynamic threshold is set in combination with the running speed of the conveying belt to judge the abnormal state of the equipment with a continuously rising trend, and finally the spatial positioning of the conveying system failure is realized by locking the position of the conveying belt encoder and the carrier roller camera identifier when the abnormal state occurs. In summary, the application can improve the abnormal recognition sensitivity of the conveying equipment, reduce the hysteresis of the equipment abnormal detection, and enhance the accuracy of the operation monitoring of the conveying system. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 The figure is a schematic diagram of the steps of the application. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical scheme and advantages of the application more clear, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application.

[0016] Please refer to Figure 1 The application provides a technical scheme, a power plant coal conveying system operation monitoring method based on an Internet of Things sensing system, which comprises the following steps: Based on the conveying belt surface image collected by the multispectral camera and the carrier roller image collected by the camera, the near-infrared band fusion is performed on the surface image of the conveying belt, and the end face profile is extracted from the carrier roller image to establish the equipment running state characterization data; Based on the equipment running state characterization data, the difference operation is performed on the conveying belt characterization data of the continuous time frames to obtain the short-time change atlas of the conveying belt surface, and based on the short-time change atlas of the conveying belt surface, the dynamic threshold is set according to the statistical distribution of the pixel value in the atlas for segmentation processing to identify the potential longitudinal crack growth point; Based on the position of the potential longitudinal crack growth point and the equipment operating state representation data, the edge gradient cumulative value of the growth point along the running direction of the conveyor belt is calculated, and the angular displacement of the roller profile in the continuous time frame is calculated to obtain the equipment failure key characteristic value. Based on the equipment failure key characteristic value, the dynamic threshold value set according to the current running speed of the conveyor belt is compared and judged to generate a failure mode quantitative evaluation result. Based on the failure mode quantitative evaluation result of the continuous time frame, it is judged whether there is a monotonic increasing trend, and a persistent equipment abnormal state is established. Based on the persistent equipment abnormal state, the conveyor belt encoder position and the roller camera identifier corresponding to the state occurrence are locked to obtain a conveyor system fault positioning result.

[0017] The acquisition step of the equipment operating state representation data is: Based on the conveyor belt surface images collected by the multispectral camera and the roller images collected by the camera, the frame number, image number and timestamp of each image are recorded. The time difference between adjacent frames in the recorded image number and timestamp list is calculated, and the image frames with a time error less than a preset tolerance range are selected to obtain a timestamp aligned image frame pair. Based on the timestamp aligned image frame pair, the image level channel information corresponding to the near-infrared waveband in the conveyor belt surface image is extracted. After gray scale normalization processing of the channel information according to the pixel bit sequence number, the superposition is carried out. The superimposed gray value and the image coordinates are bound during the fusion process to generate a fusion image, and a near-infrared waveband fusion image is obtained. Based on the near-infrared waveband fusion image, the edge of the roller image is extracted by combining the corresponding timestamp aligned image frame pair, the roller image is edge extracted, the roller end face edge is extracted, and the curvature value, closedness and main shaft direction continuous change characteristics of the end face edge line are used to construct a structure feature vector associated with the roller state. The structure feature vector, the average brightness value and the slope gradient value in the near-infrared waveband fusion image are combined and arranged as a multi-dimensional vector set according to the frame index number to generate equipment operating state representation data.

[0018] Specifically, based on the conveyor belt surface images collected by the multispectral camera and the roller images collected by the camera, the system automatically records the frame number, image number and acquisition timestamp for each frame of conveyor belt surface image. Similarly, the frame number, image number and acquisition timestamp are also recorded for each frame of roller image. Then, in order to match the conveyor belt images and roller images collected at similar time, the system traverses the timestamp list of one type of image (for example, conveyor belt surface image). For each timestamp in the list , it is found whether there is a timestamp in the timestamp list of the other type of image (roller image) such that the absolute value of the time difference between the two is less than a preset time tolerance . The time tolerance The setting needs to consider the sampling frequency difference of two cameras and the data transmission delay inside the system. For example, if the frame rate of the conveyor surface image camera is 50 frames per second (i.e. 20 milliseconds per frame interval), and the frame rate of the roller image camera is 25 frames per second (i.e. 40 milliseconds per frame interval), considering the maximum possible sampling asynchronization and the delay of about 5 milliseconds introduced by data buffering and network transmission, the time difference threshold can be set to 15 milliseconds, and the calculation method is as follows: The calculation method is as follows: Here, 15 ms is taken for simplicity, and the screening criteria are adjusted as follows: if the timestamp of the conveyor surface image is , then find the image in the roller image whose timestamp is in the interval If multiple images are found, select the one with the smallest time difference, and if none is found, the conveyor surface image frame does not participate in subsequent pairing. For all conveyor image frames that meet the time difference condition, combine them with the corresponding roller image frames to form a timestamp-aligned image frame pair containing the conveyor image, the roller image, and their respective original timestamps and frame numbers.

[0019] Based on the timestamp-aligned image frame pair, the image data of a specific near-infrared waveband is extracted from the conveyor surface multispectral image in each pair, for example, the image level channel information corresponding to the near-infrared waveband with a center wavelength of 850 nanometers and a bandwidth of 40 nanometers is selected. This waveband is selected because it is more sensitive to defects such as aging and early cracking of the conveyor belt material. Then, for each pixel of the extracted single-waveband near-infrared image, the gray scale normalization processing is performed according to its position sequence number (i.e. row and column coordinates) in the image. This normalization processing linearly maps the pixel gray scale value of the current frame in the near-infrared channel from its original range (e.g. 0 to 4095 for a 12-bit image) to the standard 8-bit gray scale range of 0 to 255. The calculation process is as follows: first, traverse all pixels in the near-infrared channel of the current frame to find the minimum gray scale value and the maximum gray scale value Then, for any pixel , the normalized gray scale value of the original gray scale value is calculated by the formula If , then the normalized gray scale value of all pixels is set to 0 or 127. The normalized gray scale value of each pixel after processing is directly bound to its corresponding image coordinates together form a new single-channel gray scale image. This normalized single-waveband near-infrared image is the near-infrared waveband fusion image.

[0020] Based on the near-infrared band fusion image, and combined with the corresponding idler image in each timestamp aligned image frame pair, first, edge detection operation is performed on the idler image to extract the end face profile of the idler, specifically, the Canny edge detection algorithm can be used, and the low threshold value and the high threshold value are set according to experience, for example, if the overall contrast of the idler image is high, the low threshold value and the high threshold value may be set as 0.5 times the optimal threshold value, for example, the low threshold value is the optimal threshold value, is 0.5 times the optimal threshold value, after edge detection, the approximate circular or elliptical closed profile representing the idler end face is identified and extracted from the edge pixels through Hough circle transformation or ellipse fitting algorithm, for the extracted idler end face edge profile, the geometric feature parameters are calculated, including the average and maximum values of the curvature of each point on the profile, the closedness of the profile (for example, by calculating the distance between the start point and the end point of the profile, if it is less than a preset number of pixels, such as 5 pixels, it is considered to be closed, and the closedness parameter is assigned a value of 1, otherwise 0, or a certain ratio of the perimeter to the covered area is calculated), and the principal axis direction angle determined by principal component analysis, these calculated curvature values, closedness measures and principal axis direction angles are combined into a numerical vector to form a structural feature vector associated with the current idler state, at the same time, the global pixel average brightness value is calculated from the corresponding near-infrared band fusion image, and the average gradient amplitude value of the image is calculated as the slope gradient value using the Sobel operator, finally, the aforementioned idler structural feature vector containing curvature, closedness and principal axis direction is combined with the pixel average brightness value and slope gradient value extracted from the near-infrared band fusion image according to the unified index number of the current image frame to form a multi-dimensional feature vector, and the multi-dimensional feature vector set of all frames is the device running state representation data.

[0021] The acquisition steps of the short-time change map of the conveyor belt surface are: Based on the device running state representation data, the conveyor belt image sequence within the unified idler camera identifier and the synchronous timestamp range is extracted, the image sequence is arranged in ascending order of image frame number, the pixel gray scale matrix information of each image frame is read and the shooting time difference of adjacent image frames and the corresponding time point conveyor belt running speed are recorded to form a continuous time frame data set; According to the continuous time frame data set, the unit moving distance gray scale change index between adjacent frames is calculated, and the calculation formula is: ; Wherein, is the unit moving distance gray scale change index between the i-th frame and the (i+1)-th frame, is the unit moving distance gray scale change index between the (i+1)-th frame and the (i+2)-th frame.The grayscale change index per unit movement distance corresponding to frame grouping (unit: grayscale value / meter). The number of pixels in the vertical direction of each frame of the image. The number of pixels in the horizontal direction of each frame of the image. For the first The first frame of the image Line number The pixel grayscale value of the column. For the first Frame and the The time interval between frames (in seconds). For the first The conveyor belt speed (in meters per second) corresponding to the frame image. It is a very small constant, used to avoid when When the denominator is zero (unit: meters); Based on the grayscale change index per unit movement distance, and combined with the pixel grayscale difference image matrix of the corresponding frame, the grayscale difference image of each frame and the corresponding grayscale change index per unit movement distance are mapped in two layers on the time axis. The grayscale image matrix is ​​expressed in the form of thermal mapping, and a time-series mapping atlas is generated with the grayscale change index sequence per unit movement distance as the index of change amplitude, forming a short-time change map of the conveyor belt surface.

[0022] Specifically, based on the equipment running state representation data, the representation data has already contained the frame number, image number and accurate to millisecond shooting time stamp of each frame of conveyor belt surface image (i.e. near-infrared waveband fusion image) and roller image, and the roller structure feature vector and the pixel average brightness value, slope gradient value of the conveyor belt surface image obtained through image analysis, first, all the data collected by a specific camera are screened out according to the "unified roller camera identification", then, in these data, the conveyor belt surface image is further screened according to the "synchronous time stamp range", the synchronous time stamp range is dynamically set, for example, the system can set a 5-second sliding time window, or define an observation section according to the running distance of the conveyor belt (for example, every 10 meters) feedback by the conveyor belt encoder, to ensure that the selected image sequence corresponds to the complete or partial transit process of a continuous physical section on the conveyor belt in the camera field of view, the screened conveyor belt surface image (i.e. near-infrared waveband fusion image) sequence is arranged in ascending order according to its original "image frame number" to ensure the time continuity of the image, for each frame of image in this ordered image sequence, the system directly reads its complete pixel gray matrix information from the equipment running state representation data, and records the shooting time difference between the frame image and its subsequent image, this time difference is obtained by querying and calculating the difference value of the shooting time stamps recorded by the two frames of images, at the same time, the system obtains the instantaneous running speed of the conveyor belt at the shooting time point of each frame of image from the external conveyor belt speed monitoring system (for example, the speed encoder connected with the conveyor belt driving motor or driven wheel), and integrates the pixel gray matrix of these images, the time difference between adjacent frames and the corresponding conveyor belt running speed to form a continuous time frame data group.

[0023] Formula: The advantage of the formula is that it normalizes the average pixel gray scale change between image frames to the unit conveying distance, providing a quantitative index of the surface change intensity independent of the conveyor belt running speed, which can more accurately reflect the real change rate of the conveyor belt surface state, avoiding misjudgment caused by speed fluctuations, for example, even if the surface wear rate is constant, the frame difference will be larger in a short time when the speed is fast, and vice versa when the speed is slow, this formula eliminates this influence by dividing by (the inter-frame conveyor belt moving distance), so that the value can directly compare the surface change under different speeds, the introduction of the parameter ensures that in the rare case of zero moving distance caused by the calculation of the conveyor belt at rest ( ) or extremely low speed, the denominator will not be zero, ensuring the robustness of the calculation; The parameter acquisition steps are: represents the number of pixels in the vertical direction of each frame of image, which is determined by the hardware specifications and configuration of the multispectral camera used to capture the image. During the system initialization or camera parameter configuration phase, the image sensor resolution of the camera is set. For example, if a face array camera is selected with a sensor vertical pixel of 1024, the value of is 1024, which remains fixed during the image acquisition process unless the camera configuration is changed. The resolution parameter can be obtained by reading the metadata of the camera output image or the preset camera configuration file. For example, for an image with a resolution of 1280x1024, pixels.

[0024] The acquisition step of the parameter is: represents the number of pixels in the horizontal direction of each frame of image, which is similar to , which is determined by the hardware specifications and current configuration of the multispectral camera used to capture the image. For example, if the camera sensor has a horizontal pixel of 1280, the value of is 1280, which also remains fixed during the image acquisition process. The resolution parameter can be obtained by checking the camera specification book, reading the metadata of the camera output image, or the camera resolution parameter set in the system configuration file. For example, for an image with a resolution of 1280x1024, pixels.

[0025] The acquisition step of the parameter is: represents the gray value of the pixel located at the th row and the th column in the th frame of image. These gray values come from the "near-infrared band fusion image" in the "continuous time frame data set" generated in the previous step, and the value range is usually 0 to 255 after normalization (for 8-bit grayscale image). The gray value of each pixel is directly read from the image data according to the coordinates , for example, in the th frame of image, the pixel with coordinates (100, 200) has a gray value of 150, then .

[0026] The acquisition step of the parameter is: represents the time interval between the th frame of image and the th frame of image, with the unit of seconds. This value is obtained from the "continuous time frame data set" by subtracting the shooting timestamp of the th frame from the The capture timestamp of each frame is calculated. The capture timestamp is recorded by the multispectral camera when acquiring the image, accurate to milliseconds or higher. For example, if the frame's timestamp is... The frame's capture timestamp is "10:00:01.200", the first... If the frame's capture timestamp is "10:00:01.240", then... .

[0027] The steps to obtain the parameters are as follows: Indicates that during the filming of the first In the frame image, the instantaneous operating speed of the conveyor belt, in meters per second, is measured. This data originates from the real-time readings of a speed sensor integrated with the conveyor system (such as a rotary encoder mounted on the drive roller or tensioner). The speed sensor converts the detected pulse count into a speed value and provides it to the monitoring system via a data interface (such as a PLC or industrial bus). The frame image's capture timestamp is used to query the corresponding conveyor belt speed record at that moment. For example, by querying the speed record closest to the timestamp "10:00:01.200", the following can be obtained: meters per second.

[0028] The steps to obtain the parameters are as follows: It is a very small positive constant, for example, rice.

[0029] Calculation process: Given two consecutive images with the following parameters: Pixel; Pixel; Consider only one Let's illustrate with a small region. For example, the grayscale values ​​of these two images in this small region are as follows: No. Pixel grayscale values ​​of a frame image (Value range 0-255): ; No. Pixel grayscale values ​​of a frame image (Value range 0-255): ; Time interval between adjacent frames Seconds (corresponding to a sampling rate of 25 frames per second); No. Conveyor belt running speed corresponding to the frame image m / s; very small constant m; calculation process: calculating the sum of absolute values of differences between corresponding pixel gray values of two frames of images : for the example region: ; ; ; ; sum of absolute differences (for the region) = .

[0030] for example, for the entire image, the calculated sum of absolute differences of pixel gray values is .

[0031] calculate the average absolute difference of gray value per pixel: average difference = ; average difference = (gray unit); calculate the denominator : denominator = ; denominator = ; calculate the gray value change index per unit moving distance : ; the result shows that in the first group of frames (i.e. between the first frame and the first frame), the average pixel gray value of the image changes about 12.5 units per meter of movement of the surface of the conveying belt, and this value is an important basis for generating the short-term change map of the surface of the conveying belt subsequently according to the sequence of gray value change index per unit moving distance calculated in the previous step, and the pixel gray difference image matrix corresponding to each frame (or frame pair) generated synchronously in the process of calculating the index, the system begins to construct the short-term change map of the surface of the conveying belt, specifically, for each time point (representing the first frame and the first The pixel gray difference value image matrix corresponding to the time point is visualized, that is, converted into a heat map. In generating the heat map, the size of the pixel gray difference value is mapped to different colors. For example, the area with small difference value is displayed in cold tone (such as blue), and the area with large difference value is displayed in warm tone (such as red). The transition range of color is dynamically adjusted according to the minimum value and maximum value of the difference value in the current batch of images (for example, the images in the past 1 minute), or a fixed mapping range is adopted, for example, 0 to 50 gray scale difference, to ensure the comparability of different time maps. At the same time, the value of the unit movement distance gray change index associated with the frame pair is taken as a key attribute or label of the heat map on the time axis. This series of heat maps arranged in time sequence each reflect the "spatial distribution" of the change of the conveyor belt surface at a specific time, and the value sequence associated therewith constitutes the "amplitude time sequence" of the change. By combining the heat map sequence with the value sequence, for example, superimposing the value sequence in the form of a line graph below or beside the heat map sequence, or adjusting the overall brightness or contrast of the corresponding heat map using the size of the value (for example, the greater the value, the higher the color saturation or the stronger the contrast of the heat map), a comprehensive visualization result capable of simultaneously showing the spatial position of the change of the conveyor belt surface and the intensity trend of the change over time, that is, the conveyor belt surface short-time change map, is finally formed.

[0032] The acquisition steps of the potential longitudinal crack growth point are as follows: Based on the conveyor belt surface short-time change map, the gray value of all pixels in the map is extracted frame by frame, and a gray value histogram is constructed. The occurrence frequency of the gray value in each interval is counted, the probability distribution of the frequency of each interval is calculated, and the gray value probability distribution is sorted to obtain a gray value cumulative probability curve. Based on the gray value cumulative probability curve, the gray value corresponding to the first time when the gray value cumulative probability exceeds 95% percentile is selected as a dynamic threshold. The dynamic threshold is applied to the conveyor belt surface short-time change map for pixel gray binarization frame by frame, and the pixel region higher than the dynamic threshold is marked as a potential abnormal region to form a potential abnormal region mask. Based on the potential abnormal region mask, the geometric features of each region, including the aspect ratio and the area, are extracted, and the pixel region with an aspect ratio greater than a set value, an area exceeding a minimum area limit and a continuous boundary is selected as a potential longitudinal crack growth point.

[0033] ​​​​​​Specifically, based on the short-time change map of the conveying belt surface, the map is composed of a series of heat maps arranged in time sequence, each heat map represents a pixel gray difference value image matrix at a time point, and is associated with a corresponding unit moving distance gray scale change index The system processes these heat maps frame by frame. For each heat map (the underlying gray difference value image matrix data, the pixel value of which represents the magnitude of gray scale change), first, all the pixels in the image are traversed to extract the gray value of each pixel (here, the gray value refers to the pixel value of the difference value image, for example, 0-255, representing the degree of change), then a gray value histogram of the frame image is constructed, the horizontal axis of the histogram is the gray value, and the vertical axis is the number of pixels with the gray value, the interval division of the gray value is set according to the actual situation, for example, for a gray scale range of 0-255, it can be divided into 256 intervals, each interval width is 1, or it can be divided into fewer intervals, such as 32 intervals, each interval width is 8, the frequency (i.e. the number of pixels) of the pixels appearing in each preset gray value interval is counted, then the frequency of each interval is divided by the total number of pixels in the frame image to obtain the probability of the occurrence of the gray value in the interval, forming a probability distribution of the frequency of each interval, finally, the probabilities of these gray value intervals are accumulated in order from small to large gray value to calculate the cumulative probability corresponding to each gray value, and these cumulative probability values and the corresponding gray values are plotted into a curve, which is the gray value cumulative probability curve of the frame image.

[0034] Based on the gray value cumulative probability curve of each frame image obtained in the previous paragraph, a dynamic threshold is determined for each frame image. The specific operation is to find the first gray value that makes the cumulative probability exceed or equal to 95% from the low end of the gray value cumulative probability curve of the frame, and set this gray value as the dynamic segmentation threshold of the current frame. For example, if the cumulative probability corresponding to the gray value 180 is 94.8%, and the cumulative probability corresponding to the gray value 181 is 95.1%, then the dynamic threshold is set to 181. The selection of the 95% percentile is based on experience and aims to exclude most of the background or non-significant change areas in the image and retain a few pixels with relatively intense gray scale changes. Then, the calculated dynamic threshold is applied to the original gray difference value image (i.e. the original data of the heat map) in the short-time change map of the conveying belt surface corresponding to the frame to perform binaryzation processing on each pixel in the image: if the gray value (difference value) of the pixel is greater than the dynamic threshold, the pixel is marked as 1 (or white), indicating that the pixel belongs to the potential abnormal area; if the gray value of the pixel is less than or equal to the dynamic threshold, the pixel is marked as 0 (or black), indicating that the pixel belongs to the normal area. After this processing, the original gray difference value image is converted into a binary image, and the area marked as 1 constitutes a potential abnormal area mask.

[0035] Based on the frame-by-frame potential abnormal area mask generated in the previous step, which marks the area where the pixel gray value changes beyond the dynamic threshold, then the geometric feature analysis is performed on each connected potential abnormal area in the mask, that is, the connected block composed of pixels with a value of 1. First, all independent white areas in the image are identified using the connected component labeling algorithm. For each identified independent area, the minimum circumscribed rectangle is calculated, and the aspect ratio of the area is calculated according to the length and width of the rectangle. At the same time, the total number of pixels contained in the area is calculated as its area. Next, the extracted geometric features are screened. The set value of the aspect ratio needs to be determined according to the typical shape of the longitudinal crack on the actual conveying belt. For example, through the analysis of historical crack images, it is found that the longitudinal crack usually appears in an elongated shape, with a length much greater than the width. Therefore, the set value of the aspect ratio can be set to greater than 5, for example, set to 8. That is, only when the aspect ratio of an area is greater than 8 is it retained. The minimum area limit is set to filter out small areas caused by noise or minor irrelevant changes. The value can be estimated according to the width of the conveyor belt and the resolution of the camera. For example, if the minimum crack length detected is 5 cm, and each pixel on the image represents 0.1 cm, the number of pixels corresponding to the minimum length is 50 pixels. Considering that the crack width is at least a few pixels, the minimum area limit can be set to square pixels. Boundary continuity is determined by checking whether the boundary pixels of the area form one or a few closed or nearly closed contours. The connectivity within the area is usually guaranteed by the connected component labeling algorithm itself. For the screening conditions, an area must meet the following conditions simultaneously: its aspect ratio is greater than the set aspect ratio threshold (e.g., 8), its area exceeds the set minimum area limit (e.g., 150 square pixels), and its boundary exhibits a certain continuity on the image (excluding excessively fragmented or scattered point sets). The pixel area that meets all these conditions is finally determined as a potential longitudinal crack growth point.

[0036] The device failure critical feature value acquisition step is: Based on the position data of the potential longitudinal crack growth point, the edge pixel value in the corresponding conveyor belt surface image is extracted along the running direction of the conveyor belt. The difference between the gray value of each pixel point and the gray value of the adjacent pixel point is calculated and taken as the absolute value. The difference value is accumulated point by point to form the edge gradient cumulative value of the longitudinal crack growth point position. Based on the edge gradient cumulative value of the longitudinal crack growth point position, the boundary point coordinates of the roller end face contour are extracted frame by frame in combination with the roller end face contour image data of the corresponding continuous time frames. The rotation angle change value of each roller end face contour center point between adjacent frames is calculated by fitting the ellipse center of the roller end face contour boundary points. The rotation angle change values of continuous multiple frames are accumulated to form the angular displacement of the roller end face contour. Based on the edge gradient cumulative value of the angular displacement of the roller end face profile and the position of the longitudinal crack growth point, the numerical sequence of the two parameters is combined to obtain the key characteristic value of the equipment failure.

[0037] Specifically, based on the position data of the potential longitudinal crack growth point determined in the previous stage, which usually includes the center coordinates of the growth point in the image or its contour pixel set, the system first needs to determine the running direction of the conveyor belt in the image, which can be obtained by analyzing the displacement of the feature points in consecutive multiple frames of images or by pre-calibration according to the geometric relationship of the camera installation. Once the running direction is determined (for example, the vertical direction of the image upwards represents the forward progress of the conveyor belt), for each potential longitudinal crack growth point, the system will extract the pixel gray value on one or more line segments in the growth point area or its adjacent area from the corresponding original conveyor belt surface image (i.e. the "near-infrared band fusion image" mentioned earlier) along the running direction. The length of the line segment can be set according to the size of the potential crack growth point, for example, 1.2 times its length, and the width can be a few pixels (e.g. 3 to 5 pixels) for averaging. For each pixel point on the line segment (or multiple line segments), calculate the absolute value of the difference between its gray value and the gray value of the previous adjacent pixel point on the line segment, which represents the local gray level change intensity, i.e. the edge strength. Add up all the gray difference absolute values between adjacent pixels along the line segment (or the average of multiple line segments) to obtain a total value, which is the edge gradient cumulative value of the potential longitudinal crack growth point position.

[0038] Based on the edge gradient cumulative value of each potential longitudinal crack growth point position calculated as described above, and combined with the roller image data of the corresponding consecutive time frames extracted from the "equipment running state representation data" (especially the roller end face profile obtained after edge extraction processing), the dynamic behavior of the roller is analyzed. For each frame of roller image in the time sequence, first extract the boundary point coordinate set constituting the roller end face profile from the existing roller end face edge pixel data, use these boundary point coordinates to fit a best ellipse through the least squares method to accurately represent the end face profile of the roller, and calculate the center point coordinates of the fitted ellipse and the long and short axis direction or the principal axis direction angle of the ellipse, for two consecutive frames of images (e.g. frame and frame ), the two ellipse center points and , and the two principal axis direction angles and If the roller is rotating normally and the camera is fixed, the center of the end face ellipse should remain relatively stable or only have a slight displacement in the image, and the main axis direction angle will change with the rotation of the roller. Here, the calculated value is the change in the rotation angle of the center point of the roller end face profile between adjacent frames, which usually refers to the change in the angular position of a specific marker on the roller surface (if visible and stable) by comparing the continuous frames, or in the absence of obvious markers, by analyzing the rotation angle of the entire profile (for example, if the roller end face has an incomplete symmetric wear or an attached object causing profile feature points, the angular displacement of these feature points can be tracked, or the overall rotation angle can be estimated by more complex image registration techniques). If the main axis direction angle is used, the calculation is This is the change in the rotation angle between single frames. Then, the single-frame rotation angle change values of multiple consecutive frames (for example, a preset time window, such as covering the time from a certain point on the conveyor belt entering the camera's field of view to leaving the field of view, or a fixed number of frames, such as 10 frames) are accumulated and summed to obtain the total angular displacement of the roller end face profile within a certain time period.

[0039] Based on the angular displacement sequence of the roller end face profile calculated in the previous steps (each time point or each small time window corresponds to an angular displacement value), and the edge gradient accumulation value sequence of the potential longitudinal crack growth point position calculated on the conveyor belt surface within the same time frame or time window (if there are multiple growth points in the same frame, the maximum value, average value, or each value can be recorded), the system combines these two different source but time-synchronized parameter value sequences. The specific combination method can be to directly store the two sequences side by side to form a two-dimensional or higher-dimensional feature vector sequence, or to align them by timestamp to form a record set containing the changes of the two parameters over time. Each record in this set contains the roller angular displacement and crack edge gradient accumulation value at a specific time. This entire time-ordered combined parameter value sequence constitutes the device failure key feature value.

[0040] The acquisition step of the failure mode quantitative evaluation result is: Based on the device failure key feature value, extract the numerical value of each feature index in the device failure key feature value sequence, calculate the statistical quantity of each index value, including the maximum value, average value, and variance value of the feature value sequence, and record the statistical quantity corresponding to the feature index name, forming the device failure feature statistical result. Based on the statistical results of equipment failure characteristics, the current operating speed data of the conveyor belt is called. According to the current operating speed value of the conveyor belt, the safe value range of the historical characteristic statistical results that match the speed range is found. The limit threshold that is suitable for the current operating speed is determined and compared with the statistical quantities of each indicator of the equipment failure characteristic statistical results one by one. The statistical indicators that exceed the limit threshold are marked to form the threshold comparison judgment result. Based on the threshold comparison results, the number of statistical indicators and the names of characteristic indicators that exceed the limit threshold are recorded. According to the number of statistical indicators and the names of characteristic indicators that exceed the dynamic threshold, the corresponding failure mode category and quantification level are determined and matched in combination with the predefined failure mode risk mapping rules to form the failure mode quantitative assessment result.

[0041] Specifically, based on the key feature values ​​of equipment failure obtained in the previous step, this is a sequence of combined parameter values ​​arranged in chronological order, including "angular displacement of the idler roller end face profile" and "cumulative edge gradient value of potential longitudinal crack growth point location". First, extract the numerical lists of these two feature indicators from this sequence. For example, obtain the time series of an angular displacement value. A time series of marginal gradient accumulation values ,in It is the length of the sequence, corresponding to a specific observation time window. Then, statistical analysis is performed independently on the numerical sequence of each feature indicator to calculate its value within this time window. The maximum value at each observation point (e.g., the maximum angular displacement in the angular displacement sequence). The maximum cumulative gradient value in the sequence of marginal gradient cumulative values ), average (e.g., the average of an angular displacement sequence) The average value of the cumulative edge gradient sequence ), and variance values ​​(e.g., the variance of the angular displacement sequence). Variance of the cumulative marginal gradient sequence The calculated statistics (maximum value, average value, variance) are associated with their corresponding original feature index names ("angular displacement of roller end face profile", "cumulative edge gradient value of potential longitudinal crack growth point location") and recorded to form the statistical results of equipment failure characteristics.

[0042] Based on the device failure feature statistics results formed in the previous paragraph, which include, for example, the maximum value, average value, and variance value of the "angular displacement of the end face profile of the carrier roller", and the maximum value, average value, and variance value of the "edge gradient cumulative value of the potential longitudinal crack growth point position", the system then calls the current running speed data of the conveyor belt from the conveyor belt speed monitoring system in real time, for example, the current speed is 2.8 meters / second. Then, the system accesses a pre-established historical feature statistics database or rule base, which stores the numerical value ranges of each device failure feature statistics indicator under different conveyor belt running speed intervals during normal or safe operation. These historical safe numerical value ranges are obtained by long-term monitoring of the conveyor system under normal operating conditions, collecting a large amount of device failure key feature value data, and performing segmented statistical analysis according to the conveyor belt speed. For example, for the speed interval of 2.5 to 3.0 meters / second, historical data shows that the normal average value of "angular displacement of the end face profile of the carrier roller" is usually between 0.5 and 1.5 degrees / second, the maximum value is not more than 3 degrees / second, and the variance is not more than 0.25 (degrees / second) , while the average value of "edge gradient cumulative value of the potential longitudinal crack growth point position" should be less than 5000, the maximum value should be less than 10000, and the variance should be less than The system finds the corresponding speed interval (2.5 to 3.0 meters / second) according to the current obtained conveyor belt running speed (2.8 meters / second), and extracts the upper limit (or lower limit, depending on the meaning of the indicator) of the safe numerical value range of all related feature statistics indicators in this interval as the limiting threshold for the current judgment, for example, for the average value of "angular displacement of the end face profile of the carrier roller", the limiting threshold is 1.5 degrees / second. Compare each indicator statistic in the current calculated device failure feature statistics results (such as the current average angular displacement of 1.8 degrees / second) with these limiting thresholds determined from historical data and adapted to the current running speed one by one. If the value of a certain statistic exceeds its corresponding limiting threshold range (for example, 1.8 degrees / second > 1.5 degrees / second), mark that statistic as abnormal, and record all marked statistics and their exceeding conditions to form the threshold comparison judgment results.

[0043] Based on the threshold contrast judgment result generated in the previous step, which clearly indicates which device failure feature statistical indicators have current values that exceed the limit threshold set according to the current conveyor belt running speed, the system first records the number of statistical indicators that exceed the limit threshold and the specific names of these indicators, for example, it is recorded that there are 3 statistical indicators that exceed the limit, which are: the average value of "the angular displacement of the end face profile of the carrier roller", the maximum value and average value of "the edge gradient cumulative value of the potential longitudinal crack growth point position", then, the system queries a pre-defined failure mode risk mapping rule table according to the number of these over-limit statistical indicators and which indicators exceed the limit, this rule table is established based on expert experience, device maintenance manual and historical failure data analysis, it associates different combinations of over-limit indicators with specific device failure modes and their risk levels, for example, the rule table defines: if only the average value and maximum value of "the edge gradient cumulative value of the potential longitudinal crack growth point position" exceed the limit, it may correspond to "early wear or slight crack on the surface of the conveyor belt", the risk level is "low"; if the average value and variance of "the angular displacement of the end face profile of the carrier roller" both exceed the limit, and the related indicators of "the edge gradient cumulative value of the potential longitudinal crack growth point position" also exceed the limit, it may correspond to "carrier roller jam or damage causing serious scratch or risk of serious crack expansion on the conveyor belt", the risk level is "high", the system finds and matches the most consistent entry in the risk mapping rule table according to the actual number and name combination of the indicators that exceed the limit (i.e. the 3 indicators and their names recorded earlier), thereby determining the most likely failure mode category (such as "risk of longitudinal tear of the conveyor belt" or "damage to the belt surface caused by carrier roller failure") and the quantitative risk level (such as 1-5 levels, 1 being the lowest risk and 5 being the highest risk, or directly described as "warning", "general", "serious", etc.) of the current conveyor system, finally forming the failure mode quantitative evaluation result of this evaluation.

[0044] The acquisition step of the persistent device abnormal state is: Based on the failure mode quantitative evaluation result, select a sequence of conveyor belt image frames for the past 30 minutes, extract 1 frame of image per minute, obtain a total of 30 frames of images and timestamp information, and extract the failure mode quantitative evaluation result and the average edge intensity of each frame of image, forming a sequence data set; According to the sequence data set, calculate the failure trend enhancement response factor, the calculation formula is: ; Wherein, is the failure trend enhancement response factor in the 30 frame data segment, is the average edge intensity of the th frame of image, is the failure mode quantitative evaluation result of the th frame of image, represents the total number of frames, represents the acquisition timestamp of the 1st image in the sequence, represents the acquisition timestamp of the 30th image in the sequence; Based on the failure trend enhancement response factor, a time window composed of 30 consecutive frames is continuously slid, and the failure trend enhancement response factors in adjacent windows are compared. If the failure trend enhancement response factors in three consecutive windows are continuously positive and exceed a preset threshold, it is considered that an upward trend is formed, and the corresponding time period is marked as a persistent equipment abnormal state.

[0045] Specifically, based on the failure mode quantitative evaluation result generated in the preceding step, which gives a category judgment and risk level of the failure mode for each evaluated time point (or short time window), the system will then select data in a fixed-length time window for trend analysis. The specific operation is to take the current time point as the benchmark and backtrack to select the image frame sequence collected in the past 30 minutes. In order to reduce the amount of calculation and maintain the smoothness of the trend analysis, the system samples according to the rule of extracting 1 frame of image per minute, that is, from the 30-minute image sequence, 30 frames of images are selected at equal intervals, and the accurate acquisition timestamps of these selected images are recorded. For each of the 30 selected images, the system will re-call or find the failure mode quantitative evaluation result (usually a numerical risk level, such as an integer from 1 to 5, or a continuous risk score) that has been calculated at that time point. At the same time, for each selected original conveyor surface image (i.e. "near-infrared band fusion image"), the image average edge intensity is calculated. The average edge intensity can be obtained by applying a Sobel operator or other edge detection operator to the entire image to obtain an edge map, and then calculating the average of all pixel intensities in the edge map, or calculating the proportion of edge pixel number to total pixel number. Finally, the 30 frames of images are organized into a time series data set containing 30 records, each record containing (timestamp, failure mode quantitative evaluation result, image average edge intensity) these three data items.

[0046] Formula: The advantage of the formula is that it quantifies the degree of failure trend enhancement by calculating the cumulative sum of the product of the change in image average edge intensity and the change in failure mode quantitative evaluation result, and dividing by the total time span. When the changes in physical characteristics (such as edge intensity) and risk assessment results are in the same direction (i.e. both increase or both decrease), the product is positive, indicating that the physical change and the risk assessment change are consistent in trend. The accumulation of this consistency can reflect whether the failure trend is continuously enhancing or weakening, Item ensures the change range of edge intensity involved in the calculation, while Item directly reflects the change direction and range of risk assessment results, and the multiplication of the two emphasizes the synergy between physical changes and risk assessment changes, divided by the total time In order to obtain an average change rate, so that the response factors of different length time windows are comparable, and the trend is more stable; The acquisition steps of the parameters are: represents the average edge intensity of the image corresponding to the frame in the sequence data set, which is obtained by image processing on the original conveyor belt surface image (i.e. "near-infrared band fusion image") of the frame, first, apply Sobel operator to the image to calculate the gradient amplitude of each pixel point, forming a gradient image, then calculate the arithmetic mean of all pixel gradient amplitudes in this gradient image, which is the average edge intensity of the image, for example, for an 8-bit grayscale image, the average edge intensity value may float in a small range between 0 and 255, for example, the average edge intensity of the frame image calculated after Sobel operator processing is .

[0047] The acquisition steps of the parameters are: represents the failure mode quantitative assessment result corresponding to the image of the frame in the sequence data set, which is obtained in the previous step according to the "device failure key feature value" and "predefined failure mode risk mapping rule", it is a numerical value, which quantifies the risk level of the device state reflected by the image, for example, the risk level can be from 1 (no risk or extremely low risk) to 5 (extremely high risk), or a continuous risk score between 0 and 1, here is directly extracted from the "sequence data set", for example, for the image of the frame, its failure mode quantitative assessment result recorded in the sequence data set is 3 (representing medium risk), then .

[0048] The acquisition steps of the parameters are: represents the total number of frames involved in the calculation, the past 30 minutes of conveyor belt image frame sequence is selected, and 1 frame of image is extracted per minute, so the total number of frames is fixed at 30, this parameter is pre-set, for example, .

[0049] The acquisition step of the parameter is: represents the acquisition timestamp of the 1st image in the sequence dataset, which is recorded from the original image frame information when the "sequence dataset" is constructed, for example, if the acquisition time of the 1st image in the sequence is "2023-10-26 10:00:00.000", converted into Unix timestamp, for example seconds.

[0050] The acquisition step of the parameter is: represents the acquisition timestamp of the 1st image in the sequence dataset, which is recorded from the original image frame information when the "sequence dataset" is constructed, for example, if the acquisition time of the 1st image in the sequence is "2023-10-26 10:00:00.000", converted into Unix timestamp, for example seconds.

[0051] Substitute the actual parameters into the formula to get 0.5, which indicates that the average enhancement rate of the failure trend in this 30-frame data segment is 0.5, and the positive value indicates that, on average, the change in image average edge intensity is in the same direction as the change in failure mode quantitative evaluation result, and the overall trend is enhanced. If this value remains positive and exceeds a certain preset significance threshold, it may indicate that the device state is deteriorating.

[0052] Based on the failure trend enhancement response factor calculated in the previous paragraph , a sliding time window method is used to continuously monitor whether this trend is persistent, and each time window is still based on the past 30 minutes of continuous data (i.e. 30 frames of images) to calculate a value, this time window slides forward at a fixed step, for example, every 5 minutes (or shorter, such as 1 minute, corresponding to the newly collected 1 frame of data and the update of the sequence dataset) to recalculate , so a time series of values will be obtained, and the system will then compare the failure trend enhancement response factors calculated in the continuous windows (e.g. three consecutive windows) to determine whether there is a persistent upward trend. A preset threshold value, for example , this threshold value is set based on the historical normal operation and known failure development process ​​The value distribution is statistically analyzed to obtain, for example, 1.5 times the upper limit of the normal fluctuation range plus one standard deviation, or according to the fault case Typical value setting before significant increase, if the failure trend enhancement response factor calculated by the continuous three sliding windows All greater than 0 (indicating that the trend is enhanced), and each value exceeds the preset threshold (e.g. , , ), the system determines that a significant and continuous device state deterioration rising trend has been formed, and marks the entire time period covered by the three consecutive windows (from the start of the first window to the end of the third window) as a persistent device abnormal state.

[0053] The acquisition step of the conveying system fault positioning result is: Based on the abnormal state occurrence period determined by the persistent device abnormal state, the conveying belt image frame index corresponding to the abnormal state period is called, the frame acquisition time stamp recorded in the image frame index is extracted, and the real-time position information of the conveying belt encoder corresponding to the time stamp is queried according to the time stamp, forming the conveying belt encoder position sequence of the abnormal state; According to the conveying belt encoder position sequence of the abnormal state, the image frame index corresponding to the position sequence is matched one by one, the camera identification code corresponding to each image frame index is called, the camera identification code is classified and counted according to the conveying belt encoder position, and the camera identification code with the highest occurrence frequency is screened as the abnormal area key camera identification to generate a key camera identification classification result; According to the key camera identification determined in the key camera identification classification result, combined with the physical installation position information of the corresponding camera, a mapping relationship between the conveying belt encoder position and the camera installation position is established, the abnormal occurrence area corresponding to the conveying belt space position is locked, and the conveying system fault positioning result is obtained.

[0054] Specifically, based on the persistent device abnormal state determined in the preceding step and its corresponding abnormal state occurrence period, for example, determining that the period from "2023-10-26 10:15:00" to "2023-10-26 10:45:00" is a persistent device abnormal state period, the system first retrieves and calls the index information of all collected conveyor belt image frames within the abnormal state occurrence period. These image frame index information already contains the unique number of each frame image and the frame acquisition timestamp accurate to milliseconds. Then, for this batch of image frames marked as abnormal, the system extracts the frame acquisition timestamp one by one and uses these timestamps to query a synchronous recorded conveyor belt encoder database or real-time data stream. The encoder is usually installed on the drive wheel or driven wheel of the conveyor belt and can provide physical position information of a certain point on the conveyor belt relative to a certain fixed reference point (such as the starting point of the conveyor belt or a specific sensor position), usually expressed in meters or encoder pulse numbers. By matching the acquisition timestamp of each abnormal image frame with the timestamp recorded in the encoder database (finding the closest encoder reading in time), the corresponding encoder position value of the photographed area on the conveyor belt at the time the image was collected is obtained. Arrange all these conveyor belt encoder position values obtained during the abnormal state period in chronological order to form a conveyor belt encoder position sequence of the abnormal state.

[0055] According to the conveyor belt encoder position sequence of the abnormal state formed in the previous step, each position value in the sequence corresponds to an image frame collected during the persistent device abnormal state period. The system then processes each encoder position value in the position sequence one by one and finds the associated original image frame index. From each image frame index information, extract the unique identification code of the camera responsible for collecting the image (for example, camera ID "CAM001", camera ID "CAM002", etc. These IDs have been assigned to the cameras installed at different positions along the conveyor belt when the system is deployed). Then, the system classifies and counts these camera identification codes according to the conveyor belt encoder position. Specifically, a certain encoder position interval can be set (for example, every 5 meters or according to the camera field of view range), and the system counts which cameras have captured the image frames determined to be abnormal within each encoder position interval and calculates the frequency of each camera in each interval. Alternatively, more directly, the system counts the total frequency of all participating cameras in the entire abnormal state conveyor belt encoder position sequence covering the physical section, and filters out the camera identification code with the highest frequency in the entire abnormal period or in the most concentrated encoder position section. These filtered camera identification codes are identified as the key camera identification most relevant to the abnormal area. Summarize the key camera identification and its frequency or relevance score to generate a key camera identification classification result.

[0056] According to the key camera identification classification result generated in the last paragraph, which indicates which cameras most frequently capture the conveyor belt section in a persistent device abnormal state, the next step is to combine these determined key camera identifiers with the physical installation location information of each camera pre-stored in the system configuration, which is usually described in absolute coordinates in the conveyor belt system or distance relative to a reference point (for example, "CAM001" is installed 15 meters behind the transfer station of conveyor belt No. 0, and the monitoring range covers the 15-18 meter section of the conveyor belt). By associating the key camera identifier (such as "CAM001") with its corresponding physical installation location (15-18 meter section) and combining the abnormal state conveyor encoder position sequence obtained in the first step (for example, the sequence shows that the anomaly mainly concentrates in the range of 14500-14800 encoder readings, which corresponds to the 14.5-14.8 meter range of the conveyor belt according to calibration), the system establishes an accurate mapping relationship between the conveyor encoder position, camera identifier, and the actual monitoring physical space position of the conveyor belt. By integrating these information, the system can more accurately lock the specific conveyor belt space area where the anomaly occurs. For example, if the monitoring range of the key camera "CAM001" highly coincides with the physical location indicated by the abnormal encoder position sequence, it can be determined that the fault occurred in the conveyor belt section monitored by the camera. Finally, the specific conveyor system fault positioning result is obtained, such as "the fault is located on the conveyor belt surface about 14.5-14.8 meters behind the transfer station of conveyor belt No. 0, which is mainly monitored by the camera CAM001".

[0057] The above is only a preferred embodiment of the present application, and does not limit the form of the present application in other ways. Any skilled person in the art can use the disclosed technical content to make changes or modifications as equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification made according to the technical essence of the present application to the above embodiments without departing from the technical solution content of the present application still belongs to the protection scope of the present application.

Claims

1. A method for monitoring the operation of a coal conveying system of a power plant based on an Internet of Things (IoT) sensing system, characterized in that, The method comprises the following steps: Based on the conveyor belt surface image collected by the multispectral camera and the idler image collected by the camera, the surface image of the conveyor belt is fused in the near-infrared wave band, and the end face profile of the idler image is extracted, and the equipment operation state representation data is established; Based on the equipment operation state representation data, the difference operation is performed on the conveyor belt representation data of continuous time frames to obtain the conveyor belt surface short-time change graph, based on the conveyor belt surface short-time change graph, the dynamic threshold is set for segmentation processing according to the statistical distribution of the pixel value in the graph, and the potential longitudinal crack growth point is identified; Based on the position of the potential longitudinal crack growth point and the equipment operation state representation data, the edge gradient cumulative value of the growth point along the running direction of the conveyor belt is calculated, and the angular displacement of the idler profile in continuous time frames is calculated to obtain the equipment failure key characteristic value, based on the equipment failure key characteristic value, the dynamic threshold set according to the current running speed of the conveyor belt is compared and judged to generate the failure mode quantitative evaluation result; Based on the failure mode quantitative evaluation result of continuous time frames, it is judged whether there is a monotone increasing trend, and a persistent equipment abnormal state is established, based on the persistent equipment abnormal state, the conveyor belt encoder position and the idler camera identifier corresponding to the state occurrence are locked to obtain the conveyor system fault positioning result.

2. The method of claim 1, wherein the method further comprises: The acquisition step of the equipment operation state representation data is: Based on the conveyor belt surface image collected by the multispectral camera and the idler image collected by the camera, the frame number, image number and timestamp of the two types of images are recorded respectively, the time difference of adjacent frames in the recorded image number and timestamp list is screened, the image frames with a time error less than a preset tolerance range are obtained, and a timestamp aligned image frame pair is obtained; Based on the timestamp aligned image frame pair, the image level channel information corresponding to the near-infrared wave band in the conveyor belt surface image is extracted, the channel information is subjected to gray scale normalization processing according to the pixel bit serial number, and then superimposed, the superimposed gray scale value and the image coordinates are bound to generate a fusion image during the fusion process, and a near-infrared wave band fusion image is obtained; Based on the near-infrared wave band fusion image, the idler image in the corresponding timestamp aligned image frame pair is combined to extract the edge of the idler end face, the curvature value, closedness and main shaft direction continuous change characteristics of the end face edge line are used to construct a structure feature vector associated with the idler state, and the structure feature vector and the pixel average brightness value and slope gradient value in the near-infrared wave band fusion image are combined and arranged as a multi-dimensional vector set according to the frame index number to generate the equipment operation state representation data.

3. The method of claim 1, wherein the method further comprises: The acquisition step of the conveyor belt surface short-time change graph is: Based on the equipment operation state representation data, the conveyor belt image sequence within a unified idler camera identifier and a synchronous timestamp range is extracted, the image sequence is arranged in ascending order according to the image frame number, the pixel gray scale matrix information of each frame image is read, and the shooting time difference of adjacent frame images and the running speed of the conveyor belt at the corresponding time point are recorded to form a continuous time frame data group; According to the continuous time frame data group, the unit moving distance gray scale change index between adjacent frames is calculated; According to the unit moving distance gray change index, the gray difference image of each frame is double-mapped with the corresponding unit moving distance gray change index on the time axis, the gray image matrix is expressed in the form of heat mapping, and a time sequence mapping set is generated by taking the unit moving distance gray change index sequence as the change amplitude index to form the short-time change atlas of the conveyor belt surface.

4. The method of claim 1, wherein the method further comprises: The obtaining step of the potential longitudinal crack growth point is: Based on the short-time change atlas of the conveyor belt surface, the gray values of all pixels in the atlas are extracted frame by frame to construct a gray value histogram, the occurrence frequencies of the gray values in each interval are counted, the probability distribution of the frequencies in each interval is calculated, and a gray value cumulative probability curve is obtained by sorting the gray value probability distribution; Based on the gray value cumulative probability curve, the gray value corresponding to the first time when the cumulative probability exceeds 95% is selected as a dynamic threshold, the dynamic threshold is applied to the short-time change atlas of the conveyor belt surface for pixel gray binarization frame by frame, and the pixel regions higher than the dynamic threshold are marked as potential abnormal regions to form a potential abnormal region mask; Based on the potential abnormal region mask, the geometric features of each region, including the aspect ratio and the area, are extracted, the pixel regions with an aspect ratio greater than a set value, an area exceeding a minimum area limit and a continuous boundary are screened, and the pixel regions are determined as potential longitudinal crack growth points.

5. The method of claim 1, wherein the method further comprises: The obtaining step of the equipment failure key feature value is: Based on the position data of the potential longitudinal crack growth point, the edge pixel values in the corresponding conveyor belt surface image are extracted along the running direction of the conveyor belt, the difference between the gray value of each pixel point and the gray value of the adjacent pixel point is calculated and the absolute value is taken, the difference is accumulated point by point to form an edge gradient cumulative value of the longitudinal crack growth point position; Based on the edge gradient cumulative value of the longitudinal crack growth point position, the boundary point coordinates of the roller end face profile are extracted frame by frame in combination with the roller end face profile image data of the corresponding continuous time frames, the center of the ellipse fitting the boundary points of the roller end face profile is calculated, the rotation angle change value of each center point of the roller end face profile between adjacent frames is calculated, the rotation angle change values of continuous multiple frames are accumulated to form the angular displacement of the roller end face profile. Based on the angular displacement of the roller end face profile and the edge gradient cumulative value of the longitudinal crack growth point position, the numerical sequences of the two parameters are combined to obtain the equipment failure key feature value.

6. The method of claim 1, wherein the method further comprises: The obtaining step of the failure mode quantitative evaluation result is: Based on the equipment failure key feature value, the numerical values of each feature index in the equipment failure key feature value sequence are extracted, the statistics of each index value are calculated, including the maximum value, the average value and the variance value of the feature value sequence, and the feature index name corresponding to the statistics is recorded to form the equipment failure feature statistical result; Based on the device failure feature statistical result, the current running speed data of the conveying belt is called, and according to the current running speed value of the conveying belt, the safety value range of the historical feature statistical result matched with the speed interval is searched, the limited threshold value suitable for the current running speed is determined, and each index statistical quantity of the device failure feature statistical result is compared one by one, the statistical index exceeding the limited threshold value is marked, and the threshold value comparison judgment result is formed; Based on the threshold value comparison judgment result, the number and feature index name of the statistical index exceeding the limited threshold value are recorded, the corresponding failure mode category and quantitative level are judged and matched according to the number and feature index name of the statistical index exceeding the dynamic threshold value, combined with the pre-defined failure mode risk mapping rule, and the failure mode quantitative evaluation result is formed.

7. The method of claim 1, wherein the method further comprises: The acquisition step of the persistent device abnormal state is: Based on the failure mode quantitative evaluation result, a sequence of conveying belt image frames in the past 30 minutes is selected, 1 frame of image is extracted per minute, a total of 30 frames of images and timestamp information are acquired, and the failure mode quantitative evaluation result and image average edge intensity of each frame of image are extracted, and a sequence data set is formed; According to the sequence data set, a failure trend enhancement response factor is calculated; Based on the failure trend enhancement response factor, a time window composed of every 30 frames is continuously slid, the failure trend enhancement response factors in adjacent windows are compared, and if the failure trend enhancement response factors in the continuous three windows are continuously positive and exceed the preset threshold, it is considered that an upward trend is formed, and the corresponding time period is marked as a persistent device abnormal state.

8. The method of claim 1, wherein the method further comprises: The acquisition step of the conveying system fault positioning result is: Based on the abnormal state occurrence period determined by the persistent device abnormal state, the image frame index corresponding to the abnormal state period is called, the frame acquisition timestamp recorded in the image frame index is extracted, and the real-time position information of the conveying belt encoder corresponding to the timestamp is queried according to the timestamp, and a conveying belt encoder position sequence of the abnormal state is formed; According to the conveying belt encoder position sequence of the abnormal state, the image frame index corresponding to the position sequence is matched one by one, the camera identification code corresponding to each image frame index is called, the camera identification code is classified and counted according to the conveying belt encoder position, and the camera identification code with the highest occurrence frequency is selected as the key camera identification of the abnormal area, and a key camera identification classification result is generated; According to the key camera identification determined in the key camera identification classification result, combined with the physical installation position information of the corresponding camera, a mapping relationship between the conveying belt encoder position and the camera installation position is established, the abnormal occurrence area corresponding to the conveying belt space position is locked, and the conveying system fault positioning result is obtained.

Images