Thermal imaging and multi-dimensional quality screening of a scraper pull-off broken link identification system and method

Abstract

This application discloses a scraper deflection and chain breakage identification system and method based on thermal imaging and multi-dimensional quality screening, relating to the field of industrial equipment fault detection technology. The system includes a thermal imaging data acquisition module that acquires thermal imaging images or videos of a scraper conveyor and outputs raw data containing thermal features of the chain and scraper; a multi-dimensional quality screening module that preprocesses the raw data and outputs a valid image dataset; an instance segmentation model training module that trains a recognition model based on the dataset and outputs the trained model; a mask post-processing module that uses the model to infer real-time images and outputs a target mask; a line fitting module that processes the target mask and fits and outputs a straight line equation; a chain breakage feature detection module that performs chain link continuity analysis, gap detection, and thermal feature mutation identification and outputs a chain breakage judgment result; and an angle calculation and comprehensive judgment module that calculates three types of angles and, combined with the mask integrity and chain breakage result, outputs a warning level and judgment result.

Description

Technical Field

[0001] This application relates to the field of industrial equipment fault detection technology, and more specifically, to a scraper pull-off chain breakage identification system and method using thermal imaging and multi-dimensional quality screening. Background Technology

[0002] As a key transportation equipment in industrial settings such as coal mining and coal mining, the operational stability of scraper conveyors directly affects production efficiency and operational safety. Scraper misalignment and chain breakage are the two most common serious failures of scraper conveyors. Misalignment can lead to chain wear and scraper deformation, while chain breakage can cause major accidents such as equipment shutdown, material accumulation, and even personal injury or death.

[0003] In view of this, there is an urgent need for a scraper pull-off chain breakage identification system and method that combines thermal imaging and multi-dimensional quality screening. Summary of the Invention

[0004] The main objective of this application is to provide a scraper pull-off chain breakage identification system and method based on thermal imaging and multi-dimensional quality screening, so as to solve the problem of scraper pull-off chain breakage identification.

[0005] To achieve the above objectives, the first aspect of this application proposes a scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening, comprising: The thermal imaging data acquisition module is used to acquire thermal imaging images or videos of the scraper conveyor during operation using a thermal imaging camera, and output raw data containing the thermal characteristics of the chain and scraper. A multi-dimensional quality screening module is connected to the thermal imaging data acquisition module. It is used to receive the raw data, preprocess the raw thermal imaging data, and output a valid image dataset that meets the preset quality requirements. The instance segmentation model training module is connected to the multi-dimensional quality screening module. It is used to construct a scraper and chain recognition model, and to train it using an effective image dataset, and output the trained instance segmentation model. The mask post-processing module, connected to the multi-dimensional quality screening module, is used to infer the real-time thermal imaging image acquired by the thermal imaging data acquisition module using the trained instance segmentation model, extract the binary mask of the chain and the scraper, remove noise and holes through morphological operations, and output the target mask and mask integrity. The straight line fitting module, connected to the mask post-processing module, is used to extract the skeleton and filter noise from the target mask, obtain the coordinate set of the center line of a single pixel in the target mask, and use a fusion method of initial fitting by least squares method and correction by Hough transform to output the straight line equation of the chain and the scraper based on the solution of the straight line equation and the distance formula. The chain breakage feature detection module, connected to the straight line fitting module, is used to output the chain breakage determination result based on the target mask and the single pixel center line coordinate set, through chain link continuity analysis, break gap detection and thermal feature mutation identification of the chain. The angle calculation and comprehensive judgment module is connected to the straight line fitting module and the chain break feature detection module, respectively. It is used to calculate the parallelism angle of the chain on both sides of the scraper, the perpendicularity angle between the chain and the scraper, and the angle between the chain and the frame baseline based on the formula of the normal vector angle of the straight line equation. It also combines the mask integrity and the chain break judgment result to perform pull deviation and chain break classification judgment, and output the warning level and judgment result.

[0006] In some possible implementations, the thermal imaging data acquisition module is configured as follows: Using a thermal imaging camera that vibrates parallel to the plane of the chain and at the same frequency, the thermal radiation differences between the chain, the scraper, and the surrounding environment, as well as the abrupt changes in thermal characteristics at the chain breakage point, are collected during the operation of the scraper conveyor under different ambient light and material coverage conditions. Thermal imaging images or videos are generated, and raw data containing the thermal characteristics of the chain and scraper are output. The different ambient light includes natural daylight, nighttime artificial light, strong light reflection, and local shadow conditions. The material coverage includes material occlusion of 0, material coverage of the chain within a first threshold, and material coverage of the chain within a second threshold. The first threshold is greater than 0 and less than the second threshold.

[0007] In some feasible implementations, the multi-dimensional quality screening module is configured as follows: The system receives the raw data and extracts image frames from the thermal imaging image or video frames from the thermal imaging video according to the set frame rate, forming an image frame set. In the target image frame set, a first preprocessing is performed to obtain an intermediate image frame set. The first preprocessing includes calculating the dHash value of the target image frame and removing duplicates based on the dHash value, retaining clear frames. The intermediate image frame set is subjected to a second preprocessing to output a valid image dataset that meets the preset quality requirements. The second preprocessing includes the removal of sharpness, extreme frames, and thermally blurred frames.

[0008] Some feasible approaches also include a data annotation and preprocessing module, which is connected to the multi-dimensional quality screening module and the instance segmentation model training module, respectively. The data annotation and preprocessing module is used to receive the effective image dataset, perform polygon contour annotation on the left and right chains of the chain and the scraper in each effective image in the effective image dataset, and output the labeled effective image dataset to the instance segmentation model training module. The labeled effective image dataset includes a training set, a validation set and a test set.

[0009] In some possible implementations, the instance segmentation model training module is configured as follows: Based on the YOLOv8-Seg.pt pre-trained model, a scraper and chain recognition model is constructed, and parameters are configured for the scraper and chain recognition model to obtain the scraper and chain recognition model; Using the labeled valid image dataset, the scraper and chain recognition model is trained, and the convergence of the training loss function and the intersection-union ratio index is monitored. When the convergence reaches the preset convergence value, the trained instance segmentation model is output.

[0010] In some possible implementations, the mask post-processing module is configured as follows: Using n×n rectangular structuring elements, morphological closing operations are performed to fill in the holes in the binary mask of the chain and the scraper, and the holes in the binary mask of the chain and the scraper are extracted by inference on the real-time thermal imaging image acquired by the thermal imaging data acquisition module using the trained instance segmentation model, thus obtaining the continuous outline of the chain and the scraper. Morphological opening operations are performed using m×m rectangular structuring elements to remove noise from the continuous contours of the chain and the scraper, resulting in a target mask. The target mask represents a binarized image containing the target regions of the chain and the scraper. The contour area of ​​the target mask is calculated using the polygon area formula to obtain the maximum contour area. The polygon area formula is as follows: For the coordinates of the contour vertices ,area ; The mask integrity is obtained based on the maximum contour area, wherein when the maximum contour area is greater than or equal to a preset contour area value, the mask integrity is considered complete; otherwise, the mask integrity is considered incomplete.

[0011] In some possible implementations, the line fitting module is configured as follows: Using the Zhang-Suen thinning algorithm, the target mask is transformed into a single-pixel centerline while preserving the contour topology of the target mask; The isolated line segments of the single-pixel centerline are filtered out, and the contour arc lengths that are greater than or equal to a preset pixel length are retained to obtain the target single-pixel centerline. The formula for calculating the contour arc length is as follows: For consecutive vertices and Arc length L = ; in, This represents the x-axis coordinate of pixel i within the center line of a single pixel. The arc length L represents the y-axis coordinate of pixel i in the center line of a single pixel, and the arc length L represents the total length of the outline of the center line of a single pixel. Based on the set of centerline pixel coordinates corresponding to the centerline of the target single pixel, the least squares method is used to fit the straight line equation to obtain the initial fitted straight line equation. The Hough transform is used to detect straight lines with a length greater than or equal to 50 pixels in the center line of the target single pixel, the angle of the main direction is determined, and the deviation between the angle of the main direction and the angle of the straight line formed by the initial fitted straight line equation is calculated to obtain the calculation result. If the deviation of the calculation result is greater than the preset angle deviation threshold, then after removing the abnormal points of pixels whose distance from the main direction is greater than the preset angle deviation threshold, the straight line equation of the chain and the scraper is refitted and output. If the deviation of the calculation result is less than or equal to the preset angle deviation threshold, the initial fitted straight line equation is directly used to output the straight line equation of the chain and the scraper.

[0012] In some implementable methods, the chain break feature detection module is configured as follows: Based on the target mask of the chain, the number of connected components in the chain outline is calculated. When the number of connected components is ≥2 and the distance between adjacent connected components is greater than a preset distance threshold, it is determined to be a suspected chain break. The distance between adjacent connected components represents the center-to-center distance between the minimum bounding rectangles of two connected components, and the formula is: ; Where d represents the distance between the centers of the minimum bounding rectangles of the two connected components. and Indicates the coordinates of the center of the first connected domain. and Indicates the coordinates of the center of the second connected domain; The distance between adjacent pixels is calculated based on the coordinate set of the single pixel centerline. When the distance is greater than a preset gap threshold and there is no material obstruction in the corresponding area, the break gap between adjacent pixels is obtained. The formula for the distance between adjacent pixels is: ; in, Indicates the distance between adjacent pixels. and This represents the coordinates of the i-th pixel. and Represented as the coordinates of the j-th pixel; Based on the calorific value data of the fracture gap and the chain links on both sides, the calorific value difference rate of the chain links on both sides is calculated and compared with a preset reference temperature range. When the calorific value difference rate is greater than a preset difference threshold, or when the temperature of the fracture gap exceeds the reference temperature range and reaches a preset temperature threshold, it is determined to be a thermal characteristic abrupt change. The formula for calculating the calorific value difference rate is as follows: ; in, Indicates the difference rate of calorific value. This represents the average heat value of the fracture gap region. This indicates the average heat value of the normal area of ​​the chain; The chain breakage determination result is obtained by comprehensively judging the suspected chain breakage, the fracture gap, and the thermal characteristic mutation. If the chain breakage determination result satisfies the suspected chain breakage result, the existence of at least one fracture gap, and the thermal characteristic abrupt change, a first output chain breakage determination result is obtained, wherein the first output chain breakage determination result represents either a broken chain or a non-broken chain. If the chain breakage determination result satisfies the suspected chain breakage result and the existence of at least one breakage gap, then a supplementary determination is made in combination with the scraper running speed to obtain a second output chain breakage determination result. The second output chain breakage determination result indicates that when the scraper running speed is ≥ the preset speed, it is determined to be suspected breakage, and when the scraper running speed is < the preset speed, it is determined to be under observation.

[0013] In some feasible implementation methods, the angle calculation and comprehensive determination module is configured as follows: Based on the formula for the included angle of the normal vector according to the equation of the straight line, the parallelism angle of the chain on both sides of the scraper, the perpendicularity angle of the chain to the scraper, and the angle between the chain and the frame reference line are calculated. The formula for calculating the included angle of the normal vector is as follows: ; in, , This represents the normal vector between two lines. Represents the dot product. and Indicates the modulus length. , Represent the equation of the first straight line The coefficients of the x and y terms in the equation. , Indicate the equation of the second line The coefficients of the x and y terms in the equation, where x represents the x-axis coordinate of the pixel, y represents the y-axis coordinate of the pixel, and c represents the constant term of the equation of the line; The chain parallelism angle Based on the equations of the left and right chain lines, the angle between the normal vectors is calculated using the formula: Equation of the straight line of the left chain The calculation formula is ; Equation of the straight line of the right chain The calculation formula is ; Calculated using the formula for the included angle of the normal vectors; in, Indicates the straight line of the left chain. The constant term, Indicates the straight line of the chain on the right. The constant term, , They represent lines respectively. and The normal vector; The perpendicularity angle between the chain and the scraper Based on the equations of the chain and the scraper, the angle between the normal vectors is calculated using the formula: Chain straight line equation The calculation formula is ; The formula for calculating the straight line of the scraper is: ; Calculated using the formula for the included angle of the normal vectors; in, and Represents a straight chain The coefficients of the x and y terms in the equation. Represents a straight chain The constant term, and Indicates the straight line of the scraper The coefficients of the x and y terms in the equation. Indicates the straight line of the scraper The constant term, , They represent lines respectively. and The normal vector; The angle between the chain and the frame baseline Based on the equations of the chain straight line and the frame baseline straight line, the angle between the normal vectors is calculated using the formula: Chain straight line equation The calculation formula is ; Rack baseline The calculation formula is ; Calculated using the formula for the included angle of the normal vectors; in, and Indicates the rack baseline The coefficients of the x and y terms in the equation. Indicates the rack baseline The constant term, , They represent lines respectively. and The normal vector; The integrity of the mask is obtained, and the chain breakage determination result is combined to perform a pull-off and chain breakage classification determination to obtain the output warning level and determination result. The chain breakage classification determination includes at least a first-level warning, a second-level warning, and a normal condition. The first-level warning indicates that the chain breakage determination result is a chain breakage, or the parallelism angle of the two sides of the chain is greater than the first angle and the angle between the chain and the frame baseline is greater than the second angle. The second-level warning indicates that the second angle is less than the parallelism angle of the two sides of the chain and less than the first angle, or the perpendicularity angle of the chain and the scraper is less than the third angle, or the perpendicularity angle of the chain and the scraper is greater than the fourth angle, or the chain breakage is suspected but does not meet the thermal characteristic change condition, or the mask integrity is incomplete. The normal condition indicates that the parallelism angle of the two sides of the chain is less than the second angle, the third angle is less than the perpendicularity angle of the chain and the scraper and less than the fourth angle, the angle between the chain and the frame baseline is less than the fifth angle, and the chain breakage determination result is no chain breakage.

[0014] Secondly, this application provides a method for identifying broken chains by scraper pull using thermal imaging and multi-dimensional quality screening, applied to the aforementioned system, the method comprising: S1: Acquire thermal imaging images or videos of the scraper conveyor during operation using a thermal imaging camera, and output raw data containing the thermal characteristics of the chain and scraper. S2: Preprocess the raw data to output a valid image dataset that meets the quality requirements. The preprocessing includes image generation, deduplication, bad sample filtering, quality score calculation, and valid images. S3: Use LabelStudio to annotate the left chain, right chain and scraper in the valid image dataset with polygonal outlines, and convert the annotated JSON format data into YOLOv8-Seg supported format data, and then divide the valid image dataset to form training data; S4: Construct a YOLOv8-Seg model based on YOLOv8-Seg, configure training parameters, train using training data, monitor the training metrics of the YOLOv8-Seg model, and output the trained instance segmentation model when the YOLOv8-Seg model converges. S5: Use the instance segmentation model to infer the real-time thermal imaging image, extract the binary mask of the chain and the scraper, remove noise and holes through morphological operations, and output the target mask and mask integrity. S6: Extract the skeleton and filter noise from the target mask to obtain the coordinate set of the center line of a single pixel in the target mask. Using the fusion method of least squares and Hough transform, based on the solution of the straight line equation and the distance formula, output the straight line equation of the chain and the scraper. S7: Based on the target mask and the coordinate set of the center line of a single pixel, the chain breakage determination result is output by analyzing the continuity of chain links, detecting the break gap, and identifying abrupt changes in thermal features, combined with the spacing formula and the heat value difference rate formula. S8: Calculate three types of core angles based on the formula of the angle between the normal vectors, and combine the mask integrity and chain breakage judgment results to make graded judgments on pull deviation and chain breakage, and output the warning level and judgment result. The three types of core angles include the chain parallelism angle, the chain perpendicularity angle to the scraper, and the angle between the chain and the frame baseline.

[0015] The technical solution provided in this application may include the following beneficial effects: This application integrates thermal imaging data acquisition, multi-dimensional quality screening, instance segmentation, linear fitting, and multi-feature fault detection technologies to accurately identify two core faults of scraper conveyors: misalignment and chain breakage, in complex industrial scenarios such as coal mines and coal mines. The system overcomes the limitations of traditional fault identification technologies, which are heavily influenced by operating conditions, have high dependence on fault samples, and struggle to collaboratively identify both types of faults. It can output tiered early warning results in real time, effectively preventing chain wear and scraper deformation caused by misalignment, as well as major accidents such as equipment downtime, material accumulation, and even personnel injuries caused by chain breakage. This significantly improves the stability and safety of scraper conveyor operation, ensuring industrial production efficiency. Attached Figure Description

[0016] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 A logic diagram of a scraper pull-off chain breakage identification system for thermal imaging and multi-dimensional quality screening provided in this application; Figure 2 The flowchart of a scraper pull-off chain breakage identification method provided in this application is based on thermal imaging and multi-dimensional quality screening. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0020] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0021] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linked," and "socketing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0022] The following explanations of some terms used in this application are provided to aid in understanding the technical solution of this application: LabelStudio is an open-source multimodal data annotation tool that supports annotation of various data types, including images, text, audio, and video, meeting the annotation needs of artificial intelligence fields such as computer vision, natural language processing, and speech recognition. In the thermal imaging and multi-dimensional quality screening scraper pull-off chain breakage identification system, this tool is mainly used for polygonal contour annotation of chains and scraper targets within thermal imaging images, providing standardized annotation data for instance segmentation model training.

[0023] The Zhang-Suen thinning algorithm is a classic binary image thinning algorithm. Its core function is to transform a binary target contour with a certain width into a centerline with a single pixel width, while completely preserving the topological structure of the original contour, including key features such as contour curvature and connectivity.

[0024] Singular Value Decomposition (SVD) is a matrix decomposition technique widely used in numerical analysis and linear algebra. Its core function is to decompose any non-square or square matrix into the product of three special matrices, thereby achieving matrix dimensionality reduction, noise reduction, and optimization of ill-conditioned problems.

[0025] like Figure 1 As shown, this application provides a scraper deflection and chain breakage identification system based on thermal imaging and multi-dimensional quality screening, including a thermal imaging data acquisition module, a multi-dimensional quality screening module, an instance segmentation model training module, a mask post-processing module, a line fitting module, a chain breakage feature detection module, and an angle calculation and comprehensive judgment module.

[0026] The thermal imaging data acquisition module is used to acquire thermal images or videos of the scraper conveyor during its operation using a thermal imaging camera, and output raw data containing the thermal characteristics of the chain and scraper.

[0027] The thermal imaging data acquisition module is configured as follows: Using a thermal imaging camera that is parallel to the plane of the chain and vibrates at the same frequency, the thermal radiation differences between the chain, the scraper and the surrounding environment, as well as the abrupt change in thermal characteristics at the chain break point, are collected during the operation of the scraper conveyor under different ambient light and material coverage conditions. Thermal imaging images or videos are generated and raw data containing the thermal characteristics of the chain and scraper are output.

[0028] The different ambient light includes natural light during the day, artificial light at night, strong light reflection, and local shadow conditions. The material coverage includes material occlusion of 0, material coverage chain within a first threshold, and material coverage chain within a second threshold. The first threshold is greater than 0 and less than the second threshold.

[0029] Specifically, the thermal imaging data acquisition module can acquire thermal images or videos of the scraper conveyor during operation through a thermal imaging camera, capture the thermal radiation characteristics of the chain and scraper, and the thermal state changes when the chain breaks, and finally output raw data containing the thermal characteristics of the chain and scraper.

[0030] The thermal imaging camera has a resolution of ≥1280×720, an image signal-to-noise ratio of ≥30dB, a lens distortion coefficient of ≤1%, and a thermal sensitivity of ≤50mK. It supports capturing the differences in thermal radiation between the chain, scraper, and the surrounding environment, as well as abrupt changes in thermal characteristics at the point of chain breakage.

[0031] The thermal imaging camera captures images covering daytime natural light, nighttime artificial light, strong light reflection, and partial shadow conditions, as well as conditions with no material, a small amount of material (covering less than 10% of the chain), and a medium amount of material (covering less than 30% of the chain). It does not capture images where the material completely obscures the chain / scraper. Specifically, it covers four scenarios under different ambient light conditions: daytime natural light, nighttime artificial light, strong light reflection, and partial shadow. The principle is that thermal imaging technology is not affected by light intensity and only captures the thermal radiation of the target. It can adapt to complex lighting changes in industrial scenarios and ensure that the chain, scraper, and surrounding environment can be distinguished under any lighting conditions.

[0032] Furthermore, different material coverage conditions: Material obstruction is 0: the chain and scraper are completely exposed with no material covering them. The material covering the chain is within a first threshold: the first threshold is 10% of the chain surface area, that is, the area of ​​the material covering the chain does not exceed 10% of the total surface area of ​​the chain; The material covering the chain is within a second threshold: the second threshold is 30% of the chain surface area, that is, the area of ​​the material covering the chain does not exceed 30% of the total chain surface area; Among them, if the first threshold (10%) is greater than 0 and less than the second threshold (30%), images of materials completely obscuring the chain / scraper will not be collected (to avoid the absence of effective thermal features to capture).

[0033] The thermal imaging camera is fixed 1.5-2m to the side of the scraper conveyor, with the lens parallel to the chain plane. The frame rate is ≥10fps, adaptable to minor vibrations of ≤5mm amplitude during normal scraper conveyor operation, ensuring no thermal features are missed during chain breakage. Furthermore, the parallel alignment of the camera lens with the chain plane ensures perspective distortion-free imaging of the chain and scraper contours, accurately reflecting their actual shape. The camera is also designed for synchronous vibration, adapting to minor vibrations of ≤5mm amplitude during normal scraper conveyor operation. This is achieved by synchronously canceling the relative displacement between the camera and the scraper conveyor, reducing image blurring caused by vibration and ensuring continuous thermal feature capture. The frame rate requirement is ≥10fps (frames / second) to ensure no abrupt changes in thermal features during chain breakage and to avoid loss of crucial information due to a low frame rate.

[0034] It should be noted that thermal imaging cameras work based on the thermal radiation characteristics of the target object. The chain, scraper and the surrounding environment have different materials and temperatures, resulting in differences in thermal radiation intensity. Thermal imaging cameras capture these differences to form thermal images. When the chain breaks, the local temperature at the break point will rise or fall sharply due to instantaneous friction or stress release, forming obvious thermal feature changes. Thermal imaging cameras can accurately capture this change signal, providing raw data support for subsequent chain break identification.

[0035] Finally, the output contains raw data containing the thermal characteristics of the chain and scraper. The data is in the form of thermal imaging images or thermal imaging videos. The raw data must retain the thermal radiation difference information between the chain, scraper and the surrounding environment, as well as the thermal characteristic mutation signal that may be caused by chain breakage. No additional preprocessing is required, and the data is directly transmitted to the multi-dimensional quality screening module.

[0036] A multi-dimensional quality screening module is connected to the thermal imaging data acquisition module. It is used to receive the raw data, preprocess the raw thermal imaging data, and output a valid image dataset that meets the preset quality requirements.

[0037] The multi-dimensional quality screening module is configured as follows: The system receives the raw data and extracts image frames from the thermal imaging image or video frames from the thermal imaging video according to the set frame rate, forming an image frame set. In the target image frame set, a first preprocessing is performed to obtain an intermediate image frame set. The first preprocessing includes calculating the dHash value of the target image frame and removing duplicates based on the dHash value, retaining clear frames. The intermediate image frame set is subjected to a second preprocessing to output a valid image dataset that meets the preset quality requirements. The second preprocessing includes the removal of sharpness, extreme frames, and thermally blurred frames.

[0038] Specifically, the parameter configuration for the multi-dimensional quality screening module is as follows: Video to image conversion: Extract thermal imaging video frames at a fixed frame rate (≥10fps). Total number of extracted images = video duration (seconds) × frame rate; Image deduplication: Calculate the dHash value of the image, similarity = 1 - hash Hamming distance / 64, if the similarity is >90%, it is judged as a duplicate, and only the clearest frame is kept; Filtering bad samples: Remove unclear frames (Laplacian variance < 50), extreme frames (mean of all-black frames < 10 / mean of all-white frames > 245), and frames with blurred thermal features (thermal contrast < 0.1). The number of effective images (preprocessing) = number of images after deduplication - number of unclear frames - number of extreme frames - number of frames with blurred thermal features. Effective image rate calculation: Effective image rate = number of effective images (preprocessed) / total number of extracted images, with a value range of 0~1, and 1 when ≥1; Quality score calculation: Quality score (out of 100) = (Scene score + Sharpness score + Completeness score + No interference score + Thermal feature score) × Effective image rate; Valid image filtering: The filtering criteria are scene conformity score ≠ 0, quality score ≥ 60 and number of valid images ≥ 5. The output is a valid image dataset.

[0039] Furthermore, the multi-dimensional quality screening module is connected to the thermal imaging data acquisition module. The multi-dimensional quality screening module receives the raw data (thermal imaging images or videos) output by the thermal imaging data acquisition module, filters invalid samples and evaluates data quality through multi-step preprocessing, and finally outputs a valid image dataset that meets the preset quality requirements, providing reliable data support for subsequent model training and real-time inference.

[0040] The multi-dimensional quality screening module is configured as follows: Image frame extraction: After receiving the raw data, first determine whether the data type is a single thermal imaging image or a continuous thermal imaging video.

[0041] If it is a single thermal imaging image, it is directly included in the image frame set; If it is a thermal imaging video, extract image frames according to the set frame rate. The total number of extracted images = video duration (seconds) × set frame rate. The set frame rate is consistent with the acquisition frame rate of the thermal imaging data acquisition module (≥10fps) to ensure that key thermal feature data is not lost at the moment of link failure.

[0042] The final result is an image frame set containing all extracted images, with each image in the set retaining the thermal feature information from the original thermal imaging data.

[0043] The first preprocessing step of the multi-dimensional quality screening module (deduplication (preserving clear frames)) is to deduplicat by calculating the dHash value (difference hash value) of the image frames. The smaller the difference in the dHash values ​​of two images, the more similar the image content is.

[0044] Specific steps: Calculate the dHash value for each image frame: First, reduce the image to a fixed size (e.g., 8×8), convert it to grayscale, and then generate a binary string, i.e., the dHash value, by comparing the grayscale differences between adjacent pixels; Calculate similarity: Calculate image similarity using the hash Hamming distance (the number of different binary bits in two dHash values). The similarity calculation formula is: Similarity = 1 - Hash Hamming distance / 64; Deduplication: When the similarity between two images is >90%, they are determined to be duplicate frames, and only the image with higher clarity is retained (judged by the average image clarity; the higher the average, the clearer the image). Output: A set of intermediate image frames after removing duplicate frames. The set contains no redundant data, and all retained images are clear frames from the same scene.

[0045] The second preprocessing step of the multi-dimensional quality screening module is to filter out bad samples, removing those from the image frame set that are of unacceptable quality and cannot be used for subsequent processing. These bad samples include three categories: unclear frames, extreme frames, and frames with blurred thermal features. The specific judgment criteria are as follows: The principle for removing blurry frames is as follows: the image sharpness mean (reflecting the sharpness of image details) is used to determine the image. The smaller the sharpness mean, the blurrier the image. The criteria are: when the Laplacian variance of image sharpness is <50, it is judged as a blurry frame and removed; images with a sharpness mean ≥100 are retained, among which those ≥200 are high-definition images.

[0046] For extreme frame removal, extreme frames refer to completely black or completely white frames. These images have no effective thermal feature information and cannot be used to identify targets. The judgment principle is to judge by calculating the average pixel value of the image (the average of the gray values ​​of all pixels). The judgment criteria are: when the average pixel value is <10, it is judged as a completely black frame, and when the average pixel value is >245, it is judged as a completely white frame. Both types of extreme frames are removed.

[0047] For the removal of frames with blurred thermal features, the judgment principle is as follows: Thermal contrast is used to determine the difference in thermal radiation between the target area (chain, scraper) and the background area. The smaller the difference, the more blurred the thermal features, and the more difficult it is to distinguish the target from the background. (The target area is the pixel area of ​​the chain and scraper in the image (which can be defined by manual annotation or a pre-trained model); the background area is the other areas in the image besides the target area). The thermal contrast formula is: Thermal contrast = (Average thermal value of the target area - Average thermal value of the background area) / Average thermal value of the background area. Here, the average thermal value of the target area is the average thermal value of all pixels in the area containing the chain and scraper; the average thermal value of the background area is the average thermal value of all pixels in the other areas of the image besides the chain and scraper. The judgment criterion is: when the thermal contrast < 0.1, it is judged as a frame with blurred thermal features and is removed; images with a thermal contrast ≥ 0.1 are retained. The final output is a set of intermediate image frames after removing the three types of bad samples. All images in the set have clear details and distinguishable thermal features.

[0048] Quality score calculation and effective image selection in the multi-dimensional quality screening module: Effective image rate calculation: To quantify the effectiveness of the preprocessed data, the effective image rate is calculated using the following formula: Effective image rate = Number of effective images (number of images retained after the second preprocessing) / Total number of extracted images (total number of image frames extracted in the first step). The effective image rate ranges from 0 to 1. When the calculation result is ≥1, it is taken as 1, which is used to fine-tune the quality score.

[0049] Quality score calculation: A multi-dimensional scoring system is used to comprehensively evaluate image quality. Quality score (out of 100) = (scene score + sharpness score + integrity score + interference-free score + thermal feature score) × effective image rate. The scoring criteria for each dimension are as follows: Scene score (30 points): Evaluates the presence of target elements (chain, scraper) in the image and their scene suitability. 30 points are awarded for an element presence rate ≥ 90% and a scene proportion ≥ 80%; 15 points are awarded for an element presence rate of 70%-90% or a scene proportion of 60%-80%; 0 points are awarded for an element presence rate < 70% or a scene proportion < 60% (a scene suitability score of 0 indicates an invalid sample). Clarity score (25 points): Score based on image resolution and average clarity. Resolution ≥ 1080P gets 10 points, 480P-720P gets 5 points, < 480P gets 0 points; Average clarity ≥ 200 gets 10 points, 100-200 (excluding endpoints) gets 5 points, ≤ 100 gets 0 points; Bitrate ≥ 2Mbps gets no deduction, < 1Mbps deducts 5 points, Total score = (Resolution score + Clarity score) - Bitrate deduction. Completeness score (15 points): Evaluates the continuity of extracted images from the video (frame breaks). 15 points for ≤3 frame breaks, 8 points for 4-6 frame breaks, and 0 points for >6 frame breaks; a frame break is defined as the time interval between two consecutive frames being greater than twice the set frame rate; No interference score (10 points): Evaluate whether the image has interference such as watermarks or occlusion. No watermark and occlusion <10% gets 10 points; slight interference (shallow watermark or occlusion 10%-30%) gets 5 points; severe interference (clear watermark or occlusion >30%) gets 0 points; Thermal characteristic score (20 points): Based on thermal contrast ratio. 20 points for thermal contrast ratio ≥ 0.3, 10 points for 0.2-0.3, 5 points for 0.1-0.2, and 0 points for < 0.1.

[0050] Valid image screening: The screening criteria are set as "scene conformity score ≠ 0 + quality score ≥ 60 + number of valid images (number of images retained after the second preprocessing) ≥ 5". Images that meet the criteria are included in the valid image dataset; images that do not meet any of the criteria are determined to be invalid samples and are not included.

[0051] The final output is a valid image dataset that meets the preset quality requirements. Each valid image in the dataset satisfies the following criteria: no duplication, no blurring, no extreme interference, clear and distinguishable thermal features, and good scene adaptability. It can be directly used for subsequent operations in the data annotation and preprocessing modules.

[0052] In one embodiment, a scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening further includes a data annotation and preprocessing module, which is connected to the multi-dimensional quality screening module and the instance segmentation model training module, respectively. The data annotation and preprocessing module is used to receive the effective image dataset, perform polygon contour annotation on the left and right chains of the chain and the scraper in each effective image in the effective image dataset, and output the labeled effective image dataset to the instance segmentation model training module. The labeled effective image dataset includes a training set, a validation set and a test set.

[0053] Specifically, the data annotation and preprocessing module receives the valid image dataset output by the multi-dimensional quality screening module, and generates a labeled valid image dataset that meets the training requirements of the instance segmentation model through standardized annotation and data preprocessing. Finally, it outputs standardized data containing training set, validation set and test set, providing high-quality and unambiguous labeled samples to support model training.

[0054] The data annotation and preprocessing module can use the LabelStudio annotation tool. The annotation objects include three types of targets: the left chain (corresponding to the drive chain on the left side of the scraper conveyor), the right chain (corresponding to the drive chain on the right side of the scraper conveyor), and the scraper (the component that connects the two chains and is used to transport materials). When annotating, irrelevant areas such as the frame, materials, and surrounding environment should be excluded, and only the target itself should be focused on.

[0055] Annotation Standards and Principles: Use the polygon annotation function in the tool to outline the target contour, with a vertex spacing of ≤5 pixels (to ensure the contour fits the actual edge of the target and reduce shape deviation); Special area handling: Densify vertices at chain link bends (the contour curvature is large at bends, and densifying vertices can avoid contour distortion); Leave a 1-2 pixel gap in areas where the scraper and chain are closely connected (to prevent the masks of the two types of targets from sticking together during subsequent model inference, affecting segmentation accuracy); Accuracy requirements: The deviation between the annotated contour and the actual edge of the target is ≤2 pixels (to ensure the accuracy of the annotated data and provide reliable supervision signals for model training); Completely cover the target contour, with no omissions (such as continuous chain links, the main body of the scraper, and the ears connected to the chain must all be annotated), and no extra annotations (areas outside the target are not annotated separately).

[0056] For the screening of unqualified samples, manual review can be used, or a conventional script can be constructed for screening. This application does not limit the script. Unqualified samples include those with contour deviations exceeding 2 pixels, missing target categories, multiple irrelevant areas, excessive vertex spacing leading to contour distortion, and masks of two types of targets sticking together. Such samples need to be re-labeled or removed.

[0057] Data integrity check: Ensure that the annotation file (JSON format) corresponding to each annotation image contains the contour coordinate information of three types of targets, and that the number of contour vertex coordinates of each type of target is ≥10 (chain, because the contour of the chain is long and curved) or ≥6 (scraper, because the contour of the scraper is relatively regular), to avoid the inability to accurately restore the contour due to insufficient number of vertices.

[0058] Data is categorized and stored according to the naming rule of "collection date_operating condition type" (e.g., "20240520_no material operating condition", "20240521_medium material operating condition"), which facilitates the balance of samples for each operating condition when dividing the dataset later.

[0059] Annotation format conversion: The raw annotation data generated by the annotation tool is in JSON format. It needs to be converted to the "image + txt annotation" format supported by the instance segmentation model (YOLOv8-Seg) to ensure that the model can read the annotation information and perform training. Category ID mapping: Establish a fixed category ID mapping relationship, with ID=0 for the left chain, ID=1 for the right chain, and ID=2 for the scraper. The mapping relationship is consistent throughout.

[0060] Coordinate transformation converts absolute pixel coordinates (x, y) in JSON format to relative coordinates. The conversion formula is: Relative x = Absolute x / Image width; Relative y = Absolute y / Image height; Formula meaning: Absolute x, Absolute y: The actual pixel coordinates of the labeled contour vertices in the image (e.g., the x-axis coordinate of a vertex in the image is 500 pixels and the y-axis coordinate is 300 pixels); Image width, Image height: The effective pixel size of the image (e.g., 1280×720, i.e., width = 1280 pixels and height = 720 pixels); Relative coordinates: The value range is 0~1, which can avoid coordinate failure caused by image size scaling and adapt to the input requirements of the model.

[0061] Post-conversion file storage: Each valid image corresponds to a TXT annotation file, with the TXT files organized by "category ID relative" relatively relatively relatively …relatively relatively The annotation information is stored in the format of "" and the file name of the txt file is consistent with the file name of the corresponding image, which facilitates the model association and reading.

[0062] After processing the valid image dataset, the dataset is then partitioned. Specifically, stratified sampling can be used to divide the labeled valid image dataset into training, validation, and test sets at a ratio of 80%, 10%, and 10%, respectively. The principle of stratified sampling is to ensure that the proportion of samples from each collection condition (such as no material, small amount of material, medium amount of material, natural light during the day, artificial light at night, etc.) is consistent across the three subsets, avoiding the concentration of samples from a certain condition in a single subset, which could lead to insufficient generalization ability of the model.

[0063] Further, the subset functions are explained as follows: Training set: used for parameter learning of the model, providing the main supervision signals for the model; Validation set: used for performance evaluation during model training, monitoring whether the model is overfitting, and assisting in adjusting training parameters; Test set: used for final performance testing after model training is completed, objectively reflecting the model's recognition accuracy on unseen samples.

[0064] Finally, data augmentation is required. Data augmentation operations are only performed on the training set samples, while the validation and test sets remain unchanged. The purpose is to diversify the training set samples, expand the distribution range of the training data, improve the model's adaptability to complex working conditions (such as changes in lighting, slight material occlusion, and slight deformation of the target), and reduce overfitting.

[0065] Methods and principles of training set sample augmentation: Brightness adjustment: Adjust image brightness within ±30% range (simulating thermal imaging effects under different light intensities); Contrast adjustment: Adjust image contrast within ±20% (enhance or reduce the difference in thermal features between the target and the background). Gaussian blur: A light blurring process is applied (blurring coefficient = 0.5, simulating the blurring of thermal imaging images caused by slight vibrations); Slight rotation: Rotate the image within ±3° (simulating a small deviation in the installation angle of the acquisition device); Material occlusion simulation: Randomly add a small amount of virtual material occlusion to the image (occlusion ratio ≤10%, simulating the scenario where a small amount of material covers the target in actual working conditions). Thermal feature perturbation: Fine-tuning the thermal data of the target area within ±5% (simulating the small measurement error of a thermal imaging camera).

[0066] Enhancement method disabled: Do not perform horizontal or vertical flipping (the left and right chains of the scraper conveyor have a fixed left-right position relationship; flipping them will cause confusion in the target position and affect the model's recognition logic).

[0067] The final output is a labeled, valid image dataset, which includes three subsets: training, validation, and test. Each subset consists of "valid images + corresponding TXT annotation files". The annotation file format is adapted to the YOLOv8-Seg model. The data-augmented training set samples cover various working conditions, and the sample distribution of the three subsets is balanced. It can be directly input into the instance segmentation model training module for model training.

[0068] The instance segmentation model training module is connected to the multi-dimensional quality screening module. It is used to construct the scraper and chain recognition model, and to train it using an effective image dataset, and output the trained instance segmentation model.

[0069] Specifically, the instance segmentation model training module is configured as follows: Based on the YOLOv8-Seg.pt pre-trained model, a scraper and chain recognition model is constructed, and parameters are configured for the scraper and chain recognition model to obtain the scraper and chain recognition model; Using the labeled valid image dataset, the scraper and chain recognition model is trained, and the convergence of the training loss function and the intersection-union ratio index is monitored. When the convergence reaches the preset convergence value, the trained instance segmentation model is output.

[0070] Specifically, the instance segmentation model training module is connected to the data labeling and preprocessing module and the mask postprocessing module. The instance segmentation model training module builds a scraper and chain recognition model based on the pre-trained model. It uses the labeled effective image dataset for model training, indicator monitoring and convergence determination. Finally, it outputs an instance segmentation model that can accurately extract the pixel-level contours of the chain (left chain and right chain) and the scraper, providing a reliable inference basis for the subsequent mask postprocessing module.

[0071] The pre-trained model uses the YOLOv8-Seg.pt pre-trained model as the basic framework. This model is lightweight, has fast inference speed, and high pixel-level segmentation accuracy. It has the ability to extract features of general targets. Fine-tuning based on this model can significantly reduce the amount of training data and training time, and adapt to the deployment needs of industrial edge devices.

[0072] Configure the files for the YOLOv8-Seg.pt pre-trained model: Configuration file settings: A custom configuration file (named scraper_chain_seg.yaml) is used, and its core content includes: Dataset path: Specifies the storage path for images and annotation files in the training, validation, and test sets to ensure the model can accurately read the data; Category Configuration: The number of categories is explicitly defined as 3, and the corresponding category name and ID mapping relationship is as follows: The mapping relationship between the left chain (left_chain=0), the right chain (right_chain=1), and the scraper (scraper=2) is consistent with the data labeling and preprocessing module to avoid category confusion. Training parameter configuration for the YOLOv8-Seg.pt pre-trained model: The training parameters are set using the Ultralytics library, with 100 training epochs, a batch size of 16, an input image size of 640×640, an initial learning rate of 0.01, a final learning rate of 0.01, a momentum of 0.937, a weight decay of 0.0005, an IoU threshold of 0.7, and data augmentation and mixup (mixup=0.1) enabled. Training and monitoring of the YOLOv8-Seg.pt pre-trained model: The model is trained by dividing the dataset into 80% training set, 10% validation set, and 10% test set. For example, TensorBoard is used to monitor metrics such as training loss (train / loss_seg ≤ 0.1), validation loss, intersection-over-union ratio (val / seg_iou ≥ 0.85), and detection accuracy (val / precision ≥ 0.9, val / recall ≥ 0.9). The model is considered converged when val / seg_iou remains consistently above 0.85 for 5 consecutive rounds and val / loss_seg shows no decrease for 3 consecutive rounds. The intersection-over-union ratio is calculated based on pixel-level mask overlap regions and satisfies the formula: seg_iou = number of pixels in the overlapping region / (number of pixels in the predicted region + number of pixels in the true region - number of pixels in the overlapping region). Evaluate model performance using the test set. If seg_iou, precision, or recall is not up to standard, optimize by increasing the number of samples, improving annotation accuracy, or expanding the data augmentation range. After training, export the model in ONNX format.

[0073] The mask post-processing module, connected to the multi-dimensional quality screening module, is used to infer the real-time thermal imaging image acquired by the thermal imaging data acquisition module using the trained instance segmentation model, extract the binary mask of the chain and the scraper, remove noise and holes through morphological operations, and output the target mask and mask integrity.

[0074] Specifically, the mask post-processing module is configured as follows: Using n×n rectangular structuring elements, morphological closing operations are performed to fill in the holes in the binary mask of the chain and the scraper, and the holes in the binary mask of the chain and the scraper are extracted by inference on the real-time thermal imaging image acquired by the thermal imaging data acquisition module using the trained instance segmentation model, thus obtaining the continuous outline of the chain and the scraper. Morphological opening operations are performed using m×m rectangular structuring elements to remove noise from the continuous contours of the chain and the scraper, resulting in a target mask. The target mask represents a binarized image containing the target regions of the chain and the scraper. The contour area of ​​the target mask is calculated using the polygon area formula to obtain the maximum contour area. The polygon area formula is as follows: For the coordinates of the contour vertices ,area ; The mask integrity is obtained based on the maximum contour area, wherein when the maximum contour area is greater than or equal to a preset contour area value, the mask integrity is considered complete; otherwise, the mask integrity is considered incomplete.

[0075] Specifically, the mask post-processing module is connected to the instance segmentation model training module and the line fitting module, respectively. To load the trained instance segmentation model, it performs inference on the real-time thermal imaging image input from the thermal imaging data acquisition module to extract the binary mask of the chain and scraper. Through morphological operations, it eliminates holes and noise in the mask to obtain a clean target mask. The integrity of the mask is determined by combining the contour area calculation results, and finally the target mask and mask integrity are output. This provides high-quality mask data support for the centerline extraction of the subsequent line fitting module and the fault identification of the chain break feature detection module.

[0076] It should be noted that real-time inference extracts the binary mask, loads the trained instance segmentation model output from the instance segmentation model training module, and ensures that the model parameters are adapted to the inference environment to meet the frame rate requirements (≥10fps) for real-time detection. Real-time thermal imaging images acquired by the thermal imaging data acquisition module are input into the model, and a confidence threshold (e.g., 0.5) is set to filter low-confidence prediction results, retaining only the predicted contours of the target areas for the chain and scraper. The pixel values ​​of the predicted target areas are set to 255 (representing the foreground), and the pixel values ​​of the background areas are set to 0 (representing the background), generating a binary mask for the chain and scraper. This binary mask can accurately distinguish between the target and the background, but it may contain small holes and isolated noise points due to material occlusion and image noise.

[0077] Furthermore, the receiving element and parameter settings can use a 3×3 rectangular structure element (i.e., n=3 in the configuration). This size adapts to the contour features of the chain and scraper, effectively filling small holes without destroying the original shape of the target contour. A "dilation-erosion" operation logic is adopted. The dilation operation slightly expands the target contour, filling small holes inside the contour and connecting broken local contours; the erosion operation shrinks the expanded contour back to its original size, avoiding excessive contour expansion. Next, after performing a closing operation on the binary mask, the small holes (hole area < 10 pixels) within the chain and scraper contours are filled, resulting in continuous, unbroken chain and scraper contours, eliminating incomplete contours caused by image noise or slight material occlusion.

[0078] Morphological opening operation: Removes mask noise. Structural element and parameter settings: A 2×2 rectangular structuring element (m=2 in the configuration) is used. This size accurately identifies and removes isolated small noise points in the mask without damaging the main outline of the chain and scraper. Opening operation principle and function: The opening operation uses an "erosion-dilation" logic. The erosion operation eliminates isolated noise points of 1-2 pixels in the mask; the dilation operation restores the eroded target outline to its original size and shape, ensuring the integrity of the target outline. Target mask generation: The opening operation is performed on the outline after the closing operation to remove isolated noise points around the outline, resulting in a clean target mask. The target mask is a binary image containing the target areas of the chain and scraper. The target area outline is continuous, noise-free, and without obvious holes, and can be directly used for subsequent outline analysis.

[0079] Contour extraction and maximum contour area calculation: Target contour extraction: Identify all connected target contours in the target mask. Each contour corresponds to the complete region of a chain or scraper. Minimal residual noise contours with an area < 5 pixels are excluded to ensure that only the contour area of ​​valid targets is calculated. Polygon area formula calculation: For each extracted target contour, the contour area is calculated using the polygon area formula. S represents the area of ​​a single target contour; and This represents the pixel coordinates of the i-th contour vertex, where i ranges from 1 to n, and n is the total number of contour vertices. and The outline is a closed polygon, and the last vertex must coincide with the first vertex. The maximum outline area is determined by comparing the areas of all valid target outlines and selecting the area with the largest value as the maximum outline area. This area corresponds to the main area of ​​the chain or scraper and can reflect the integrity of the target mask.

[0080] Mask integrity determination: Supplementary Criteria for Judgment: To improve the accuracy of the judgment, the maximum outline area ratio is used as the core judgment indicator, rather than a single absolute area threshold. The formula for calculating the maximum outline area ratio is: Maximum outline area ratio = Maximum outline area / Total non-zero pixel area of ​​the mask; The total non-zero pixel area of ​​the mask is the total number of pixels with a value of 255 in the target mask, representing the total area of ​​the predicted chain and scraper target regions.

[0081] Integrity judgment criteria: When the maximum contour area accounts for ≥80%, the integrity of the characterization mask is complete, indicating that there is no large area missing in the target contour, which can support subsequent line fitting and chain break detection. When the maximum outline area ratio is less than 80%, the integrity of the characterization mask is incomplete. This is most likely due to the large area of ​​material obscuring the target outline or equipment failure, which requires manual review.

[0082] Preset contour area value reference: The preset contour area value can be determined based on the complete mask of the chain and scraper under normal working conditions. The maximum contour area under normal working conditions can be used as a benchmark value to help verify the rationality of the maximum contour area ratio.

[0083] The final output consists of two core results: the target mask and the mask integrity. The target mask is a clean, hole-free, and noise-free binary image that can be directly used for single-pixel centerline extraction in the line fitting module. The mask integrity result (complete / incomplete) is transmitted to the angle calculation and comprehensive judgment module as an important reference for determining the level of deviation and chain breakage.

[0084] The straight line fitting module, connected to the mask post-processing module, is used to extract the skeleton and filter noise from the target mask, obtain the coordinate set of the center line of a single pixel in the target mask, and output the straight line equation of the chain and the scraper based on the solution of the straight line equation and the distance formula by using the fusion method of initial fitting by least squares method and Hough transform correction.

[0085] Specifically, the line fitting module is configured as follows: Using the Zhang-Suen thinning algorithm, the target mask is transformed into a single-pixel centerline while preserving the contour topology of the target mask; The isolated line segments of the single-pixel centerline are filtered out, and the contour arc lengths that are greater than or equal to a preset pixel length are retained to obtain the target single-pixel centerline. The formula for calculating the contour arc length is as follows: For consecutive vertices and ; Arc length L = ; in, This represents the x-axis coordinate of pixel i within the center line of a single pixel. The y-axis coordinate of pixel i within the center line of a single pixel is represented, and the arc length L represents the total length of the outline of the center line of a single pixel. This indicates that the distance values ​​between all adjacent pixels along the center line are summed. Based on the set of centerline pixel coordinates corresponding to the centerline of the target single pixel, the least squares method is used to fit the straight line equation to obtain the initial fitted straight line equation. The Hough transform is used to detect straight lines with a length greater than or equal to 50 pixels in the center line of the target single pixel, the angle of the main direction is determined, and the deviation between the angle of the main direction and the angle of the straight line formed by the initial fitted straight line equation is calculated to obtain the calculation result. If the deviation of the calculation result is greater than the preset angle deviation threshold, then after removing the abnormal points of pixels whose distance from the main direction is greater than the preset angle deviation threshold, the straight line equation of the chain and the scraper is refitted and output. If the deviation of the calculation result is less than or equal to the preset angle deviation threshold, the initial fitted straight line equation is directly used to output the straight line equation of the chain and the scraper.

[0086] Specifically, the straight line fitting module is connected to the mask post-processing module, the chain break feature detection module, and the angle calculation and comprehensive judgment module. The purpose of the straight line fitting module is to receive the target mask output by the mask post-processing module, obtain the accurate single-pixel centerline coordinate set through skeleton extraction and noise filtering, and solve and optimize the straight line equation by adopting the fusion scheme of "least square initial fitting + Hough transform correction". Finally, the accurate straight line equation of the chain and scraper is output, providing core data support for chain break gap detection and pull angle calculation.

[0087] Furthermore, the steps of the line fitting module are as follows: Single-pixel centerline extraction: Preserving the contour topology, the Zhang-Suen thinning algorithm is used to process the target mask. This method works by peeling away the edge pixels of the target contour layer by layer, while preserving the endpoints and branch points of the contour, ultimately transforming the wide contour mask into a single-pixel centerline. During processing, the topology of the target mask must be preserved; that is, the curvature of the chain and the straight shape of the scraper must be completely consistent with the original contour to ensure that the centerline accurately reflects the actual position and shape of the chain and scraper, without distortion, breakpoints, or extra branches. This yields a single-pixel centerline corresponding one-to-one with the target mask contour. This centerline consists of continuous pixels, each with clearly defined x-axis and y-axis coordinates.

[0088] Noise filtering: Retain the effective contour arc length. There may be isolated short line segments in the center line of a single pixel caused by mask noise or minor material occlusion. These line segments will interfere with the accuracy of subsequent straight line fitting and need to be removed by contour arc length screening.

[0089] Arc length calculation principle: For consecutive pixel vertices in the center line of a single pixel, treat two adjacent pixel points as the endpoints of a line segment, and accumulate the lengths of all adjacent line segments to obtain the arc length of the entire center line.

[0090] The formula for arc length is: Arc length L = ; Where L represents the total length of the centerline outline of a single pixel; and This represents the x-axis and y-axis coordinates of the i-th pixel in the center line of a single pixel. and represents the coordinates of the next adjacent pixel of the i-th pixel, and n represents the total number of pixels along the center line of a single pixel.

[0091] Filtering criteria: Set the preset pixel length threshold to ≥5 pixels, retain only centerline segments with an arc length ≥ this threshold, and remove isolated short segments with an arc length <5 pixels to obtain the target single-pixel centerline without noise interference.

[0092] Coordinate set extraction: Traverse all pixels of the target single-pixel centerline, extract the coordinate information of each point, and form an ordered centerline pixel coordinate set to provide a data foundation for subsequent line fitting.

[0093] Initial fitting using the least squares method: Solve the equation of the line. The core logic of the least squares method is to minimize the sum of squared errors of all centerline pixels to the fitted line, ensuring that the fitted line can approximate the distribution trend of all pixels as closely as possible.

[0094] Setting the equation of the line: Set the general equation of the fitted line as follows , where a and b represent the coefficients of the line equation, and are not both 0; c represents the constant term of the line equation; and x and y represent the coordinates of the center line pixel.

[0095] The fitting operation and optimization involve substituting the centerline pixel coordinate set into the error function and calculating the coefficients a, b, and c that minimize the sum of squared errors to obtain the initial fitted line equation. To avoid fitting bias caused by ill-conditioned matrices during the solution process, the SVD decomposition method can be used to optimize the calculation process. After the solution is completed, the coefficients are normalized (to make...). This ensures the uniformity and stability of the equation parameters. Initial fitted straight line equations for the chain and scraper are obtained, which preliminarily reflect the overall direction of the target.

[0096] Hough Transform Correction: Optimizing the accuracy of straight line angles. The core principle of the Hough Transform is to map pixels in image space to parameter space, and detect the most prominent straight line in the image by statistically analyzing the accumulated peak values ​​in the parameter space. This step is used to detect the main direction straight line in the center line of a single target pixel, correcting the angle deviation of the initial fitted equation.

[0097] For main direction line detection, the target's single-pixel centerline is converted into a binary image, and then Hough transform is applied for line detection. A detection threshold of ≥50 pixels in length is set to ensure the detection result represents the target's main direction line (excluding interference from minor local bends). The angle of the main direction line is extracted; this angle represents the true direction of the chain or scraper.

[0098] Angle deviation determination and outlier removal: The difference between the angle of the initial fitted straight line equation and the principal direction angle detected by the Hough transform is calculated to obtain the angle deviation value.

[0099] Set the preset angle deviation threshold to >2°: If the deviation value is ≤2°, it means that the accuracy of the initial fitted line meets the requirements and no correction is needed; if the deviation value is >2°, abnormal pixels that deviate from the main direction need to be removed.

[0100] Abnormal point removal criteria: Using the distance formula from a point to a line, calculate the distance from all centerline pixels to the main direction line, remove abnormal points whose distance is greater than the preset pixel threshold (2 pixels), and retain valid pixels that fit the main direction.

[0101] Distance formula and the meaning of letters: ; Where d represents the perpendicular distance from the pixel to the main direction line, and a, b, and c represent the coefficients and constant terms of the equation of the main direction line. and This represents the coordinates of the centerline pixel.

[0102] After removing outliers, the least squares method is used again to fit the remaining valid pixels, resulting in a corrected final straight line equation. This equation accurately reflects the actual position and direction of the chain and scraper, and is directly output to the chain breakage feature detection module and the angle calculation and comprehensive judgment module.

[0103] The chain breakage feature detection module, connected to the straight line fitting module, is used to output the chain breakage determination result based on the target mask and the single pixel centerline coordinate set, through chain link continuity analysis, break gap detection and thermal feature mutation identification.

[0104] Specifically, the chain break feature detection module is configured as follows: Based on the target mask of the chain, the number of connected components in the chain outline is calculated. When the number of connected components is ≥2 and the distance between adjacent connected components is greater than a preset distance threshold, it is determined to be a suspected chain break. The distance between adjacent connected components represents the center-to-center distance between the minimum bounding rectangles of two connected components, and the formula is: ; Where d represents the distance between the centers of the minimum bounding rectangles of the two connected components. and Indicates the coordinates of the center of the first connected domain. and Indicates the coordinates of the center of the second connected domain; The distance between adjacent pixels is calculated based on the coordinate set of the single pixel centerline. When the distance is greater than a preset gap threshold and there is no material obstruction in the corresponding area, the break gap between adjacent pixels is obtained. The formula for the distance between adjacent pixels is: ; in, Indicates the distance between adjacent pixels. and This represents the coordinates of the i-th pixel. and Represented as the coordinates of the j-th pixel; Based on the calorific value data of the fracture gap and the chain links on both sides, the calorific value difference rate of the chain links on both sides is calculated and compared with a preset reference temperature range. When the calorific value difference rate is greater than a preset difference threshold, or when the temperature of the fracture gap exceeds the reference temperature range and reaches a preset temperature threshold, it is determined to be a thermal characteristic abrupt change. The formula for calculating the calorific value difference rate is as follows: ; in, Indicates the difference rate of calorific value. This represents the average heat value of the fracture gap region. This indicates the average heat value of the normal area of ​​the chain; The chain breakage determination result is obtained by comprehensively judging the suspected chain breakage, the fracture gap, and the thermal characteristic mutation. If the chain breakage determination result satisfies the suspected chain breakage result, the existence of at least one fracture gap, and the thermal characteristic abrupt change, a first output chain breakage determination result is obtained, wherein the first output chain breakage determination result represents either a broken chain or a non-broken chain. If the chain breakage determination result satisfies the suspected chain breakage result and the existence of at least one breakage gap, then a supplementary determination is made in combination with the scraper running speed to obtain a second output chain breakage determination result. The second output chain breakage determination result indicates that when the scraper running speed is ≥ the preset speed, it is determined to be suspected breakage, and when the scraper running speed is < the preset speed, it is determined to be under observation.

[0105] It should be noted that the chain break feature detection module is connected to the mask post-processing module and the line fitting module respectively. The purpose of the chain break feature detection module is to receive the target mask and the coordinate set of the center line of a single pixel. Through three core links, chain link continuity analysis, fracture gap detection and thermal feature mutation identification, it realizes the quantitative judgment of chain breakage faults and finally outputs the chain breakage judgment result, providing fault data support for the angle calculation and comprehensive judgment module.

[0106] Specifically, link continuity analysis: Determining suspected link breakage: The principle of connected component extraction is as follows: Traverse the target mask of the chain and identify all independent connected regions (i.e., connected components) in the mask. Each connected component corresponds to a continuous chain outline. For a normal chain without breaks, the number of connected components in the target mask is 1; if the chain is broken, the break will divide the chain into multiple segments, and the number of connected components will be ≥2.

[0107] Calculation of distance between adjacent connected components: For every two adjacent connected components, calculate the center coordinates of their minimum bounding rectangles, and then calculate the Euclidean distance between the two centers using the distance formula: ; Criteria for suspected chain breakage: A preset spacing threshold of 5 pixels is set. When the number of connected components is ≥2 and the spacing between adjacent connected components is >5 pixels, it is judged as a suspected chain breakage. This threshold can exclude interference from minor gaps between normal chain segments, ensuring the accuracy of the judgment.

[0108] Fracture gap detection: Marking gap location: Adjacent pixel spacing calculation: Extract the pixel coordinate set of the center line of a single pixel, traverse adjacent pixels in sequence, and calculate the straight-line distance between the two points using the spacing formula. The calculation formula is as follows: ; Material occlusion judgment criteria: Query the target mask pixel value corresponding to the location where the spacing exceeds the standard. If the mask value is 0, it means that there is no chain target in the area and no material covering it. If the mask value is not 0, it means that there is material occlusion in the area, which is excluded (to avoid misjudging material occlusion as a break gap).

[0109] Break gap marking rules: A preset gap threshold of 10 pixels can be set. When the distance between adjacent pixels is greater than 10 pixels and there is no material obstruction in the corresponding area, the position is marked as a break gap, and the start coordinates, end coordinates and gap length of the gap are recorded.

[0110] Thermal signature mutation identification: Verifying the authenticity of the fault. Calorific value data extraction range: Accurately extract calorific value data of the fracture gap area and the chain links within a 50-pixel range on both sides of the gap, ensuring a high degree of matching between the extracted area and the fault location; the calorific value data comes directly from the thermal imaging data acquisition module, with a temperature acquisition accuracy of ≤0.1℃, ensuring data accuracy.

[0111] Reference parameter settings: Preset reference temperature range: Based on the average chain temperature during normal operation of the scraper conveyor, for example, the average chain temperature ±5℃. If it exceeds this range, it is judged as a temperature abnormality; Preset difference threshold: The difference rate of heat value between the two chain links, for example, >20%, or the difference rate of heat value between the broken gap area and the normal area of ​​the chain, for example, >30%.

[0112] The rate of difference in calorific value between the two types of regions is calculated using the following formula: ; The formula letters and two calculation scenarios are explained below: Scenario 1 (Comparison of links on both sides): This indicates the average heat value of the chain segment on one side of the fracture gap. This indicates the average heat value of the chain link on the other side of the fracture gap; Scenario 2 (Comparison between gap and normal area): This represents the average heat value of the fracture gap region. This indicates the average heat value of the normal area of ​​the chain.

[0113] For example, the criteria for determining a sudden change in thermal characteristics are as follows: a sudden change in thermal characteristics can be determined if any of the following conditions are met (chain breakage will cause friction or stress release, resulting in a sudden increase / decrease in local temperature): The difference in calorific value between the two chain segments is greater than 20%; The temperature in the fracture gap region exceeds the reference temperature range by ≥8℃; The difference in calorific value between the fracture gap region and the normal region of the chain is >30%.

[0114] For example, based on comprehensive judgment: output the chain break result: First output chain breakage determination result: When the three conditions of suspected chain breakage, existence of at least one fracture gap, and abrupt change in thermal characteristics are met simultaneously, the first determination result is output. This result clearly indicates whether the chain is broken or not, and is the core determination conclusion of chain breakage.

[0115] Second output chain breakage determination result: When only the two conditions of suspected chain breakage and existence of at least one fracture gap are met, but no sudden change in thermal characteristics is detected, supplementary determination is required in conjunction with the scraper conveyor's operating speed: Set the preset speed threshold to 0.5 m / s; When the scraper running speed is ≥0.5m / s, it is judged as a suspected fracture; When the scraper running speed is <0.5m / s, it is determined to be under observation (when running at low speed, the chain gap may be caused by slack, and further monitoring and confirmation are required).

[0116] The final output includes the chain breakage determination result (first output / second output), details of the fracture gap (location, length, coordinates), and abnormal thermal characteristic data (abnormal temperature value, calorific value difference rate). All data are synchronously transmitted to the angle calculation and comprehensive determination module for graded early warning of chain deviation and chain breakage.

[0117] The angle calculation and comprehensive judgment module is connected to the straight line fitting module and the chain break feature detection module, respectively. It is used to calculate the parallelism angle of the chain on both sides of the scraper, the perpendicularity angle between the chain and the scraper, and the angle between the chain and the frame baseline based on the formula of the normal vector angle of the straight line equation. It also combines the mask integrity and the chain break judgment result to perform pull deviation and chain break classification judgment, and output the warning level and judgment result.

[0118] Specifically, the angle calculation and comprehensive judgment module is configured as follows: Based on the formula for the included angle of the normal vector according to the equation of the straight line, the parallelism angle of the chain on both sides of the scraper, the perpendicularity angle of the chain to the scraper, and the angle between the chain and the frame reference line are calculated. The formula for calculating the included angle of the normal vector is as follows: ; in, , Represents the normal vector of two lines ， Represents the dot product. and These represent the magnitudes of the two normal vectors, i.e., the lengths of the vectors. , Represent the equation of the first straight line The coefficients of the x and y terms in the equation. , Indicate the equation of the second line The coefficients of the x and y terms in the equation, where x represents the x-axis coordinate of the pixel, y represents the y-axis coordinate of the pixel, and c represents the constant term of the equation of the line; The chain parallelism angle Based on the equations of the left and right chain lines, the angle between the normal vectors is calculated using the formula: Equation of the straight line of the left chain The calculation formula is ; Equation of the straight line of the right chain The calculation formula is ; Calculated using the formula for the included angle of the normal vectors; in, Indicates the straight line of the left chain. The constant term, Indicates the straight line of the chain on the right. The constant term, , They represent lines respectively. and The normal vector; The perpendicularity angle between the chain and the scraper Based on the equations of the chain and the scraper, the angle between the normal vectors is calculated using the formula: Chain straight line equation The calculation formula is ; The formula for calculating the straight line of the scraper is: ; Calculated using the formula for the included angle of the normal vectors; in, and Represents a straight chain The coefficients of the x and y terms in the equation. Represents a straight chain The constant term, and Indicates the straight line of the scraper The coefficients of the x and y terms in the equation. Indicates the straight line of the scraper The constant term, , They represent lines respectively. and The normal vector; The angle between the chain and the frame baseline Based on the equations of the chain straight line and the frame baseline straight line, the angle between the normal vectors is calculated using the formula: Chain straight line equation The calculation formula is ; Rack baseline The calculation formula is ; Calculated using the formula for the included angle of the normal vectors; in, and Indicates the rack baseline The coefficients of the x and y terms in the equation. Indicates the rack baseline The constant term, , They represent lines respectively. and The normal vector; The integrity of the mask is obtained, and the chain breakage determination result is combined to perform a graded determination of pull deviation and chain breakage, resulting in an output warning level and determination result. The chain breakage graded determination includes at least a first-level warning, a second-level warning, and normal. A first-level warning indicates that the chain breakage determination result is a chain breakage, or the parallelism angle between the two sides of the chain is greater than a first angle and the angle between the chain and the frame baseline is greater than a second angle. A second-level warning indicates that the second angle < the parallelism angle between the two sides of the chain ≤ the first angle, or (the perpendicularity angle between the chain and the scraper is less than a third angle, or the perpendicularity angle between the chain and the scraper is greater than a fourth angle, for example...). <88° or >92°), or suspected chain breakage but not meeting the thermal characteristic mutation condition, or incomplete mask integrity; the normal condition means that the parallelism angle of the two sides of the chain is ≤ the second angle, the third angle is ≤ the perpendicularity angle between the chain and the scraper is ≤ the fourth angle, the angle between the chain and the frame baseline is ≤ the fifth angle, and the chain breakage determination result is that the chain is not broken.

[0119] It should be noted that the angle calculation and comprehensive judgment module is connected to the straight line fitting module, the chain break feature detection module, and the mask post-processing module, respectively. The purpose of the angle calculation and comprehensive judgment module is to receive the straight line equations of the chain and scraper output by the straight line fitting module, calculate three types of core angles based on the formula of the angle between the normal vectors; obtain the mask integrity result output by the mask post-processing module, and combine it with the chain break judgment result output by the chain break feature detection module to perform graded judgment of pull deviation and chain breakage, and finally output the warning level and judgment result, providing a clear basis for handling scraper conveyor fault monitoring.

[0120] Specifically, the principle behind the formula for the angle between normal vectors is that a normal vector is a vector perpendicular to a line, and the angle between two lines can be represented by the angle between their normal vectors. The formula works by calculating the ratio of the dot product of the two normal vectors to their magnitudes, obtaining the cosine of the angle between the vectors, then converting it to an angle using the inverse cosine function, and finally completing the conversion from radians to degrees, ensuring the result is in commonly used angle units.

[0121] Calculation of three core perspectives: Based on the formula for the angle between the normal vectors, and combined with the parameters of different straight line equations, three types of core angles are calculated, each corresponding to a fault characteristic of the scraper conveyor: Parallelism angle of the two chains The angle reflects the parallelism of the two chains. The closer the angle value is to 0°, the more parallel the two chains are; the larger the angle value, the more serious the pull-off fault.

[0122] Perpendicularity angle between chain and scraper The angle reflects the perpendicularity between the chain and the scraper. Under normal working conditions, the angle should be close to 90°. The greater the deviation of the angle from 90°, the worse the perpendicularity between the scraper and the chain, which is prone to wear and jamming.

[0123] The angle between the chain and the frame baseline The angle reflects the degree of chain offset relative to the frame. The closer the angle value is to 0°, the more standard the chain running direction is; the larger the angle value, the more obvious the unilateral pull fault is.

[0124] For mask integrity acquisition, the mask integrity determination result can be obtained from the mask post-processing module. This result is calculated based on the contour area of ​​the target mask.

[0125] Supplement to the criteria for determining mask integrity: Mask integrity is determined by the percentage of the largest outline area, calculated using the following formula: Maximum outline area ratio = Maximum outline area / Total non-zero pixel area of ​​the mask; Maximum contour area: refers to the area of ​​the largest connected region in the target mask, representing the main area of ​​the chain or scraper; Total non-zero pixel area of ​​the mask: refers to the total number of pixels with a pixel value of 255 in the target mask, representing the area of ​​all target regions predicted by the model; When the maximum outline area accounts for ≥80%, the mask is considered complete; when the proportion is <80%, the mask is considered incomplete, indicating that there may be a large area of ​​material occlusion or incomplete target recognition.

[0126] The comprehensive classification and judgment of pull-off and chain breakage is based on three core angle parameters, mask integrity, and chain breakage judgment results. A classification judgment is then performed according to preset thresholds, and the judgment criteria are clearly defined as follows: For example, the threshold parameter is defined as follows: First angle: 3°; Second angle: 2°; Third angle: 88°; Fourth angle: 92°; Fifth angle: 1°.

[0127] Level 1 Warning (Emergency Shutdown): Triggering conditions: The chain breakage determination result is a chain breakage; or the parallelism angle of the two chains is... >3° and the angle between the chain and the frame baseline >2°; Judgment meaning: This indicates that the scraper conveyor has a broken chain or severe misalignment fault, and the machine must be stopped immediately for inspection to avoid equipment damage or safety accidents; The output prompts: "Chain breakage / severe misalignment confirmed, stop the machine immediately for inspection," and simultaneously outputs angle parameters or fracture gap coordinates.

[0128] Level 2 warning (suspected fault): Triggering condition: 2° < ≤3°; or 88°> >92°; or the chain break determination result is suspected chain break but does not meet the thermal characteristic abrupt change condition; or the mask integrity determination is incomplete; Judgment meaning: It indicates that the scraper conveyor has a slight pull, abnormal verticality, or suspected chain breakage, or that the data is unreliable due to material obstruction; The output prompts: "Suspected misalignment / chain breakage, manual verification required," and simultaneously outputs details of abnormal parameters.

[0129] Normal (no abnormal operation): Triggering conditions: ≤2°、88°≤ ≤92° ≤1°; and the chain break determination result is no chain break; and the mask integrity determination result is complete; Judgment meaning: It means that the operating parameters of the scraper conveyor chain and scraper are within the normal range, and there is no pull-off or chain breakage fault; The output message reads: "The device is operating normally."

[0130] The final output includes the warning level (Level 1 Warning / Level 2 Warning / Normal), core judgment parameters (three types of angle values, mask integrity result, chain break judgment result), and handling prompts. All output data can be uploaded to the scraper conveyor monitoring platform in real time, supporting fault tracing and historical data query.

[0131] The following describes a specific embodiment of a scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening.

[0132] A scraper conveyor chain breakage identification system based on thermal imaging and multi-dimensional quality screening is applied to the monitoring scenario of scraper conveyors in underground coal mines.

[0133] Thermal imaging data acquisition module: A thermal imaging camera with a resolution of 1280×720 (thermal sensitivity of 40mK) is installed 2m to the side of the scraper conveyor, with the lens parallel to the plane of the chain, at a frame rate of 15fps, to acquire 300 minutes of video of the scraper conveyor in operation, covering daytime natural light, nighttime lighting, local shadows, and working conditions with no material, a small amount of material, and a medium amount of material.

[0134] Multi-dimensional quality screening module: The video is converted into 270,000 images at 15fps. The dHash value is calculated to remove duplicate frames, resulting in 180,000 deduplicated images. 12,000 blurry frames, 3,000 extreme frames, and 8,000 hot feature blurry frames are removed, resulting in 157,000 effective images (preprocessed). Effective image rate = 157,000 / 270,000 ≈ 0.581. Scene score: 28 points; Clarity score: 22 points; Integrity score: 14 points; No interference score: 8 points; Hot feature score: 18 points; Quality score = (28+22+14+8+18)×0.581 ≈ 60.8 points; 155,000 effective images are selected.

[0135] Data labeling and preprocessing module: Polygon annotation is performed on valid images using LabelStudio to generate JSON files; the data is converted to a YOLOv8-Seg supported format using a Python script, and the training set (124,000 images), validation set (15,500 images), and test set (15,500 images) are divided in an 8:1:1 ratio; data augmentation is performed using brightness adjustment, contrast adjustment, Gaussian blur, slight rotation, material occlusion simulation, and thermal feature perturbation.

[0136] Instance segmentation model training module: The yolov8s-seg.pt pre-trained model is used, and a custom scraper_chain_seg.yaml configuration file is used. The training parameters are set and the model is trained on a 16GB GPU. After training, train / loss_seg drops to 0.06, val / seg_iou stabilizes at 0.90 (calculated according to the seg_iou formula), val / precision and val / recall are both 0.94, and the model is exported in ONNX format.

[0137] Post-mask processing module: Loads the ONNX model, acquires images in real time, infers and extracts binary masks, processes them through 3×3 closing operations and 2×2 opening operations, and filters the contours using the polygon area formula to obtain a clean target mask.

[0138] Line Fitting Module: The mask is refined by Zhang-Suen and noise filtered to extract the coordinate set. The line equation is obtained by using the least squares method (minimizing the error function) and Hough transform correction (based on the Hough space parameter formula and the point-to-line distance formula).

[0139] Chain breakage feature detection module: Analyzes the chain mask with a connected component count of 1. Calculates the distance between adjacent pixels along the center line using the adjacent pixel spacing formula, ensuring it is less than 10 pixels. Simultaneously extracts the entire chain temperature data, calculating the average temperature to be 32.5℃. Set the normal temperature range to 27.5-37.5℃, with temperature fluctuations in each region ≤2℃. Calculates the calorific value difference between the two chain links using the calorific value difference rate formula, ensuring it is less than 10% and there are no abnormal temperature areas, thus determining the chain is not broken. When simulating a chain breakage, the connected component count is 2, the break gap is 15 pixels (calculated using the connected component spacing formula), and the temperature at the break point suddenly rises to 58℃, exceeding the normal range by 20.5℃. The thermal feature mutation rate is 35% (calculated using the calorific value difference rate formula), and a temperature heatmap is simultaneously generated to display the abnormally high temperature point, indicating a Level 1 warning (chain breakage). When simulating slight misalignment... =2.6°, the local temperature of the chain rose to 41°C, exceeding the normal range by 3.5°C, and was marked as a level 2 warning (suspected pull-off accompanied by local overheating); Angle Calculation and Comprehensive Judgment Module: Calculation based on the formula for the angle between normal vectors =1.3°、 =90.2° =0.7°, the maximum outline area of ​​the mask accounts for 86%, which is considered normal operation; when simulating a chain break, the number of connected components = 2, the break gap = 15 pixels (calculated according to the connected component spacing formula), and the thermal feature mutation = 35% (calculated according to the calorific value difference rate formula), which is considered a level one warning (chain break); when simulating a slight pull, =2.6°, judged as a level 2 warning (suspected deviation).

[0140] like Figure 2 As shown, this application also provides a method for identifying broken chains by scraper pull using thermal imaging and multi-dimensional quality screening, applied to the aforementioned system. The method includes: S1: Acquire thermal imaging images or videos of the scraper conveyor during operation using a thermal imaging camera, and output raw data containing the thermal characteristics of the chain and scraper. S2: Preprocess the raw data to output a valid image dataset that meets the quality requirements. The preprocessing includes image generation, deduplication, bad sample filtering, quality score calculation, and valid images. S3: Use LabelStudio to annotate the left chain, right chain and scraper in the valid image dataset with polygonal outlines, and convert the annotated JSON format data into YOLOv8-Seg supported format data, and then divide the valid image dataset to form training data; S4: Construct a YOLOv8-Seg model based on YOLOv8-Seg, configure training parameters, train using training data, monitor the training metrics of the YOLOv8-Seg model, and output the trained instance segmentation model when the YOLOv8-Seg model converges. S5: Use the instance segmentation model to infer the real-time thermal imaging image, extract the binary mask of the chain and the scraper, remove noise and holes through morphological operations, and output the target mask and mask integrity. S6: Extract the skeleton and filter noise from the target mask to obtain the coordinate set of the center line of a single pixel in the target mask. Using the fusion method of least squares and Hough transform, based on the solution of the straight line equation and the distance formula, output the straight line equation of the chain and the scraper. S7: Based on the target mask and the coordinate set of the center line of a single pixel, the chain breakage determination result is output by analyzing the continuity of chain links, detecting the break gap, and identifying abrupt changes in thermal features, combined with the spacing formula and the heat value difference rate formula. S8: Calculate three types of core angles based on the formula of the angle between the normal vectors, and combine the mask integrity and chain breakage judgment results to make graded judgments on pull deviation and chain breakage, and output the warning level and judgment result. The three types of core angles include the chain parallelism angle, the chain perpendicularity angle to the scraper, and the angle between the chain and the frame baseline.

[0141] It should be noted that for a description of a scraper pull-off chain breakage identification method based on thermal imaging and multi-dimensional quality screening, please refer to the description of the scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening, which will not be repeated here.

[0142] The following specific embodiment further illustrates the scraper pull-off chain breakage identification method using thermal imaging and multi-dimensional quality screening.

[0143] A method for identifying broken chains by scraper pull using thermal imaging and multi-dimensional quality screening includes the following steps: S1: Thermal imaging data acquisition, using a thermal imaging camera to collect 120 minutes of video of the scraper conveyor in operation, with a resolution of 1280×720 and a frame rate of 12fps, covering different lighting and material coverage conditions. S2: Multi-dimensional quality screening. Converted to 86,400 images at 12fps, after deduplication, 60,000 images are obtained. 4,500 blurry frames, 1,500 extreme frames, and 3,000 hot feature blurry frames are removed, leaving 51,000 valid images (preprocessed). Valid image rate = 51,000 / 86,400 ≈ 0.590. Quality score = (26+20+13+7+16)×0.590 ≈ 61.2 points. 50,000 valid images are selected. S3: Data labeling and preprocessing. Use LabelStudio to perform polygon labeling on valid images, convert them to a YOLOv8-Seg supported format, divide the dataset, and perform data augmentation on the training set samples. S4: Instance segmentation model training, build the model based on YOLOv8-Seg, configure the training parameters, train until val / seg_iou=0.88 (calculated according to the intersection-union formula), and output the trained model; S5: Mask post-processing, the model infers and extracts binary masks in real time, removes noise and holes through morphological operations, performs quality checks by combining the contour area formula, and outputs clean masks. S6: Line fitting, extract the coordinate set of the mask center line, and obtain the accurate line equation by using the "least square method + Hough transform" fusion scheme based on the solution of the line equation, the Hough space parameter formula and the point-to-line distance formula. S7: Chain break feature detection. The number of connected components in the chain mask is 2. The distance between adjacent connected components is calculated to be 12 pixels using the formula for distance between adjacent connected components. The break gap is marked as 12 pixels using the formula for distance between adjacent pixels. The temperature data of the broken area and the chain links on both sides are extracted. The average temperature of the normal chain link is 33℃. The temperature at the break gap drops sharply to 18℃ (due to the lack of frictional heat generation after the chain breaks). The temperature difference rate is 28%, which exceeds the normal temperature range (28-38℃) by 10℃. This is determined to be a sudden change in thermal characteristics. The chain is determined to be broken. S8: Angle Calculation and Comprehensive Judgment, based on the formula for the angle between normal vectors. =1.8° =89.5° =0.9°, the mask is intact, which is judged as a level 1 warning (link failure), prompting an immediate shutdown for inspection.

[0144] In summary, this application has the following beneficial effects: 1. Multi-fault collaborative identification: Simultaneously achieves accurate identification of two core faults, scraper misalignment and chain breakage, overcoming the limitations of single fault detection in existing technologies and meeting the comprehensive monitoring needs of industrial scenarios; 2. Strong adaptability to complex working conditions: Data is collected using thermal imaging technology, which is not limited by lighting conditions. It can effectively capture sudden changes in thermal characteristics at the chain break and is suitable for complex working conditions such as weak light, strong light reflection, and material coverage. 3. Reliable data quality: Through multi-dimensional quality screening technology, invalid samples such as duplicates, fuzzy samples, and samples with unclear hot features are filtered out. Combined with hot feature scoring evaluation, the effectiveness of model training and inference data is ensured. 4. High recognition accuracy: The YOLOv8-Seg model is used to achieve pixel-level extraction of targets. The "least square method + Hough transform" fusion fitting scheme and standardized geometric formulas are combined to improve the accuracy of line fitting. The chain break recognition integrates connected component analysis, gap detection and thermal feature mutation verification. Multi-dimensional quantification formulas are used to reduce the rate of missed and false positives. 5. No fault sample support required: By labeling normal samples, and combining instance segmentation, geometric calculation and thermal feature analysis, two types of faults can be identified, solving the problem of scarce fault samples in industrial scenarios; 6. Strong industrial applicability: The model structure and parameters are optimized to adapt to industrial edge devices, support real-time inference at ≥10fps, and have implementation guarantee measures such as dynamic benchmark updates and manual backup mechanisms. It can be directly applied to complex industrial scenarios such as coal mines and mines.

[0145] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0146] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0147] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening, characterized in that, include: The thermal imaging data acquisition module is used to acquire thermal imaging images or videos of the scraper conveyor during operation using a thermal imaging camera, and output raw data containing the thermal characteristics of the chain and scraper. A multi-dimensional quality screening module is connected to the thermal imaging data acquisition module. It is used to receive the raw data, preprocess the raw thermal imaging data, and output a valid image dataset that meets the preset quality requirements. The instance segmentation model training module is connected to the multi-dimensional quality screening module. It is used to construct a scraper and chain recognition model, and to train it using an effective image dataset, and output the trained instance segmentation model. The mask post-processing module, connected to the multi-dimensional quality screening module, is used to infer the real-time thermal imaging image acquired by the thermal imaging data acquisition module using the trained instance segmentation model, extract the binary mask of the chain and the scraper, remove noise and holes through morphological operations, and output the target mask and mask integrity. The straight line fitting module, connected to the mask post-processing module, is used to extract the skeleton and filter noise from the target mask, obtain the coordinate set of the center line of a single pixel in the target mask, and use a fusion method of initial fitting by least squares method and correction by Hough transform to output the straight line equation of the chain and the scraper based on the solution of the straight line equation and the distance formula. The chain breakage feature detection module, connected to the straight line fitting module, is used to output the chain breakage determination result based on the target mask and the single pixel center line coordinate set, through chain link continuity analysis, break gap detection and thermal feature mutation identification of the chain. The angle calculation and comprehensive judgment module is connected to the straight line fitting module and the chain break feature detection module, respectively. It is used to calculate the parallelism angle of the chain on both sides of the scraper, the perpendicularity angle between the chain and the scraper, and the angle between the chain and the frame baseline based on the formula of the normal vector angle of the straight line equation. It also combines the mask integrity and the chain break judgment result to perform pull deviation and chain break classification judgment, and output the warning level and judgment result.

2. The scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening as described in claim 1, characterized in that, The thermal imaging data acquisition module is configured as follows: Using a thermal imaging camera that vibrates parallel to the plane of the chain and at the same frequency, the thermal radiation differences between the chain, the scraper, and the surrounding environment, as well as the abrupt changes in thermal characteristics at the chain breakage point, are collected during the operation of the scraper conveyor under different ambient light and material coverage conditions. Thermal imaging images or videos are generated, and raw data containing the thermal characteristics of the chain and scraper are output. The different ambient light includes natural daylight, nighttime artificial light, strong light reflection, and local shadow conditions. The material coverage includes material occlusion of 0, material coverage of the chain within a first threshold, and material coverage of the chain within a second threshold. The first threshold is greater than 0 and less than the second threshold.

3. The scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening as described in claim 1, characterized in that, The multi-dimensional quality screening module is configured as follows: The system receives the raw data and extracts image frames from the thermal imaging image or video frames from the thermal imaging video according to the set frame rate, forming an image frame set. In the target image frame set, a first preprocessing is performed to obtain an intermediate image frame set. The first preprocessing includes calculating the dHash value of the target image frame and removing duplicates based on the dHash value, retaining clear frames. The intermediate image frame set is subjected to a second preprocessing to output a valid image dataset that meets the preset quality requirements. The second preprocessing includes the removal of sharpness, extreme frames, and thermally blurred frames.

4. The scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening as described in claim 1, characterized in that, It also includes a data annotation and preprocessing module, which is connected to the multi-dimensional quality screening module and the instance segmentation model training module, respectively. The data annotation and preprocessing module is used to receive the effective image dataset, perform polygon contour annotation on the left and right chains of the chain and the scraper in each effective image in the effective image dataset, and output the labeled effective image dataset to the instance segmentation model training module. The labeled effective image dataset includes a training set, a validation set and a test set.

5. The scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening as described in claim 4, characterized in that, The instance segmentation model training module is configured as follows: Based on the YOLOv8-Seg.pt pre-trained model, a scraper and chain recognition model is constructed, and parameters are configured for the scraper and chain recognition model to obtain the scraper and chain recognition model; Using the labeled valid image dataset, the scraper and chain recognition model is trained, and the convergence of the training loss function and the intersection-union ratio index is monitored. When the convergence reaches the preset convergence value, the trained instance segmentation model is output.

6. The scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening as described in claim 1, characterized in that, The mask post-processing module is configured as follows: Using n×n rectangular structuring elements, morphological closing operations are performed to fill in the holes in the binary mask of the chain and the scraper, and the holes in the binary mask of the chain and the scraper are extracted by inference on the real-time thermal imaging image acquired by the thermal imaging data acquisition module using the trained instance segmentation model, thus obtaining the continuous outline of the chain and the scraper. Morphological opening operations are performed using m×m rectangular structuring elements to remove noise from the continuous contours of the chain and the scraper, resulting in a target mask. The target mask represents a binarized image containing the target regions of the chain and the scraper. The contour area of ​​the target mask is calculated using the polygon area formula to obtain the maximum contour area. The polygon area formula is as follows: For the coordinates of the contour vertices ,area ; The mask integrity is obtained based on the maximum contour area, wherein when the maximum contour area is greater than or equal to a preset contour area value, the mask integrity is considered complete; otherwise, the mask integrity is considered incomplete.

7. The scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening as described in claim 1, characterized in that, The line fitting module is configured as follows: Using the Zhang-Suen thinning algorithm, the target mask is transformed into a single-pixel centerline while preserving the contour topology of the target mask; The isolated line segments of the single-pixel centerline are filtered out, and the contour arc lengths that are greater than or equal to a preset pixel length are retained to obtain the target single-pixel centerline. The formula for calculating the contour arc length is as follows: For consecutive vertices and Arc length L = ; in, This represents the x-axis coordinate of pixel i within the center line of a single pixel. The arc length L represents the y-axis coordinate of pixel i in the center line of a single pixel, and the arc length L represents the total length of the outline of the center line of a single pixel. Based on the set of centerline pixel coordinates corresponding to the centerline of the target single pixel, the least squares method is used to fit the straight line equation to obtain the initial fitted straight line equation. The Hough transform is used to detect straight lines with a length greater than or equal to 50 pixels in the center line of the target single pixel, the angle of the main direction is determined, and the deviation between the angle of the main direction and the angle of the straight line formed by the initial fitted straight line equation is calculated to obtain the calculation result. If the deviation of the calculation result is greater than the preset angle deviation threshold, then after removing the abnormal points of pixels whose distance from the main direction is greater than the preset angle deviation threshold, the straight line equation of the chain and the scraper is refitted and output. If the deviation of the calculation result is less than or equal to the preset angle deviation threshold, the initial fitted straight line equation is directly used to output the straight line equation of the chain and the scraper.

8. The scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening as described in claim 1, characterized in that, The chain break feature detection module is configured as follows: Based on the target mask of the chain, the number of connected components in the chain outline is calculated. When the number of connected components is ≥2 and the distance between adjacent connected components is greater than a preset distance threshold, it is determined to be a suspected chain break. The distance between adjacent connected components represents the center-to-center distance between the minimum bounding rectangles of two connected components, and the formula is: ； Where d represents the distance between the centers of the minimum bounding rectangles of the two connected components. and Indicates the coordinates of the center of the first connected domain. and Indicates the coordinates of the center of the second connected domain; The distance between adjacent pixels is calculated based on the coordinate set of the single pixel centerline. When the distance is greater than a preset gap threshold and there is no material obstruction in the corresponding area, the break gap between adjacent pixels is obtained. The formula for the distance between adjacent pixels is: ； in, Indicates the distance between adjacent pixels. and This represents the coordinates of the i-th pixel. and Represented as the coordinates of the j-th pixel; Based on the calorific value data of the fracture gap and the chain links on both sides, the calorific value difference rate of the chain links on both sides is calculated and compared with a preset reference temperature range. When the calorific value difference rate is greater than a preset difference threshold, or when the temperature of the fracture gap exceeds the reference temperature range and reaches a preset temperature threshold, it is determined to be a thermal characteristic abrupt change. The formula for calculating the calorific value difference rate is as follows: ； in, Indicates the difference rate of calorific value. This represents the average heat value of the fracture gap region. This indicates the average heat value of the normal area of ​​the chain; The chain breakage determination result is obtained by comprehensively judging the suspected chain breakage, the fracture gap, and the thermal characteristic mutation. If the chain breakage determination result satisfies the suspected chain breakage result, the existence of at least one fracture gap, and the thermal characteristic abrupt change, a first output chain breakage determination result is obtained, wherein the first output chain breakage determination result represents either a broken chain or a non-broken chain. If the chain breakage determination result satisfies the suspected chain breakage result and the existence of at least one breakage gap, then a supplementary determination is made in combination with the scraper running speed to obtain a second output chain breakage determination result. The second output chain breakage determination result indicates that when the scraper running speed is ≥ the preset speed, it is determined to be suspected breakage, and when the scraper running speed is < the preset speed, it is determined to be under observation.

9. The scraper pull-off chain breakage identification system based on thermal imaging and multi-dimensional quality screening as described in claim 1, characterized in that, The angle calculation and comprehensive judgment module is configured as follows: Based on the formula for the included angle of the normal vector according to the equation of the straight line, the parallelism angle of the chain on both sides of the scraper, the perpendicularity angle of the chain to the scraper, and the angle between the chain and the frame reference line are calculated. The formula for calculating the included angle of the normal vector is as follows: ； in, This represents the normal vector between two lines. Represents the dot product. and Indicates the modulus length. , Represent the equation of the first straight line The coefficients of the x and y terms in the equation. , Indicate the equation of the second line The coefficients of the x and y terms in the equation, where x represents the x-axis coordinate of the pixel, y represents the y-axis coordinate of the pixel, and c represents the constant term of the equation of the line; The chain parallelism angle Based on the equations of the left and right chain lines, the angle between the normal vectors is calculated using the formula: Equation of the straight line of the left chain The calculation formula is ; Equation of the straight line of the right chain The calculation formula is ; Calculated using the formula for the included angle of the normal vectors; in, Indicates the straight line of the left chain. The constant term, Indicates the straight line of the chain on the right. The constant term, , They represent lines respectively. and The normal vector; The perpendicularity angle between the chain and the scraper Based on the equations of the chain and the scraper, the angle between the normal vectors is calculated using the formula: Chain straight line equation The calculation formula is ; The formula for calculating the straight line of the scraper is: ; Calculated using the formula for the included angle of the normal vectors; in, and Represents a straight chain The coefficients of the x and y terms in the equation. Represents a straight chain The constant term, and Indicates the straight line of the scraper The coefficients of the x and y terms in the equation. Indicates the straight line of the scraper The constant term, , They represent lines respectively. and The normal vector; The angle between the chain and the frame baseline Based on the straight line equation of the chain and the straight line equation of the frame baseline, the angle between the normal vectors is calculated using the formula described above. Chain straight line equation The calculation formula is ; Rack baseline The calculation formula is ; Calculated using the formula for the included angle of the normal vectors; in, and Indicates the rack baseline The coefficients of the x and y terms in the equation. Indicates the rack baseline The constant term, , They represent lines respectively. and The normal vector; The integrity of the mask is obtained, and the chain breakage determination result is combined to perform a pull-off and chain breakage classification determination to obtain the output warning level and determination result. The chain breakage classification determination includes at least a first-level warning, a second-level warning, and a normal condition. The first-level warning indicates that the chain breakage determination result is a chain breakage, or the parallelism angle of the two sides of the chain is greater than the first angle and the angle between the chain and the frame baseline is greater than the second angle. The second-level warning indicates that the second angle is less than the parallelism angle of the two sides of the chain and less than the first angle, or the perpendicularity angle of the chain and the scraper is less than the third angle, or the perpendicularity angle of the chain and the scraper is greater than the fourth angle, or the chain breakage is suspected but does not meet the thermal characteristic change condition, or the mask integrity is incomplete. The normal condition indicates that the parallelism angle of the two sides of the chain is less than the second angle, the third angle is less than the perpendicularity angle of the chain and the scraper and less than the fourth angle, the angle between the chain and the frame baseline is less than the fifth angle, and the chain breakage determination result is no chain breakage.

10. A method for identifying broken chains by scraper pull using thermal imaging and multi-dimensional quality screening, characterized in that, The method, applied to the system of any one of claims 1 to 9, comprises: S1: Acquire thermal imaging images or videos of the scraper conveyor during operation using a thermal imaging camera, and output raw data containing the thermal characteristics of the chain and scraper. S2: Preprocess the raw data to output a valid image dataset that meets the quality requirements. The preprocessing includes image generation, deduplication, bad sample filtering, quality score calculation, and valid images. S3: Use LabelStudio to annotate the left chain, right chain and scraper in the valid image dataset with polygonal outlines, and convert the annotated JSON format data into YOLOv8-Seg supported format data, and then divide the valid image dataset to form training data; S4: Construct a YOLOv8-Seg model based on YOLOv8-Seg, configure training parameters, train using training data, monitor the training metrics of the YOLOv8-Seg model, and output the trained instance segmentation model when the YOLOv8-Seg model converges. S5: Use the instance segmentation model to infer the real-time thermal imaging image, extract the binary mask of the chain and the scraper, remove noise and holes through morphological operations, and output the target mask and mask integrity. S6: Extract the skeleton and filter noise from the target mask to obtain the coordinate set of the center line of a single pixel in the target mask. Using the fusion method of least squares and Hough transform, based on the solution of the straight line equation and the distance formula, output the straight line equation of the chain and the scraper. S7: Based on the target mask and the coordinate set of the center line of a single pixel, the chain breakage determination result is output by analyzing the continuity of chain links, detecting the break gap, and identifying abrupt changes in thermal features, combined with the spacing formula and the heat value difference rate formula. S8: Calculate three types of core angles based on the formula of the angle between the normal vectors, and combine the mask integrity and chain breakage judgment results to make graded judgments on pull deviation and chain breakage, and output the warning level and judgment result. The three types of core angles include the chain parallelism angle, the chain perpendicularity angle to the scraper, and the angle between the chain and the frame baseline.

Images