A sintering machine trolley multi-component abnormal diagnosis method based on scale perception semantic enhancement segmentation

Abstract

The application discloses a sintering machine trolley multi-component abnormality diagnosis method based on scale perception semantic enhancement segmentation, and belongs to the technical fields of industrial visual detection, intelligent manufacturing, equipment state monitoring, image segmentation and edge intelligence. The method aims to solve the problems that the existing target frame detection is difficult to describe the pixel-level shape of components, high-density grate bar segmentation is prone to adhesion, wheel appearance defects and position abnormalities are difficult to be uniformly diagnosed, and complex networks are not conducive to industrial edge deployment.

Description

Technical Field

[0001] This invention belongs to the fields of industrial visual inspection, intelligent manufacturing, equipment status monitoring, image segmentation and edge intelligence technology. Specifically, it relates to a method for diagnosing anomalies of multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation. It is applicable to the real-time or near-real-time detection and diagnosis of appearance damage, positional displacement, loosening, detachment, breakage or missing status of wheels and grates of sintering machine trolleys in sintering production environments with high temperature, dust, obstruction and high density of components. Background Technology

[0002] The sintering machine trolley is a crucial operating component in the sintering production line, carrying the mixture and completing processes such as ignition, sintering with ventilation, and unloading. During its cyclical operation, the trolley is subjected to alternating heavy loads, high-temperature baking, dust erosion, and mechanical vibration over extended periods. Key components such as its wheels and grate bars are prone to wear, deformation, loosening, breakage, misalignment, or detachment. Abnormal wheel conditions can lead to increased running resistance, trajectory deviation, or even operational accidents; abnormal grate bars can cause reduced material carrying capacity, material leakage, or partial collapse, affecting production continuity and equipment safety.

[0003] Currently, the status inspection of sintering machine trolleys mainly relies on manual inspection, shutdown inspection, or the installation of contact sensors. Manual inspection is limited by the observation angle, inspection frequency, and working environment, making it difficult to continuously cover the high-speed running trolley. Contact sensors are exposed to high temperature, high dust, and strong vibration environments for a long time, which can easily lead to shortened lifespan, signal drift, or failure. Moreover, they can usually only reflect local vibration or temperature changes and cannot directly obtain the appearance and geometric condition of wheel flange damage, tread peeling, grate breakage, and missing bars.

[0004] With the development of industrial cameras and deep learning technology, it has become possible to inspect sintering machine trolley components using non-contact machine vision. However, images of sintering machine trolleys typically exhibit characteristics such as complex backgrounds, dust occlusion, significant differences in target scale, partial occlusion of wheels by the trolley body, and numerous, small-sized, and narrowly spaced grates. Traditional target detection methods based on bounding rectangles can only provide the approximate location of components, making it difficult to obtain pixel-level morphology of component edges, and also difficult to directly determine subtle positional shifts of wheels, localized breaks in grates, or adhesion between adjacent grates using the bounding box.

[0005] For densely packed grates, traditional segmentation networks tend to lose information about narrow gaps and slender edges after multiple downsampling, causing the predicted areas of adjacent grates to stick together. Dust, accumulated material, and shadows can also produce false edges or masking holes. For wheels, while a single segmentation result can reflect the contour, its ability to classify appearance defects such as tread peeling and rim damage is limited. If only a classification network is used, it is easily affected by background and occlusion areas.

[0006] Furthermore, sintering production sites typically require detection models to be deployed on workshop-side edge computing nodes or industrial edge servers. General-purpose large-parameter segmentation networks have high computational costs, making it difficult to balance pixel-level detection accuracy with on-site inference latency. Therefore, there is a need for a method that addresses multiple components of sintering machine trolleys, enhances high-density target semantics and edge information based on lightweight feature extraction, and combines geometric analysis and morphological analysis to complete anomaly diagnosis. Summary of the Invention

[0007] The purpose of this invention is to provide a multi-component anomaly diagnosis method for sintering machine trolleys based on scale-aware semantic enhancement segmentation, to solve the problems of existing target bounding box detection's inability to describe the pixel-level morphology of components, the tendency for high-density grate segmentation to stick together, the difficulty in uniformly diagnosing wheel appearance defects and positional anomalies, and the unfavorable deployment of complex networks at industrial edge. This method takes the sintering machine trolley's running image as input and outputs pixel-level masks of wheels and grates, anomaly categories, anomaly locations, and alarm information.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: A method for diagnosing anomalies in multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation includes the following steps: Step S1: Obtain images of multiple components of the sintering machine trolley and establish the detection area.

[0009] Industrial cameras are used to capture images of the sintering machine trolley, positioned on the side, bottom, or near the unloading end of the trolley's running path. Based on the camera installation location and the trolley's running direction, wheel detection areas and grate bar detection areas are determined within the trolley images. The captured trolley images are then normalized in size, and brightness normalization, noise reduction, or contrast adjustment are performed as needed to form the image to be detected.

[0010] When creating training samples, pixel-level masking is first performed on the wheels and grates in the images to be detected. Then, for each individual wheel instance, further labeling is done to categorize it as normal, tread peeling, flange damage, or other cosmetic defects. After sample labeling is complete, the samples are divided into training, validation, and test sets for training the wheel and grate segmentation network and the wheel appearance defect classification network.

[0011] Step S2: Extract multi-scale component features using a lightweight backbone network.

[0012] The labeled image to be detected is input into a lightweight backbone network built on the MobileNetV2 inverse residual module. Multi-scale features are extracted at different downsampling rates to obtain high-resolution local features. Medium resolution structural features Low-resolution global semantic features The inverse residual module includes a main convolutional branch and a shortening branch. The main convolutional branch sequentially includes channel-expanding convolution, depthwise separable convolution, and linear projection convolution. Channel-expanding convolution increases the number of channels for intermediate features, depthwise separable convolution extracts local spatial features, and linear projection convolution maps intermediate features to a preset number of output channels. The shortening branch is a connection path that bypasses the channel-expanding convolution, depthwise separable convolution, and linear projection convolution, directly transmitting the input features of the inverse residual module to the module's output.

[0013] Different scale features output by the lightweight backbone network , , Input a lightweight feature pyramid. The lightweight feature pyramid comprises low-resolution global semantic features. To high-resolution local features The top-down path of hierarchical propagation, and the lateral connection path connecting the lightweight feature pyramid to the corresponding scale features of the lightweight backbone network. First, low-resolution global semantic features... Low-resolution fusion features are obtained by adjusting the channels. Then on Upsampling is performed, and then compared with the medium-resolution structural features after channel adjustment. Channel fusion is performed to obtain medium-resolution fusion features. ; and then High-resolution local features after channel adjustment Channel fusion is performed to obtain high-resolution fused features. .in, It features high spatial resolution to preserve component edges and gaps between adjacent grate bars; It has a large receptive field, which can be used to characterize the overall structure of the wheel and the global semantics in complex backgrounds, and suppress spurious responses caused by dust, material accumulation and complex backgrounds; Used to characterize mesoscale structural information that lies between the two.

[0014] Step S3: Perform multi-scale semantic enhancement fusion and scale-aware mask prediction.

[0015] Step 3.1: Construct the semantic insertion module: The fusion features obtained based on step S2 are: and low-resolution global semantic features , Low-resolution fusion features Medium resolution fusion features or high-resolution fusion features First, Perform size alignment to obtain Having the same spatial dimensions ;according to Generate semantic weights and utilize the semantic weights Fusion features The feature responses of each spatial location are weighted element-wise, and the weighted features are then compared with... Adding them together yields semantically enhanced features. : Where Resize represents the size alignment operation, Conv represents the convolution transformation, and Size represents the fused features. The spatial dimensions are given by σ, which represents the Sigmoid function, and ⊙, which represents element-wise multiplication.

[0016] Step 3.2, Feature Fusion After obtaining semantic enhancement features at different scales, these semantic enhancement features are again fused using a lightweight feature pyramid in step S2. Specifically, this involves: low-resolution semantic enhancement features... Channel adjustment yields semantically enhanced fusion features ; Enhance features with low-resolution semantics Based on this, upsampling is performed and combined with adjacent high-resolution semantic enhancement features. By fusing the features, semantically enhanced fusion features are obtained. Semantic enhancement fusion features Upsampling and combining with high-resolution semantic enhancement features By fusing the features, semantically enhanced fusion features are obtained. .

[0017] Step 3.3, Scale-Aware Mask Prediction The multi-scale semantic enhancement fusion features are input into the corresponding mask prediction branch to generate wheel instance masks and comb binary masks, respectively.

[0018] Step S4: Perform cascaded classification of appearance defects based on the wheel mask.

[0019] The wheel outline and its minimum bounding region are extracted from the wheel instance mask. The background region outside the mask is set to zero or the corresponding pixel weight is reduced to obtain a mask-constrained wheel image block. The wheel image block is input into the wheel appearance defect classification network, which outputs the classification results and corresponding confidence scores for wheel normal, tread peeling, flange damage, and other preset defect categories.

[0020] Step S5: Determine positional offset and detachment anomalies based on the centroid of the wheel mask and the adjacent spacing.

[0021] Mask for each wheel instance Calculate the geometric centroid coordinates ( , ).when When (x, y) is the value of the wheel instance mask at pixel position (x, y), the centroid coordinates are calculated as follows: A reference trajectory is established based on the wheel center-of-gravity sample of a normally operating trolley. The reference trajectory can be a straight line, a piecewise straight line, or a fitted curve.

[0022] For a straight line The local reference trajectory represents the vertical deviation of the i-th wheel's center of mass from the reference trajectory. for: When the reference trajectory is a segmented straight line, calculate the shortest distance from the wheel's center of mass to each reference line segment, and take the minimum value as the vertical deviation from the wheel's center of mass to the reference trajectory. .

[0023] When the reference trajectory is a fitted curve, a reference curve is obtained by using polynomial fitting or spline fitting based on multiple wheel centroid samples under normal operating conditions; the Euclidean distance between the centroid of the wheel to be detected and each candidate point on the reference curve is calculated. : The minimum Euclidean distance is taken as the vertical deviation from the wheel's center of mass to the reference trajectory. .

[0024] when If the deviation exceeds a preset trajectory deviation threshold, the corresponding wheel is deemed to have a risk of positional deviation. For curved reference trajectories, the trajectory deviation is defined as the distance from the wheel's center of mass to the nearest point on the reference trajectory.

[0025] Sort the centroids of the wheels on the same or consecutive trolleys according to their direction of travel, and calculate the Euclidean distance between the centroids of two adjacent wheels. The reference spacing is determined based on the normal trolley structural dimensions or normal sample statistical values. And calculate the relative deviation of adjacent spacing. : when If the deviation exceeds a preset spacing threshold, or if no valid wheel instance mask is generated at the predetermined wheel position, an anomaly is determined to exist, such as wheel misalignment, detachment, or missed detection requiring verification. To reduce false alarms caused by occlusion, confirmation can be made by combining detection results from multiple consecutive frames; when the same position continuously meets the abnormal conditions within a preset number of consecutive frames, an abnormal wheel position alarm is output.

[0026] Step S6: Perform mathematical morphological correction on the binary mask of the grating.

[0027] The initial grate binary mask output from the scale-aware mask prediction branch in step S3 is denoted as... To address the scattered noise and mask holes caused by airborne dust, accumulated material obstruction, and reflection, structural element B, adapted to the extension direction of the grate bars, was selected. The opening and closing operations are performed sequentially. The opening operation is used to remove discrete small regions and narrow pseudo-connections, while the closing operation is used to fill in the internal holes and short-distance breakpoints of the grate bars. in, This represents the erosion operation. This represents the expansion operation. Indicates to The result of performing the opening operation, Indicates to The result of the closing operation. Structural element B is set according to the width, extension direction, and camera angle of the grates in the image; for images with significant perspective changes, structural elements of different sizes or orientations are used in different detection sub-regions.

[0028] Step S7: Determine whether the grate bars are broken, missing, or abnormally deformed based on the geometric parameters of the connected components.

[0029] Corrected grating mask Perform connected component labeling to obtain the set of connected components. For each connected component Calculate the area , Width of the outer rectangle ,high Aspect Ratio And skeletal continuity, including aspect ratio for: Based on normal grate bar samples, thresholds for connected region area, aspect ratio, target quantity, and adjacent spacing are determined. When the connected region area is less than the preset connected region area threshold, the aspect ratio is less than the preset aspect ratio threshold, the area corresponding to a single grate bar is split into multiple separate connected regions, or the skeleton is discontinuous in the middle, the corresponding grate bar is determined to have a break or severe occlusion and is an anomaly requiring review.

[0030] When there is no connected region that meets the area and aspect ratio requirements at the predetermined grate position, or when the total number of valid connected regions of the grate in the detection area is less than the lower limit of the normal number, it is determined that there is an abnormality of missing or detached grate; when the distance between adjacent connected regions deviates from the normal distance range or the main axis direction of the connected region deviates from the normal direction range, it is determined that there is a risk of grate loosening, twisting or abnormal deformation.

[0031] Step S8: Merge the multi-branch diagnostic results and output abnormal information.

[0032] The results of wheel appearance defect classification, wheel trajectory deviation, relative deviation of adjacent wheel spacing, geometric parameters of grate connected regions, and statistical results of grate quantity are fused to generate a multi-component status record for the trolley. The status record includes at least the trolley or image number, component type, anomaly category, anomaly location, detection confidence level, and acquisition time.

[0033] When any abnormal indicator meets the corresponding alarm condition, the edge computing node on the workshop side outputs an audible and visual alarm, interface mark, or communication message, and sends an image with mask outline and abnormal location mark to the upper monitoring system so that maintenance personnel can review and arrange maintenance.

[0034] In one implementation, the wheel trajectory deviation threshold, adjacent spacing deviation threshold, grate area threshold, aspect ratio threshold, and effective quantity threshold are all obtained statistically from normal operation samples and can be configured according to the camera installation location, trolley model, and operating speed. The aforementioned thresholds are expressed as preset thresholds and are not limited to a fixed value.

[0035] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention uses pixel-level mask segmentation instead of the detection method that only outputs the outer rectangular box, which can obtain the fine contour of the wheel and the grate bar, providing a direct data basis for calculating the geometric position of the wheel and determining the breakage and missingness of the grate bar.

[0036] (2) Based on the lightweight feature pyramid, this invention introduces a semantic insertion module and a multi-scale semantic enhancement fusion mechanism. By modulating high-resolution local features with global semantics, it can reduce mask adhesion in high-density grate areas and suppress false features generated by dust, material accumulation and complex background.

[0037] (3) The present invention cascades wheel mask segmentation with a two-level ResNet appearance damage classification network, which not only uses the mask to remove irrelevant background, but also retains the wheel flange and tread area, enabling branch diagnosis of abnormal wheel position and appearance defects.

[0038] (4) The present invention uses the wheel mask centroid, reference running trajectory and Euclidean distance between the centroids of adjacent wheels for analytical calculation, which can identify wheel offset, misalignment and risk of falling off without relying on contact sensors.

[0039] (5) The present invention performs directional adaptive opening and closing operations on the grate mask and combines the area of ​​the connected domain, aspect ratio, quantity, spacing and continuity to make anomaly judgments, which can adapt to industrial scenarios with small grate size, dense arrangement and partial occlusion.

[0040] (6) The present invention uses the MobileNetV2 inverted residual module to build a lightweight backbone network, which is easy to deploy on the workshop side edge computing node or industrial edge server to realize online detection and abnormal early warning of key components of sintering machine trolley. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the overall process of the sintering machine trolley multi-component abnormality diagnosis method described in this invention.

[0042] Figure 2 This is a schematic diagram of the network structure for lightweight feature extraction, multi-scale semantic enhancement fusion, and scale-aware mask prediction described in this invention. Detailed Implementation

[0043] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Equivalent substitutions or adjustments made to camera arrangement, number of network layers, number of feature channels, size of structural elements, threshold settings, and alarm methods without departing from the technical concept of the present invention can all be included in the technical solutions of the present invention.

[0044] A method for diagnosing anomalies in multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation includes the following steps: Step S1: Obtain images of multiple components of the sintering machine trolley and establish the detection area.

[0045] Industrial cameras are used to capture images of the sintering machine trolley, positioned on the side, bottom, or near the unloading end of the trolley's running path. Based on the camera installation location and the trolley's running direction, wheel detection areas and grate bar detection areas are determined within the trolley images. The captured trolley images are then normalized in size, and brightness normalization, noise reduction, or contrast adjustment are performed as needed to form the image to be detected.

[0046] When creating training samples, pixel-level mask annotation is first performed on the wheels and grates in the images to be detected. For each individual wheel instance, further annotations are made to classify them as normal, tread peeling, flange damage, or other cosmetic defects. After sample annotation, the samples are divided into training, validation, and test sets for training the wheel and grate segmentation network and the wheel cosmetic defect classification network. During the model detection phase, the wheel centroid position and the distance between adjacent wheel centroids are calculated based on the wheel segmentation mask, and the wheel offset or detachment status is determined accordingly. The grate breakage or missing status is determined based on the connected component area, aspect ratio, and number of grate masks.

[0047] Step S2: Extract multi-scale component features using a lightweight backbone network.

[0048] The labeled image to be detected is input into a lightweight backbone network built on the MobileNetV2 inverse residual module. Multi-scale features are extracted at different downsampling rates to obtain high-resolution local features. Medium resolution structural features Low-resolution global semantic features The inverse residual module includes a main convolutional branch and a shortening branch. The main convolutional branch sequentially includes channel-expanding convolution, depthwise separable convolution, and linear projection convolution. Channel-expanding convolution increases the number of channels for intermediate features, depthwise separable convolution extracts local spatial features, and linear projection convolution maps intermediate features to a preset number of output channels. The shortening branch is a connection path that bypasses the channel-expanding convolution, depthwise separable convolution, and linear projection convolution, directly transmitting the input features of the inverse residual module to the module's output. When the convolution stride of the inverse residual module is 1, and the spatial size and number of channels of the input features are consistent with the output features of the main convolutional branch, the input features are element-wise added to the output features of the main convolutional branch to preserve the input features and reduce information loss during deep feature extraction.

[0049] Different scale features output by the lightweight backbone network , , Input a lightweight feature pyramid. The lightweight feature pyramid comprises low-resolution global semantic features. To high-resolution local features The top-down path of hierarchical propagation, and the lateral connection path connecting the lightweight feature pyramid to the corresponding scale features of the lightweight backbone network. First, low-resolution global semantic features... Low-resolution fusion features are obtained by adjusting the channels. Then on Upsampling is performed, and then compared with the medium-resolution structural features after channel adjustment. Channel fusion is performed to obtain medium-resolution fusion features. ; and then High-resolution local features after channel adjustment Channel fusion is performed to obtain high-resolution fused features. .in, It features high spatial resolution to preserve component edges and gaps between adjacent grate bars; It has a large receptive field, which can be used to characterize the overall structure of the wheel and the global semantics in complex backgrounds, and suppress spurious responses caused by dust, material accumulation and complex backgrounds; Used to characterize mesoscale structural information that lies between the two.

[0050] Step S3: Perform multi-scale semantic enhancement fusion and scale-aware mask prediction.

[0051] Step 3.1: Construct the semantic insertion module: The fusion features obtained based on step S2 are: and low-resolution global semantic features , Low-resolution fusion features Medium resolution fusion features or high-resolution fusion features First, Perform size alignment to obtain Having the same spatial dimensions ;according to Generate semantic weights and utilize the semantic weights Fusion features The feature responses of each spatial location are weighted element-wise, and the weighted features are then compared with... Adding them together yields semantically enhanced features. : Where Resize represents the size alignment operation, Conv represents the convolution transformation, and Size represents the fused features. The spatial dimensions are defined by σ, which represents the Sigmoid function, and ⊙, which represents element-wise multiplication. The semantic weights are... This is used to improve the feature response at the location of the wheel or grate bar, and reduce the feature response of dust, material accumulation and complex background areas, thereby reducing the probability of adhesion between adjacent grate bar masks and helping to preserve the feature differences corresponding to the gaps between adjacent grate bars.

[0052] Step 3.2, Feature Fusion After obtaining semantic enhancement features at different scales, these semantic enhancement features are again fused using a lightweight feature pyramid in step S2. Specifically, this involves: low-resolution semantic enhancement features... Channel adjustment yields semantically enhanced fusion features ; Enhance features with low-resolution semantics Based on this, upsampling is performed and combined with adjacent high-resolution semantic enhancement features. By fusing the features, semantically enhanced fusion features are obtained. Semantic enhancement fusion features Upsampling and combining with high-resolution semantic enhancement features By fusing the features, semantically enhanced fusion features are obtained. .

[0053] Step 3.3, Scale-Aware Mask Prediction The multi-scale semantic enhancement fusion features are input into the corresponding mask prediction branches to generate wheel instance masks and grate binary masks, respectively. In one optional implementation, auxiliary mask prediction branches are set after the semantic enhancement fusion features at different scales. During the training phase, the real mask annotations are scaled according to the output size of each prediction branch, and the corresponding mask prediction loss is calculated separately, so that the prediction branches at different scales are subject to supervised constraints. Wheel targets with a large image area are modeled as a whole using medium and low spatial resolution feature layers; grate targets with small size and dense arrangement are modeled for edges and gaps using high spatial resolution feature layers. Each mask prediction branch outputs the initial mask features at the corresponding scale.

[0054] Step S4: Perform cascaded classification of appearance defects based on the wheel mask.

[0055] The wheel contour and its minimum bounding region are extracted from the wheel instance mask. The background region outside the mask is set to zero or the corresponding pixel weight is reduced to obtain a mask-constrained wheel image block. The wheel image block is input into a wheel appearance defect classification network, which outputs the classification results and corresponding confidence scores for wheel normal, tread peeling, flange damage, and other preset defect categories. In one embodiment, the wheel appearance defect classification network adopts a ResNet network structure.

[0056] By using a cascaded approach of "pixel-level segmentation - mask constraint clipping - appearance classification", the impact of vehicle structure, track, dust and shadows on the classification results is reduced, while the geometric information of wheel flange and tread is preserved by using mask contour.

[0057] Step S5: Determine positional offset and detachment anomalies based on the centroid of the wheel mask and the adjacent spacing.

[0058] Mask for each wheel instance Calculate the geometric centroid coordinates ( , ).when When (x, y) is the value of the wheel instance mask at pixel position (x, y), the centroid coordinates are calculated as follows: A reference trajectory is established based on the wheel center-of-gravity sample of a normally operating trolley. The reference trajectory can be a straight line, a piecewise straight line, or a fitted curve.

[0059] For a straight line The local reference trajectory represents the vertical deviation of the i-th wheel's center of mass from the reference trajectory. for: When the reference trajectory is a segmented straight line, calculate the shortest distance from the wheel's center of mass to each reference line segment, and take the minimum value as the vertical deviation from the wheel's center of mass to the reference trajectory. .

[0060] When the reference trajectory is a fitted curve, a reference curve is obtained by using polynomial fitting or spline fitting based on multiple wheel centroid samples under normal operating conditions; the Euclidean distance between the centroid of the wheel to be detected and each candidate point on the reference curve is calculated. : The minimum Euclidean distance is taken as the vertical deviation from the wheel's center of mass to the reference trajectory. .

[0061] when If the deviation exceeds a preset trajectory deviation threshold, the corresponding wheel is deemed to have a risk of positional deviation. For curved reference trajectories, the trajectory deviation is defined as the distance from the wheel's center of mass to the nearest point on the reference trajectory.

[0062] Sort the centroids of the wheels on the same or consecutive trolleys according to their direction of travel, and calculate the Euclidean distance between the centroids of two adjacent wheels. The reference spacing is determined based on the normal trolley structural dimensions or normal sample statistical values. And calculate the relative deviation of adjacent spacing. : when If the deviation exceeds a preset spacing threshold, or if no valid wheel instance mask is generated at the predetermined wheel position, an anomaly is determined to exist, such as wheel misalignment, detachment, or missed detection requiring verification. To reduce false alarms caused by occlusion, confirmation can be made by combining detection results from multiple consecutive frames; when the same position continuously meets the abnormal conditions within a preset number of consecutive frames, a wheel position anomaly alarm is output.

[0063] Step S6: Perform mathematical morphological correction on the binary mask of the grating.

[0064] Let the initial grate binary mask output from the scale-aware mask prediction branch in step S3 be denoted as... To address the scattered noise and mask holes caused by airborne dust, accumulated material obstruction, and reflection, structural element B, adapted to the extension direction of the grate bars, was selected. The opening and closing operations are performed sequentially. The opening operation is used to remove discrete small regions and narrow pseudo-connections, while the closing operation is used to fill in the internal holes and short-distance breakpoints of the grate bars. in, This represents the erosion operation. This indicates the expansion operation. Indicates to The result of performing the opening operation, Indicates to The result of the closing operation. Structural element B is set according to the width, extension direction, and camera view of the grates in the image; for images with significant perspective changes, structural elements of different sizes or orientations are used in different detection sub-regions.

[0065] Step S7: Determine whether the grate bars are broken, missing, or abnormally deformed based on the geometric parameters of the connected components.

[0066] Corrected grating mask Perform connected component labeling to obtain the set of connected components. For each connected component Calculate the area , Width of the outer rectangle ,high Aspect Ratio And skeletal continuity, including aspect ratio for: Based on normal grate bar samples, thresholds for connected region area, aspect ratio, target quantity, and adjacent spacing are determined. When the connected region area is less than the preset connected region area threshold, the aspect ratio is less than the preset aspect ratio threshold, the area corresponding to a single grate bar is split into multiple separate connected regions, or the skeleton is discontinuous in the middle, the corresponding grate bar is determined to have a break or severe occlusion and is an anomaly requiring review.

[0067] When there is no connected region that meets the area and aspect ratio requirements at the predetermined grate position, or when the total number of valid connected regions of the grate in the detection area is less than the lower limit of the normal number, it is determined that there is an abnormality of missing or detached grate; when the distance between adjacent connected regions deviates from the normal distance range or the main axis direction of the connected region deviates from the normal direction range, it is determined that there is a risk of grate loosening, twisting or abnormal deformation.

[0068] Step S8: Merge the multi-branch diagnostic results and output abnormal information.

[0069] The results of wheel appearance defect classification, wheel trajectory deviation, relative deviation of adjacent wheel spacing, geometric parameters of grate connected regions, and statistical results of grate quantity are fused to generate a multi-component status record for the trolley. The status record includes at least the trolley or image number, component type, anomaly category, anomaly location, detection confidence level, and acquisition time.

[0070] When any abnormal indicator meets the corresponding alarm condition, the edge computing node on the workshop side outputs an audible and visual alarm, interface mark, or communication message, and sends an image with mask outline and abnormal location mark to the upper monitoring system so that maintenance personnel can review and arrange maintenance.

[0071] In one implementation, the wheel trajectory deviation threshold, adjacent spacing deviation threshold, grate area threshold, aspect ratio threshold, and effective quantity threshold are all obtained statistically from normal operation samples and can be configured according to the camera installation location, trolley model, and operating speed. The aforementioned thresholds are expressed as preset thresholds and are not limited to a fixed value.

[0072] like Figure 1 As shown, in this embodiment, industrial cameras are deployed near the sintering machine trolley's running path, with the camera's field of view covering the area where the wheels pass or the visible area of ​​the trolley's grate bars. For wheel detection, the camera can be positioned below the side of the trolley, so that the wheel flange, tread, and the area near the track are within the field of view; for grate bar detection, the camera can be positioned at the unloading end, the return section, or other locations where an image of the grate bar array can be obtained. The images captured by the cameras are transmitted to the workshop-side edge computing node via an industrial network.

[0073] Edge computing nodes invoke the corresponding detection configuration based on the camera number. This detection configuration includes image cropping region, input size, wheel reference trajectory, wheel reference spacing, normal range of grating quantity, connected component geometric threshold, and continuous frame confirmation parameters. Configurations corresponding to different camera viewpoints can be calibrated separately to reduce the impact of perspective changes on geometric diagnostic results.

[0074] like Figure 2 As shown, the image to be detected first enters the MobileNetV2 lightweight backbone network. The backbone network outputs feature maps with different spatial resolutions at multiple stages, and the lightweight feature pyramid upsamples high-level features and then merges them laterally with low-level features. For grating detection, a feature layer with higher spatial resolution is retained, so that the slender grates and adjacent gaps still have recognizable pixel responses after multiple feature transformations; for wheel detection, medium and low resolution features are used to obtain a larger receptive field to identify the overall outline of the wheel and its relationship with the vehicle body and track.

[0075] The semantic insertion module extracts global semantics from low-resolution features, aligns them to the local feature size after channel transformation and upsampling, and generates spatial-channel joint weights. These weights modulate the local features before fusing them with the aligned global semantics. As a result, in densely packed grate areas, elongated regions belonging to the grate body receive a sustained response, while narrow gaps between adjacent grate bars maintain a low response; in dusty, accumulated, or shaded areas, localized high responses lacking global semantic support are suppressed.

[0076] During the training phase, wheel instance masks and grate binary masks can be used as supervision information. The loss function of the segmentation network can include pixel classification loss and region overlap loss; the losses of each scale prediction branch are weighted and summed according to preset weights. The second-level ResNet classification network is trained using image patches constrained by the wheel masks, outputting categories such as normal, tread peeling, and flange damage. The form and weights of the loss function here can be set according to the number of samples and the category distribution, and are not limited to specific values.

Claims

1. A method for diagnosing anomalies in multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation, characterized in that, Includes the following steps: Step S1: Acquire images of multiple components of the sintering machine trolley and establish the detection area; Step S2: Extract multi-scale component features using a lightweight backbone network; Step S3: Perform multi-scale semantic enhancement fusion and construct scale-aware mask prediction; Step S4: Perform cascaded classification of appearance defects based on the wheel mask; Step S5: Determine positional offset and detachment anomalies based on the centroid of the wheel mask and the adjacent spacing; Step S6: Perform mathematical morphological correction on the binary mask of the grating; Step S7: Determine whether the grate bars are broken, missing, or abnormally deformed based on the geometric parameters of the connected components; Step S8: Merge the multi-branch diagnostic results and output abnormal information.

2. The method for diagnosing anomalies in multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation according to claim 1, characterized in that, The specific steps of S1 are as follows: Images of the sintering machine trolley are captured by industrial cameras installed on the side, bottom, or near the unloading end of the trolley's running channel. Based on the camera installation position and the trolley's running direction, wheel detection areas and grate detection areas are determined in the trolley images. The captured trolley images are normalized in size, and brightness normalization, noise reduction, or contrast adjustment are performed as needed to form the image to be detected. When establishing training samples, the wheels and grates in the images to be detected are first annotated with pixel-level masks. Then, for each individual wheel instance, the categories of normal, tread peeling, flange damage, or other appearance defects are further labeled. After the sample annotation is completed, the samples are divided into training set, validation set, and test set for training the wheel and grate segmentation network and the wheel appearance defect classification network.

3. The method for diagnosing anomalies in multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation according to claim 1, characterized in that, The specific steps of S2 are as follows: The labeled image to be detected is input into a lightweight backbone network built on the MobileNetV2 inverse residual module. Multi-scale features are extracted at different downsampling rates to obtain high-resolution local features. Medium resolution structural features Low-resolution global semantic features The inverse residual module includes a main convolutional branch and a shortening branch. The main convolutional branch sequentially includes channel expansion convolution, depthwise separable convolution, and linear projection convolution. Channel expansion convolution is used to increase the number of channels for intermediate features, depthwise separable convolution is used to extract local spatial features, and linear projection convolution is used to map intermediate features to a preset number of output channels. The shortening branch is a connection path that bypasses the channel expansion convolution, depthwise separable convolution, and linear projection convolution, directly transmitting the input features of the inverse residual module to the module's output. Different scale features output by the lightweight backbone network , , Input a lightweight feature pyramid; the lightweight feature pyramid includes low-resolution global semantic features. To high-resolution local features The top-down path of hierarchical propagation, and the lateral connection path connecting the lightweight feature pyramid to the corresponding scale features of the lightweight backbone network; firstly, low-resolution global semantic features... Low-resolution fusion features are obtained by adjusting the channels. Then on Upsampling is performed, and then compared with the medium-resolution structural features after channel adjustment. Channel fusion is performed to obtain medium-resolution fusion features. ; and then High-resolution local features after channel adjustment Channel fusion is performed to obtain high-resolution fused features. .

4. The method for diagnosing anomalies in multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation according to claim 1, characterized in that, The specific steps of step S3 are as follows: Step 3.1: Construct the semantic insertion module: The fusion features obtained based on step S2 are: and low-resolution global semantic features , Low-resolution fusion features Medium resolution fusion features or high-resolution fusion features First, Perform size alignment to obtain Having the same spatial dimensions ;according to Generate semantic weights and utilize the semantic weights Fusion features The feature responses of each spatial location are weighted element-wise, and the weighted features are then compared with... Adding them together yields semantically enhanced features. : Where Resize represents the size alignment operation, Conv represents the convolution transformation, and Size represents the fused features. The spatial dimensions are σ, which represents the Sigmoid function, and ⊙ represents element-wise multiplication. Step 3.2, Feature Fusion After obtaining semantic enhancement features at different scales, these semantic enhancement features are again fused using a lightweight feature pyramid in step S2. Specifically, this involves: low-resolution semantic enhancement features... Channel adjustment yields semantically enhanced fusion features ; Enhance features with low-resolution semantics Based on this, upsampling is performed and combined with adjacent high-resolution semantic enhancement features. By fusing the features, semantically enhanced fusion features are obtained. Semantic enhancement fusion features Upsampling and combining with high-resolution semantic enhancement features By fusing the features, semantically enhanced fusion features are obtained. ; Step 3.3, Scale-Aware Mask Prediction The multi-scale semantic enhancement fusion features are input into the corresponding mask prediction branch to generate wheel instance masks and grating binary masks, respectively.

5. The method for diagnosing anomalies in multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation according to claim 1, characterized in that, The specific steps of S4 are as follows: Extract the wheel outline and its minimum bounding region from the wheel instance mask, set the background region outside the mask to zero or reduce the corresponding pixel weight to obtain a wheel image block constrained by the mask; input the wheel image block into the wheel appearance defect classification network, and output the classification results and corresponding confidence scores for wheel normal, tread peeling, wheel flange damage and other preset defect categories.

6. The method for diagnosing anomalies in multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation according to claim 1, characterized in that, The specific steps of step S5 are as follows: Mask for each wheel instance Calculate the geometric centroid coordinates ( , );when When (x, y) is the value of the wheel instance mask at pixel position (x, y), the centroid coordinates are calculated as follows: A reference running trajectory is established based on the wheel center of mass samples of the normally operating trolley; the reference running trajectory is a straight line, a piecewise straight line, or a fitted curve; For a straight line The local reference trajectory represents the vertical deviation of the i-th wheel's center of mass from the reference trajectory. for: When the reference trajectory is a segmented straight line, calculate the shortest distance from the wheel's center of mass to each reference line segment, and take the minimum value as the vertical deviation from the wheel's center of mass to the reference trajectory. ; When the reference trajectory is a fitted curve, the reference curve is obtained by using polynomial fitting or spline fitting based on multiple wheel centroid samples under normal operating conditions. Calculate the Euclidean distance between the center of mass of the wheel to be detected and each candidate point on the reference curve. : The minimum Euclidean distance is taken as the vertical deviation from the wheel's center of mass to the reference trajectory. ; when When the deviation exceeds the preset trajectory deviation threshold, the corresponding wheel is deemed to have a risk of positional deviation; for curved reference trajectories, the distance from the wheel's center of mass to the nearest point on the reference trajectory is used as the trajectory deviation. Sort the centroids of the wheels on the same or consecutive trolleys according to their direction of travel, and calculate the Euclidean distance between the centroids of two adjacent wheels. The reference spacing is determined based on the normal trolley structural dimensions or normal sample statistical values. And calculate the relative deviation of adjacent spacing. : when If the deviation exceeds the preset spacing threshold, or if no valid wheel instance mask is generated at the predetermined wheel position, it is determined that there is wheel misalignment, detachment, or missed detection anomaly requiring verification. To reduce false alarms caused by occlusion, confirmation can be made by combining detection results from multiple consecutive frames; when the same position continuously meets the abnormal conditions within a preset number of consecutive frames, an abnormal wheel position alarm will be output.

7. The method for diagnosing anomalies in multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation according to claim 1, characterized in that, The specific steps of step S6 are as follows: The initial grate binary mask output from the scale-aware mask prediction branch in step S3 is denoted as... To address the scattered noise and mask holes caused by airborne dust, accumulated material obstruction, and reflection, structural element B, adapted to the extension direction of the grate bars, is selected. The opening and closing operations are performed sequentially; the opening operation is used to remove discrete small regions and narrow pseudo-connections, and the closing operation is used to fill in the internal holes and short-distance breakpoints of the grating bars. in, This represents the erosion operation. This represents the expansion operation. Indicates to The result of performing the opening operation, Indicates to The result of the closing operation; structuring element B is set according to the width, extension direction and camera view of the grating in the image; for images with obvious perspective changes, structuring elements of different sizes or orientations are used in different detection sub-regions.

8. The method for diagnosing anomalies in multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation according to claim 1, characterized in that, The specific steps of step S7 are as follows: Corrected grating mask Perform connected component labeling to obtain the set of connected components. For each connected component Calculate the area , Width of the outer rectangle ,high Aspect Ratio And skeletal continuity, including aspect ratio for: Based on normal grate bar samples, determine the connected domain area threshold, aspect ratio threshold, target quantity threshold, and adjacent spacing threshold; when the connected domain area is less than the preset connected domain area threshold, the aspect ratio is less than the preset aspect ratio threshold, the area corresponding to a single grate bar is split into multiple mutually separated connected domains, or the skeleton is discontinuous in the middle, it is determined that the corresponding grate bar has a break or severe occlusion and needs to be verified as abnormal. When there is no connected region that meets the area and aspect ratio conditions at the predetermined grate position, or when the total number of valid grate connected regions in the detection area is less than the lower limit of the normal number, it is determined that there is a grate missing or detachment abnormality. When the spacing between adjacent connected domains deviates from the normal spacing range or the direction of the main axis of the connected domain deviates from the normal direction range, it is determined that there is a risk of loosening, twisting or abnormal deformation of the grate bars.

9. The method for diagnosing anomalies in multiple components of a sintering machine trolley based on scale-aware semantic enhancement segmentation according to claim 1, characterized in that, The specific steps of S8 are as follows: The wheel appearance defect classification results, wheel trajectory deviation, relative deviation of adjacent wheel spacing, geometric parameters of the grate connected domain, and grate quantity statistics are fused to generate a trolley multi-component status record; the status record includes at least the trolley or image number, component type, anomaly category, anomaly location, detection confidence level, and acquisition time; When any abnormal indicator meets the corresponding alarm condition, the edge computing node on the workshop side outputs an audible and visual alarm, interface mark, or communication message, and sends an image with mask outline and abnormal location mark to the upper monitoring system so that maintenance personnel can review and arrange maintenance.

Images