Method and system for detecting tooth breakage defect and foreign matter invasion of comb plate of escalator

Abstract

The invention discloses an escalator comb plate broken tooth defect and foreign matter invasion detection method and system, and the method comprises the steps: employing a trained detection model to recognize broken teeth and foreign matters, and employing the detection model as a frame, and inputting the small target fusion features output by the FPN structure, the low-layer local features extracted by the Conv module in the backbone network, the high-resolution shallow-layer features extracted by the PinwhelConv module, and the fused features into a detection head for small target detection. According to the method, the geometric feature capturing capability of the broken tooth profile is remarkably improved, and the small target detection effect is effectively improved.

Description

Technical Field

[0001] The invention relates to the field of escalator abnormality monitoring, in particular to a method for detecting broken teeth defects and foreign body intrusion of an escalator comb plate. Background Art

[0002] With the acceleration of urbanization, escalators, as an essential component of modern transportation systems, are widely used in shopping malls, subway stations, high-rise buildings, and other places. Accidental dropping of objects onto the steps by passengers is a common occurrence. Smaller objects can be carried to the front edge of the comb plate during the escalator's operation. In serious cases, this can cause the escalator to stop suddenly and injure passengers. If foreign objects remain stuck in the comb plate for a long time, they can break the teeth, causing them to break. This can lead to foreign objects on the steps entering the escalator's drive motor, damaging the internal mechanical structure and affecting its normal operation.

[0003] Detecting broken teeth and foreign objects in escalator comb plates is a key technology in the field of special equipment safety monitoring. Existing technologies fall into three main categories: physical sensor-based detection methods, visual recognition methods based on traditional image processing, and deep learning-based detection methods. Among physical sensor-based detection methods, existing devices use mechanical sensors or photoelectric switches to detect foreign objects. For example, metal detectors identify magnetic metal foreign objects, and infrared sensors detect foreign object obstruction. Additionally, pressure sensors are used to detect deformation in the comb plate structure, indirectly identifying broken teeth. However, these methods can only detect specific types of foreign objects and are susceptible to electromagnetic interference and mechanical vibration, resulting in high false positive rates. With the advancement of computer vision technology, visual recognition methods based on traditional image processing are increasingly being used. Traditional algorithms extract comb plate regional features through edge detection, morphological operations, or template matching, and then use threshold segmentation to determine foreign objects or broken teeth. For example, image difference methods locate abnormal areas by comparing a standard comb plate with a real-time image. However, these methods rely on manually designed features and have poor recognition capabilities for lighting changes, background interference, and small targets. They are also prone to missed or false detections under dynamic lighting or complex background conditions. Deep learning-based detection methods often face challenges such as insufficient detection accuracy and robustness, high real-time performance and deployment costs, and weak multi-target detection and dynamic tracking capabilities. They also lack the ability to extract features for small targets and those with diverse shapes. Summary of the Invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method and system for detecting broken tooth defects and foreign object intrusion in escalator comb plates, to solve the problem of low accuracy in identifying foreign objects with highly diverse shapes and types in the prior art, to improve the ability to identify small targets, dynamic targets and diverse targets, and to improve the accuracy and comprehensiveness of broken tooth defect and foreign object intrusion detection.

[0005] Technical solution: The method for detecting broken teeth defects and foreign body intrusion in escalator comb plates of the present invention comprises the following steps: constructing a sample data set, wherein the sample data set includes images and annotation information of broken teeth of comb plates, and images and annotation information of foreign bodies;

[0006] Training a detection model using the sample data set;

[0007] Acquire an image of the comb plate to be inspected, identify broken teeth and foreign objects using the trained detection model, and obtain the categories and bounding boxes of the broken teeth and / or foreign objects, which are used to calibrate the positions of the broken teeth and / or foreign objects in the image of the escalator comb plate;

[0008] The detection model adopts yolov10n as the framework, including a backbone network, a neck network and a detection head. The backbone network extracts shallow features of the input image through two convolutional layers, then extracts low-level local features through a C2f module and a convolutional layer, and then extracts low-level features, middle-level features and deep features based on the low-level local features; the low-level features, middle-level features and deep features are input into the neck network for feature fusion to obtain small target fusion features, middle target fusion features and large target fusion features; the neck network also includes an adaptive hierarchical feature fusion block for fusing small target fusion features, low-level local features and shallow features to obtain small target mixed fusion features; the small target mixed fusion features, middle target fusion features and large target fusion features are input into the detection head for prediction.

[0009] Furthermore, the backbone network extracts shallow features of the input image through two layers of PinwheelConv convolutional layers.

[0010] Furthermore, extracting low-level features, mid-level features, and deep-level features based on the low-level local features includes:

[0011] For low-level local features, the C2f module is used to extract low-level features; for low-level features, the mid-level features are extracted through downsampling and the C2f module; for mid-level features, deep features are extracted through downsampling, the C2fCIB module, the SPPELAN module, and the PSA module.

[0012] Furthermore, the method for acquiring the comb plate image to be detected includes: using a network camera to acquire a video of the escalator comb plate in real time, and extracting the comb plate image to be detected frame by frame from the video.

[0013] Furthermore, after obtaining the boundary frame of the broken tooth and / or foreign matter, the method further includes: triggering an alarm mechanism when the same category of results are recognized in several consecutive frames of comb plate images.

[0014] Furthermore, when constructing the sample data set, images containing broken teeth of the comb plate and images containing foreign objects are preprocessed; the comb plate images to be detected are preprocessed, and the preprocessing includes: removing images with too high similarity, flipping, rotating, noising and / or blurring the images.

[0015] The system for detecting broken teeth defects and foreign matter intrusion in escalator comb plates of the present invention comprises:

[0016] A model construction unit is used to establish a detection model. The detection model adopts yolov10n as a framework, including a backbone network, a neck network and a detection head. The backbone network extracts shallow features of the input image through two convolutional layers, and then extracts low-level local features through a C2f module and a convolutional layer. Then, low-level features, middle-level features and deep features are extracted based on the low-level local features; the low-level features, middle-level features and deep features are input into the neck network for feature fusion to obtain small target fusion features, middle target fusion features and large target fusion features; the neck network also includes an adaptive hierarchical feature fusion block for fusing small target fusion features, low-level local features and shallow features to obtain small target mixed fusion features; the small target mixed fusion features, middle target fusion features and large target fusion features are input into the detection head for prediction;

[0017] A model training unit is used to construct a sample data set, wherein the sample data set includes images and annotation information of broken teeth of a comb plate, and images and annotation information of foreign objects; and train a detection model using the sample data set;

[0018] The broken tooth defect and foreign object intrusion detection unit is used to collect the comb plate image to be detected, use the trained detection model to identify the broken teeth and foreign objects, obtain the category and bounding box of the broken teeth and / or foreign objects, and calibrate the position of the broken teeth and / or foreign objects in the escalator comb plate image.

[0019] The electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the computer program is loaded into the processor, the method for detecting broken tooth defects and foreign body intrusion in the escalator comb plate is implemented.

[0020] The computer-readable storage medium of the present invention stores a computer program, and when the computer program is executed by a processor, the method for detecting broken tooth defects and foreign body intrusion in an escalator comb plate is implemented.

[0021] Beneficial effect: Compared with the existing technology, the advantages of the present invention are: the present invention adopts the YOLOv10n-AE model to detect broken tooth defects and foreign object intrusion of escalator comb plates. The model improves the receptive field through the PinwheelConv module, effectively improving the feature extraction effect of small targets in complex environments; the cross-level feature enhancement channel constructed by the SPPELAN module significantly improves the ability to capture the geometric features of broken tooth contours; the feature fusion strategy of the neck network improved by the HFFB module effectively improves the detection effect of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is the network structure diagram of the YOLOv10n model.

[0023] Figure 2 This is the network structure diagram of the YOLOv10n-AE model.

[0024] Figure 3 This is the schematic diagram of the PinwheelConv module.

[0025] Figure 4 This is the network structure diagram of the SPPF module in the YOLOv10n backbone network.

[0026] Figure 5 This is the network diagram of the SPPELAN module.

[0027] Figure 6 This is the schematic diagram of the HFFB module.

[0028] Figure 7 Schematic diagram of the escalator comb plate broken teeth and foreign object detection device.

[0029] Figure 8 This is a comparison chart of the accuracy P of the YOLOv10n model and the YOLOv10n-AE model in an embodiment of the present invention.

[0030] Figure 9 This is a comparison chart of the recall rates R of the YOLOv10n model and the YOLOv10n-AE model in an embodiment of the present invention.

[0031] Figure 10 This is a comparison chart of the average accuracy mAP50 of the YOLOv10n model and the YOLOv10n-AE model in an embodiment of the present invention.

[0032] Figure 11 This is a comparison chart of the average accuracy mAP50:95 of the YOLOv10n model and the YOLOv10n-AE model in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0034] YOLOv10 is a real-time, end-to-end, single-stage target detection algorithm proposed by a team from Tsinghua University. It aims to address the traditional YOLO series' reliance on non-maximum suppression (NMS) and computational redundancy in the model architecture, thereby achieving a better balance between detection accuracy and inference efficiency. The traditional YOLO model uses a one-to-many label assignment strategy to improve training results, but this method relies on NMS post-processing to eliminate redundant predictions, resulting in increased inference latency and sensitivity to hyperparameters, limiting the feasibility of end-to-end deployment. YOLOv10 achieves efficient detection without NMS by introducing a dual-label assignment strategy and an optimized design that balances efficiency and accuracy, thereby reducing computational complexity. YOLOv10n, a lightweight version of the YOLOv10 series, is designed for resource-constrained mobile devices and edge computing platforms.

[0035] like Figure 1 As shown in the figure, the network structure of YOLOv10 mainly consists of three parts: backbone network (Backbone), neck network (Neck) and detection head (Head).

[0036] As the core module for feature extraction, the backbone network in YOLOv10 uses an improved Cross-Stage Partial Network (CSPNet) to extract multi-level feature maps. Building on the C2f and SPPF modules of YOLOv8, YOLOv10 introduces the spatial-channel decoupled downsampling module (SCDown). By concatenating point-by-point convolution with depth-wise convolution, it achieves spatial and channel-wise downsampling, significantly reducing computational complexity. In addition, YOLOv10 designs a compact inverted block (C2fCIB) structure that uses depth-wise convolution for spatial mixing and point-by-point convolution for channel mixing to further optimize computational efficiency. To address the redundancy problem in traditional attention mechanisms, YOLOv10 also proposes an efficient partial self-attention mechanism (PSA). This mechanism feeds partial features into an NPSA module consisting of a multi-head self-attention module (MHSA) and a feed-forward network (FFN), and fuses the two features, thereby enhancing global representation learning capabilities at a lower computational cost.

[0037] The neck network (Neck), a key module for feature fusion, is responsible for integrating and optimizing the multi-scale features extracted by the backbone network. YOLOv10 has significantly optimized the neck structure, primarily adopting the FPN-PAN architecture to enhance the flow of information between features of different scales. The FPN propagates high-level semantic information to low-level features through a bottom-up feature fusion path, thereby improving the detection capability of small targets. Simultaneously, the PAN, through a top-down feature enhancement path, enables lower-level detail features to be better integrated with higher-level semantic features, further improving target positioning accuracy. This multi-scale feature fusion mechanism effectively improves the model's detection performance in complex scenarios.

[0038] In the detection head, YOLOv10n adopts a consistent dual label assignment strategy. This strategy uses a one-to-one label assignment mechanism during training to ensure that each target is matched with only one predicted box, avoiding redundancy caused by multiple matches and significantly improving label assignment accuracy and detection precision. Furthermore, a one-to-many matching strategy allows a single target to be matched with multiple predicted boxes, providing the backbone network with richer supervisory signals and enhancing the model's learning ability and robustness in complex scenes and small target detection. During the inference phase, the model retains only the one-to-one matching mechanism, further optimizing computational efficiency.

[0039] There are many limitations when using YOLOv10n to detect broken tooth defects and foreign objects in escalator comb plates. First, the image data of the comb plate is usually collected by a camera above it. Due to the long distance between the camera and the comb plate, the broken tooth defects appear very small in the image, making it difficult to effectively identify using traditional image processing methods. Second, the steps of the escalator are constantly moving during operation, which causes the position and shape of the broken tooth defects in the image to constantly change, further increasing the difficulty of identification. In addition, the shapes and types of foreign objects are highly diverse. Due to the lightweight design of the YOLOv10n model, its detection accuracy in complex scenarios has decreased, especially in the detection of small and dense targets. It performs poorly and cannot fully cover all possible defect types. In addition, its dynamic feature fusion mechanism and the simplification of the attention module also limit the model's adaptability to multi-scale targets.

[0040] Without increasing the number of model parameters, the present invention improves YOLOv10n to obtain YOLOv10n-AE, which has a higher recognition accuracy rate for broken teeth defects of comb plates and foreign body intrusion targets. Figure 2 Shown is the architecture diagram of YOLOv10n-AE.

[0041] The input image enters the backbone network, where it undergoes two layers of pinwheel convolution to output shallow image features. The C2f module fuses these shallow, high-resolution features from different layers and then feeds them back into the convolutional layer for low-level feature extraction to generate low-level local features. The C2f module then fuses these low-level features to generate low-level features, enhancing the recognition of low-level objects. The low-level features are then fed into the SCDown downsampling layer for compression and semantic enhancement. After downsampling, the features are again fed into the C2f module to extract mid-level features, supporting medium-level object detection and classification. The C2fCIB module is a further optimized version of the C2f module, processing the deep, low-resolution feature maps from the previous downsampling layer. It enhances semantic understanding of large and densely packed objects through cross-stage interactions. The SPPELAN pooling layer module, introduced at the end of the backbone network, aims to increase the receptive field of the feature map while preserving multi-scale contextual information, thereby improving the model's detection capabilities for objects of varying sizes. Subsequently, the PSA attention module adaptively adjusts feature responses through a multi-channel attention mechanism to generate deep features, further enhancing the model's ability to focus on target regions.

[0042] The features extracted by the backbone network are input into the neck network for feature fusion. In the neck structure, the model adopts a bidirectional feature fusion path of FPN and PAN. The FPN part follows a bottom-up path. The deep features extracted by the backbone network are input into the Upsample of FPN. The scale of the deep features is kept consistent with the mid-level features. The upsampled high-level features are then concatenated with the mid-level features of the backbone network through Concat splicing. The spliced features are input into the C2f module for extraction and fusion. Similarly, the fused features undergo the same upsampling, concatenation, and feature fusion to obtain the small target fusion features, which are then input into the HFFB module. The PAN part follows a top-down path. First, the small target fusion features are subjected to the convolution operation Conv to adjust the number of channels to keep them consistent with the mid-level features in FPN. The two are then concatenated and the mid-level target fusion features are obtained through feature fusion after splicing. These features are then output to the detection head for mid-level target recognition. The medium target fusion features are downsampled SCDown to reduce the resolution by half, keeping the scale consistent with the deep features output in the backbone network, and then splicing and fusion are used to obtain the large target fusion features.

[0043] The HFFB module fuses three types of feature information: small target fusion features from the FPN structure output, low-level local features extracted by the Conv module in the backbone network, and high-resolution shallow features extracted by the PinwheelConv module. The resulting fusion is called a small target hybrid fusion feature. This multi-level feature fusion enhances the recognition and spatial positioning accuracy of small targets in complex backgrounds, effectively improving the model's detection performance for small targets in occluded and densely populated scenes.

[0044] Finally, the neck network outputs feature maps of three scales (large target fusion features, medium target fusion features, and small target mixed fusion features) to the detection head, which are used for the detection of large, medium, and small targets respectively.

[0045] The following is a detailed description of the three modules in YOLOv10n-AE.

[0046] (1) PinwheelConv

[0047] The present invention uses the pinwheel convolution PinwheelConv module to optimize the traditional convolution module in the YOLOv10n model. Figure 3 The following is a network structure diagram of the PinwheelConv module. Unlike standard convolution, it uses asymmetric padding to create horizontal and vertical convolution kernels for different areas of the image, and the convolution kernels spread outward. To enhance training stability and speed, batch normalization (BN) and sigmoid linear units (SiLU) are applied after each convolution. PinwheelConv asymmetrically pads the input tensor X(h1,w1,c1) with pixel values, where p(a,b,c,d) represents the number of rows of pixel values padded to the left, right, top, and bottom, respectively. Parallel convolution is applied to the padded tensor, and the calculation formula is as follows:

[0048]

[0049] in: Indicates the execution of convolution operation; It is a 1×3 convolution kernel with an output channel of c′. After the interleaved convolution, the height (h′), width (w′), and number of channels (c′) of the output feature map satisfy the following relationship with the input feature map:

[0050]

[0051] Where c2 is the number of channels of the final output feature map of PinwheelConv, and s is the step size of the convolution operation. The results of the interleaved convolution operation are concatenated (Concat), and the concatenated tensor is passed through the convolution kernel Normalize. The final output result The calculation is as follows:

[0052]

[0053] The number of convolutions in PinwheelConv decreases from the center outward, similar to Gaussian distribution. Similarly, grouped convolution is used to significantly increase the receptive field while minimizing the number of parameters. The parameter calculation formulas for standard convolution (Conv) and pinwheel convolution (PConv) are as follows:

[0054] Conv para =c1×c2×k

[0055] PConv par =4×((c2 / 4)×c1×3×1)+4c1c2

[0056] Where k represents the size of the convolution kernel. If the output channel c2 is equal to the input channel c1, the 3×3 Conv module has 9c1c2 parameters, while the 3×3 PinwheelConv has 7c1c2 parameters and a receptive field of 25. In comparison, PinwheelConv reduces the number of parameters by 22.2% and increases the receptive field by 177%. Replacing the first two Conv layers of the YOLOv10n backbone network with the PinwheelConv module allows for efficient receptive field expansion with minimal parameter increase when extracting underlying features.

[0057] (2) Spatial Pyramid Pooling and Efficient Layer Aggregation Network (SPPELAN)

[0058] Spatial Pyramid Pooling with Efficient Layer Aggregation Network (SPPELAN) is an improved module that is mainly used to solve the multi-scale problem in target detection tasks and is optimized for small target detection. Compared with the traditional SPPF (Separable Pyramid Pooling with Fusion), SPPELAN performs better in processing small targets. Figure 4 As shown in the figure, in the SPPF in the YOLOv10n backbone network, the first pooling channel of the input feature map undergoes convolution, batch normalization, and activation functions to maintain the original size of the feature map. The second pooling channel uses a 5×5 pooling kernel; the third pooling channel uses two 5×5 pooling kernels; and the fourth pooling channel uses three 5×5 pooling kernels. Ultimately, these pooling results are stacked together, enhancing the model's ability to capture objects of different scales. The SPPF increases the receptive field through multi-scale pooling and fusion operations, improving the model's ability to capture multi-scale features. However, this may be insufficient for handling small objects. Due to the small size of the comb plate's teeth, the large kernel convolution in the SPPF may not provide sufficient spatial resolution, resulting in the dilution of small object features during the pooling process, which affects detection accuracy.

[0059] By combining the attention mechanism of spatial pyramid pooling (SPP) and efficient layer aggregation network (ELAN), SPPELAN can extract and aggregate features at different scales, enhancing the model's ability to detect small targets. Figure 5As shown in the figure, ELAN's efficient layer aggregation strategy maintains high spatial resolution by pooling and aggregating features at multiple scales, avoiding the performance degradation of the traditional SPPF method in small object detection. Furthermore, while maintaining high efficiency, SPPELAN reduces the number of model parameters and improves computational efficiency by adopting a cascade of maximum pooling layers. This structural optimization enables SPPELAN to achieve higher accuracy in small object detection tasks while effectively reducing computational overhead, making it suitable for small object detection tasks in complex environments. Therefore, SPPELAN not only addresses the shortcomings of SPPF in small object detection, but also further improves the performance and efficiency of the model.

[0060] (3) Adaptive Hierarchical Feature Fusion Block (HFFB)

[0061] In the multi-scale target detection task, Feature Pyramid Network (FPN) is a classic feature fusion method and is widely used in deep learning target detection models. FPN realizes the fusion of features at different levels through a top-down path structure, thereby enhancing the detector's ability to recognize targets of different scales. However, the inherent limitations of its structure limit the further improvement of detection performance. Specifically, the traditional FPN has low efficiency in feature information transmission. Especially in deep neural networks, fine-grained information is easily lost during the layer-by-layer transmission process. Especially for small target detection tasks, the attenuation of key information may lead to a decrease in the model's perception of small-scale targets. To address the above problems, this paper introduces an adaptive hierarchical feature fusion block (HFFB), which can adaptively fuse local features, global features and semantic information fused at the previous level according to the different levels of input features, thereby improving the multi-scale feature expression capability. As Figure 6 As shown in the figure, the network structure of HFFB consists of key components such as spatial attention (SA), channel attention (CA), residual inversion MLP (IRMLP) and shortcut connection.

[0062] In the HFFB module, the input global features are first processed by the channel attention (CA) mechanism. The CA mechanism leverages interdependencies between channels to enhance the representation of features with important semantic information. Specifically, the CA module performs global average pooling and global max pooling on the input features, transforming them using channel feature maps. The pooling results are then summed and passed through a sigmoid activation function to generate channel-level attention weights, which then adaptively adjust the input features to increase the model's focus on key channels.

[0063] CA(x)=σ(MLP(AvgPool(x)+MLP(MaxPool(x)))

[0064] On the other hand, local features are fed into the spatial attention (SA) mechanism to enhance spatial detail and suppress interference from background or irrelevant areas. The SA mechanism first performs maximum pooling and average pooling on the input feature maps in the channel dimension. The pooling results are then concatenated and a convolution operation with a 7×7 kernel is applied to generate spatial attention weights. Finally, after the input features are weighted by spatial attention, local detail is enhanced.

[0065] SA(x)=σ(f 7×7 (Concat[AvgPool(x),MaxPool(x)]))

[0066] During the multi-level feature fusion process, the output features of each attention path and the fusion path are further fused and nonlinearly transformed using the Inverted Residual MLP (IRMLP) module. IRMLP not only alleviates the problems of vanishing and exploding gradients, but its inverted residual structure, combined with lightweight MLP computation, helps improve the expressive power of feature transformations. Furthermore, the introduction of shortcut connections further enhances cross-layer feature transfer, ensuring smooth information flow within deep networks.

[0067] IRMLP(x)=f 1×1 (f 1×1 (f 3×3 (LN(x))+LN(x)))

[0068] Specifically, in HFFB, global features are obtained through the CA attention mechanism SA attention mechanism acquires local features Downsample the input to generate features Obtain global-local and previous stage fusion features through splicing and convolution calculation Finally, the fusion features local features Global Features After splicing, it is input to the IRMLP module and combined with Generate the final output feature F i The specific calculation process is as follows:

[0069]

[0070] The introduction of HFFB effectively improves the performance of FPN in small target detection tasks. Through the adaptive feature fusion strategy, it improves the robustness and accuracy of multi-scale target detection, while achieving better feature expression capabilities while keeping the computational complexity controllable.

[0071] To improve the accuracy and generalization of the object detection model in various operational scenarios, particularly for the potential issues of broken teeth and foreign matter intrusion in escalator comb plates, systematic image acquisition and experimental verification are essential. The following is a detailed experiment illustrating the method described in this invention.

[0072] like Figure 7 As shown, the network camera used in this embodiment ensures high-quality acquisition of escalator comb plate images. In actual application, the camera is connected to a switch or router via a network interface, and a computer is connected to the system via a local area network to monitor and view the image data captured by the camera in real time.

[0073] This experimental process was designed to realistically replicate comb plate breakage and foreign object intrusion by simulating various potential failure scenarios that may occur during escalator operation. Specifically, to simulate tooth breakage, the experiment used comb plates that had undergone actual wear and damage and installed them on an escalator to replicate the potential for tooth breakage. Furthermore, the experiment simulated foreign object intrusion, which is common in rail transit systems, by placing foreign objects of varying sizes, such as screws, coins, boarding passes, and keys, at the leading edge of the comb plates. These experiments were conducted after the escalator was in operation, with a webcam continuously recording and capturing image and video data during actual escalator operation, comprehensively documenting the dynamic behavior of comb plate breakage and foreign object intrusion. This experimental setup effectively simulates and validates the model's performance in real-world environments, further improving the recognition accuracy and robustness of the object detection system in complex scenarios. The experiments used comb plates that had undergone actual wear and damage and installed them on an escalator to replicate the potential for tooth breakage.

[0074] In terms of image processing, to ensure the diversity, breadth, and completeness of the dataset, frames were first extracted from the recorded images and filtered by removing images with high similarity to reduce data redundancy. Subsequently, the image data was further processed using various image enhancement algorithms (such as image flipping, rotation, noise addition, and blurring). This process aims to artificially increase image variability and simulate anomalies that may occur in different environments, thereby improving the adaptability and robustness of the object detection model and ensuring its stable operation in various practical application scenarios. Furthermore, images containing anomalies such as broken escalator comb plates and objects stuck in the comb plates were collected and acquired through web browsers and other means to expand the original dataset and further enhance its representativeness. For accurate labeling, the extracted images were manually annotated using the Labelimg tool, with the main annotated categories being "broken tooth" (flaw) and "foreign object" (yiwu). Each annotated image was then generated into a corresponding text file (txt). The image data was then divided into training, validation, and test sets in an 8:1:1 ratio. At the same time, a corresponding YAML configuration file is generated to facilitate parameter configuration for subsequent model training. During the training process, the dataset is imported into the YOLOv10n-AE model for training. Specific hyperparameter settings include: input image resolution of , initial learning rate set to 0.01, batch size of 32, and total number of training iterations (epochs) of 300. After training, the model generates two model files: the optimal model weight parameters best.pt and the final model weight parameters last.pt. In addition, some important parameters and training results from the training process are saved. These model files and parameters are the basis for subsequent object detection inference tasks.

[0075] In actual application, by calling the trained model and executing the inference code, the model will perform object detection on the input image. During the detection process, a bounding box is output for each detected object, which is used to demarcate the object's location in the image. The bounding box is typically represented by four coordinates: the center coordinates (x_center and y_center) of the box, and its width (width) and height (height). In addition, for each detected object, the model outputs a class label, indicating the object's category. For example, if a broken tooth defect is detected on a comb plate, the class label is "flaw"; if a foreign object is detected on the comb plate, the class label is "yiwu". Each object is also assigned a confidence score, indicating the model's confidence in the predicted category. Confidence scores typically range from 0 to 1, with higher scores indicating greater confidence in the model's predicted category. To further optimize detection performance, a confidence threshold can be set. When the model's confidence is low, object detection boxes may be filtered out, thereby reducing false positives. This mechanism ensures that the system accurately responds to real abnormal events while preventing false alarms from interfering with subsequent processing.

[0076] To effectively apply object detection models to rail transit systems, particularly in equipment like escalators, this paper proposes a solution that deploys the YOLOv10n-AE model and its related algorithms on edge computing units. This design not only meets the stringent real-time, low-latency, and high-performance requirements of real-world environments, but also fully leverages the significant advantages of edge computing for data processing and inference tasks. The core advantage of edge computing is that data processing can be performed locally on the device, eliminating the need to transmit large amounts of data to remote servers, effectively avoiding the problems caused by network latency and bandwidth bottlenecks. In rail transit systems, the operational scenarios of equipment like escalators place extremely high demands on the system's processing power and response speed. By deploying the object detection model on the edge device, image processing and inference tasks can be performed locally in real time, improving processing efficiency and ensuring the system's ability to rapidly respond to changes in device status. Specifically, the edge device captures high-definition video images of the escalator's comb plate in real time via a webcam and feeds these images into the object detection model for frame-by-frame analysis. The YOLOv10n-AE model accurately identifies broken teeth and foreign objects in each frame, providing real-time support for subsequent anomaly detection and resolution. When the model detects an anomaly (such as a broken tooth or foreign object), it immediately outputs the anomaly label and calibrates the coordinates of the target box to clearly identify the abnormal object in the image.

[0077] The hardware configuration for this experiment is as follows: an Intel Core i7-12700KF CPU at 3.6GHz, an NVIDIA GeForce RTX 4080 GPU with 16GB of video memory, and 32GB of host memory. The software environment is based on the Ubuntu 22.04 operating system, using Python 3.9 as the programming language, Pytorch 2.0.1 as the deep learning framework, and CUDA 12.7 for GPU acceleration. During training, the dataset was imported into the YOLOv10-AE model for training. The specific hyperparameter settings are as follows: input image resolution of 640×640, initial learning rate of 0.01, batch size of 16, and number of training iterations (epochs) of 200. To comprehensively evaluate the performance of the improved YOLOv10n algorithm, this study used the following evaluation metrics: For detection accuracy, precision (P), recall (R), and mean average precision (mAP) were used. mAP is further broken down into mAP50 (average accuracy across categories at an IoU threshold of 0.5) and mAP50:95 (average accuracy across categories at IoU thresholds ranging from 0.5 to 0.95, with a step size of 0.05). For model efficiency, the number of parameters and frames per second (FPS) were used to comprehensively measure the model's computational complexity and inference speed. The performance metrics are calculated using the following formulas.

[0078]

[0079] Where: TP represents the number of correctly predicted positive examples, FP represents the number of negative examples that are incorrectly predicted as positive examples, and FN represents the number of positive examples that are incorrectly judged as negative examples.

[0080] like Figure 8 、 Figure 9 、 Figure 10 and Figure 11The following plots show the iteration curves for P, R, mAP50, and mAP50:95 for the YOLOv10n and YOLOv10n-AE models. The training and test results for the YOLOv10n model are: P = 0.763, R = 0.688, mAP50 = 0.758, and mAP50:95 = 0.407. The optimized model's test results are: P = 0.774, R = 0.724, mAP50 = 0.784, and mAP50:95 = 0.435. The YOLOv10n-AE model achieved improvements of 1.1% in precision (P), 3.6% in recall (R), and 2.6% in average precision (mAP50), respectively. The more stringent mAP50:95 metric also saw a 2.8% improvement. This shows that the improvement in receptive field after the introduction of the PinwheelConv module effectively improves the feature extraction effect of small targets in complex environments; the cross-level feature enhancement channel constructed by the SPPELAN module significantly improves the ability to capture the geometric features of broken tooth contours; the feature fusion strategy of the neck network improved by the HFFB module effectively improves the detection effect of small targets.

[0081] The number of parameters of YOLOv10n is 2.7×10 6 , model size is 5.8MB, FPS is 317.7; the number of parameters of YOLOv10n-AE is 3.0×10 6 The model size is 6.5MB, and the FPS is 311. This slight increase in parameters achieves a comprehensive improvement in key accuracy metrics while still meeting industrial real-time detection standards. This verifies the effectiveness of the YOLOv10n-AE model in detecting broken teeth defects and foreign object intrusion in escalator comb plates in complex scenarios.

[0082] To optimize the system's response, detection results can be further analyzed and processed by setting rule-based algorithms. For example, if the object detection model identifies the same fault category for several consecutive frames, an alarm mechanism can be triggered to ensure that the system only responds to true abnormal events. First, the system records the fault image in real time and generates an image file containing the target box location and abnormality label. This image data is stored locally on the device, providing data support for subsequent fault analysis and tracing. The system then transmits the fault information, image data, and abnormality location to the management end in real time via a pre-set communication protocol. Through the remote monitoring platform, managers can view abnormal images in real time, quickly assess the equipment's operating status, and take appropriate action. This process not only improves the efficiency of abnormality handling but also optimizes the overall system's fault response mechanism, ensuring the safe and stable operation of escalator equipment in the rail transit system.

[0083] The system for detecting broken teeth defects and foreign matter intrusion in escalator comb plates of the present invention comprises:

[0084] A model construction unit is used to establish a detection model. The detection model adopts yolov10n as a framework, including a backbone network, a neck network and a detection head. The backbone network extracts shallow features of the input image through two convolutional layers, and then extracts low-level local features through a C2f module and a convolutional layer. Then, low-level features, middle-level features and deep features are extracted based on the low-level local features; the low-level features, middle-level features and deep features are input into the neck network for feature fusion to obtain small target fusion features, middle target fusion features and large target fusion features; the neck network also includes an adaptive hierarchical feature fusion block for fusing small target fusion features, low-level local features and shallow features to obtain small target mixed fusion features; the small target mixed fusion features, middle target fusion features and large target fusion features are input into the detection head for prediction;

[0085] A model training unit is used to construct a sample data set, wherein the sample data set includes images and annotation information of broken teeth of a comb plate, and images and annotation information of foreign objects; and train a detection model using the sample data set;

[0086] The broken tooth defect and foreign object intrusion detection unit is used to collect the comb plate image to be detected, use the trained detection model to identify the broken teeth and foreign objects, obtain the category and bounding box of the broken teeth and / or foreign objects, and calibrate the position of the broken teeth and / or foreign objects in the escalator comb plate image.

[0087] The electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the computer program is loaded into the processor, the method for detecting broken tooth defects and foreign body intrusion in the escalator comb plate is implemented.

[0088] The computer-readable storage medium of the present invention stores a computer program, and when the computer program is executed by a processor, the method for detecting broken tooth defects and foreign body intrusion in an escalator comb plate is implemented.

[0089] The computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store program code in the form of instructions or data structures and that can be accessed by a computer.

[0090] The processor is configured to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment.

Claims

1. A method for detecting broken teeth defects and foreign matter intrusion in escalator comb plates, characterized in that: The method comprises the following steps: constructing a sample data set, wherein the sample data set includes images and annotation information of broken teeth of a comb plate, and images and annotation information of foreign matter; Training a detection model using the sample data set; Acquire an image of the comb plate to be inspected, identify broken teeth and foreign objects using the trained detection model, and obtain the categories and bounding boxes of the broken teeth and / or foreign objects, which are used to calibrate the positions of the broken teeth and / or foreign objects in the image of the escalator comb plate; The detection model adopts yolov10n as the framework, including a backbone network, a neck network and a detection head. The backbone network extracts shallow features of the input image through two convolutional layers, then extracts low-level local features through a C2f module and a convolutional layer, and then extracts low-level features, middle-level features and deep features based on the low-level local features; the low-level features, middle-level features and deep features are input into the neck network for feature fusion to obtain small target fusion features, middle target fusion features and large target fusion features; the neck network also includes an adaptive hierarchical feature fusion block for fusing small target fusion features, low-level local features and shallow features to obtain small target mixed fusion features; the small target mixed fusion features, middle target fusion features and large target fusion features are input into the detection head for prediction.

2. The method for detecting broken teeth defects and foreign matter intrusion in escalator comb plates according to claim 1 is characterized in that: The backbone network extracts shallow features of the input image through two layers of PinwheelConv convolutional layers.

3. The method for detecting broken teeth defects and foreign matter intrusion in escalator comb plates according to claim 1 is characterized in that: The extracting of low-level features, mid-level features and deep-level features according to the low-level local features comprises: For low-level local features, the C2f module is used to extract low-level features; for low-level features, the mid-level features are extracted through downsampling and the C2f module; for mid-level features, deep features are extracted through downsampling, the C2fCIB module, the SPPELAN module, and the PSA module.

4. The method for detecting broken teeth defects and foreign matter intrusion in escalator comb plates according to claim 1 is characterized in that: The method for collecting the comb plate image to be detected includes: using a network camera to obtain a video of the escalator comb plate in real time, and extracting the comb plate image to be detected frame by frame from the video.

5. The method for detecting broken teeth defects and foreign matter intrusion in escalator comb plates according to claim 4 is characterized in that: After obtaining the boundary frame of the broken tooth and / or foreign matter, the method further includes: triggering an alarm mechanism when the same category of results are recognized in a plurality of consecutive frames of comb plate images.

6. The method for detecting broken teeth defects and foreign matter intrusion in escalator comb plates according to claim 1 is characterized in that: When constructing the sample data set, images containing broken comb plate teeth and images containing foreign objects are preprocessed; the comb plate images to be detected are preprocessed, and the preprocessing includes: removing images with high similarity, flipping, rotating, adding noise and / or blurring the images.

7. A detection system for broken teeth defects and foreign matter intrusion in escalator comb plates, characterized by: include: A model construction unit is used to establish a detection model. The detection model adopts yolov10n as a framework, including a backbone network, a neck network and a detection head. The backbone network extracts shallow features of the input image through two convolutional layers, and then extracts low-level local features through a C2f module and a convolutional layer. Then, low-level features, middle-level features and deep features are extracted based on the low-level local features; the low-level features, middle-level features and deep features are input into the neck network for feature fusion to obtain small target fusion features, middle target fusion features and large target fusion features; the neck network also includes an adaptive hierarchical feature fusion block for fusing small target fusion features, low-level local features and shallow features to obtain small target mixed fusion features; the small target mixed fusion features, middle target fusion features and large target fusion features are input into the detection head for prediction; A model training unit is used to construct a sample data set, wherein the sample data set includes images and annotation information of broken teeth of a comb plate, and images and annotation information of foreign objects; Training a detection model using the sample data set; The broken tooth defect and foreign object intrusion detection unit is used to collect the comb plate image to be detected, use the trained detection model to identify the broken teeth and foreign objects, obtain the category and bounding box of the broken teeth and / or foreign objects, and calibrate the position of the broken teeth and / or foreign objects in the escalator comb plate image.

8. The escalator comb plate broken tooth defect and foreign body intrusion detection system according to claim 7 is characterized in that: Model building unit, the backbone network extracts shallow features of the input image through two layers of PinwheelConv convolutional layers; The extraction of low-level features, mid-level features and deep-level features based on the low-level local features includes: extracting low-level features from low-level local features through the C2f module; extracting mid-level features from low-level features through downsampling and the C2f module; and extracting deep features from mid-level features through downsampling, the C2fCIB module, the SPPELAN module and the PSA module.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is loaded into the processor, the method for detecting broken teeth defects and foreign body intrusion of the escalator comb plate according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting broken teeth defects and foreign body intrusion in an escalator comb plate according to any one of claims 1 to 6 is implemented.

Images