Passenger car part anomaly detection method based on deep residual network

Abstract

The application discloses a kind of passenger car parts abnormality detection methods based on deep residual network, comprising: based on image processing technology and computer graphics remove the annotation in the full scene part abnormal image with abnormal annotation;Adopt target detection model to carry out abnormal part positioning and cutting to the full scene part abnormal image of deannotation, obtain abnormal part ROI image;Based on abnormal part ROI image and normal part image, construct sample data set;According to sample data set, train and evaluate the train part abnormality detection model based on deep residual network;When carrying out abnormal detection to train target part, obtain the image to be detected, and utilize the trained target detection model and train part abnormality detection model to carry out positioning, cutting and abnormal detection to train target part.The application improves the identification ability for small crack and other minimum target, and false alarm rate is greatly reduced.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of computer vision, deep learning and industrial intelligent inspection, and in particular relates to an anomaly detection method for bus parts based on deep residual networks. Background Technology

[0002] With the rapid development of the railway transportation industry and the continuous increase in passenger volume, the requirements for train operation safety are becoming increasingly stringent. Trains themselves have a complex structure, containing a large number of components, from the running gear and braking parts to various underframe suspension components, in vast quantities and diverse types. Statistics show that a typical passenger train contains nearly a hundred different types of components, from the wheels and bearings of the running gear to the brake discs and cylinders of the braking system, and then to the underframe suspension components. This vast system of components makes fault detection a demanding task. Furthermore, each component, due to different working environments and stress conditions, can experience a wide variety of faults. For example, wheels may experience wear, cracks, and tread peeling; brake discs may suffer from thermal cracks and uneven wear. This necessitates that the detection methods possess the ability to accurately identify multiple types of faults.

[0003] In the early stages of train component fault detection, the Support Vector Machine (SVM) algorithm combined with manual feature extraction methods was widely used. Its core idea was to find an optimal hyperplane in the data space to classify different categories of data. In train fault detection, features such as color, texture, and shape extracted manually from train component images were used, and SVM was employed to classify whether train components had faults. However, the SVM algorithm is inefficient for processing large-scale data. As the amount of training data increases, the computational complexity increases significantly, leading to a substantial increase in training time. Furthermore, the performance of SVM is highly dependent on the choice of kernel function and the setting of kernel function parameters. Different combinations of kernel functions and parameters have a significant impact on the detection results, and finding the optimal kernel function and parameters often requires extensive experimentation and debugging.

[0004] With the development of deep learning technology, many object detection algorithms have been applied to train component fault detection, such as Faster R-CNN and YOLO. Although Faster R-CNN improves detection accuracy, its inference speed is slow due to the region proposal step; and its model generalization ability and training flexibility are significantly limited when facing the continuous upgrading of nearly a hundred types of passenger car parts. Although YOLO has extremely high real-time inference speed, its direct application to industrial train fault detection exposes deep-seated architectural defects: on the one hand, directly compressing 8192×8192 pixel images into the YOLO network will cause minor defects to be completely erased by convolution and pooling operations; on the other hand, YOLO tends to extract global semantics, but its classifier's perceptual granularity is severely insufficient when analyzing texture anomalies in local micro-regions (such as micro-cracks).

[0005] The poor performance of existing technologies in actual industrial deployments can be attributed to the following three major technical challenges:

[0006] (1) The problem of strong noise and human interference in historical samples: Most of the existing train fault image databases are derived from the long-term manual judgment accumulation of front-line train inspectors. In order to identify faults, inspectors are accustomed to using high-contrast red and blue lines to select or mark digital images. During the training process of deep learning models, these unnatural features are preferentially identified as "salient features". This leads to the network falling into a serious learning error: the model no longer analyzes the real physical features such as cracks, oil seepage or deformation on the surface of parts, but mistakenly associates "the presence of colored lines in the image" with "fault category". This phenomenon causes the model to have an exponential increase in the false negative rate when facing unlabeled original images, which seriously affects the generalization performance in the actual production environment.

[0007] (2) The contradiction between ultra-high resolution panoramic images and the extraction of features for extremely small targets: In order to capture potential defects in key train components, modern TVDS systems typically use high-resolution line array cameras to acquire images in real time while the train is running at high speed. The resulting single-frame panoramic images of the chassis have extremely high resolution (usually up to 8192×8192 pixels or even higher). However, at this extreme resolution, the effective pixel percentage of tiny defects in the components under test (such as loose bolts in hydraulic shock absorbers, small cracks in springs, etc.) is usually less than 0.5% in the entire image. Under the condition of limited computing power and video memory, directly inputting an 8192px ultra-high resolution image into a neural network requires drastic downsampling (downsampling to 640×640 or even 416×416). This drastic dimensionality reduction operation will cause the defects of components with extremely small physical dimensions (such as spring cracks) to completely disappear in the deep feature map. However, if the entire image is not scaled, the model cannot complete the forward propagation operation due to the limitations of GPU video memory. In addition, the train chassis environment is extremely complex, including intricate pipelines, attached mud and dirt, and shadow areas with changing light and shadow. These environmental noise interferences can easily cause the detection system to generate a large number of false positives.

[0008] (3) Extremely Uneven Distribution of Samples and Failure of Evaluation System in Industrial Scenarios: Industrial defect samples are characterized by "sparseness" and "extreme imbalance". There are millions of images of normal components, while there may only be dozens of images of various faults. If traditional cross-entropy loss and accuracy metrics are used for optimization and evaluation, the model is very likely to fall into a local optimum where "predicting all targets as normal will achieve an accuracy of over 99%". Such inflated evaluation metrics are fatal in the field of train safety.

[0009] In response to the various deep-seated challenges described above, there is an urgent need for an intelligent detection solution for critical train components to achieve end-to-end conversion of highly reliable component diagnostic results. Summary of the Invention

[0010] To address the aforementioned technical problems, this invention provides a method for detecting anomalies in bus components based on deep residual networks. By employing a multi-stage deep cascaded architecture, it effectively improves the ability to identify extremely small targets such as minor cracks in the bolster beam and slight oil leakage in the shock absorber, thereby significantly reducing the false alarm rate.

[0011] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0012] A method for detecting anomalies in bus components based on deep residual networks includes the following steps: S1, based on image processing technology and computer graphics, process the full-scene component anomaly image with anomaly annotations to obtain the unannotated full-scene component anomaly image; S2, the trained target detection model is used to locate and crop abnormal parts in the unlabeled full-scene abnormal part images to achieve scene pruning and obtain abnormal part ROI images; S3, randomly mix and stratify the ROI images of abnormal parts and normal parts, and then standardize them to obtain a sample dataset for training and testing the train parts anomaly detection model. S4. Train and evaluate the train component anomaly detection model based on deep residual network according to the sample dataset to obtain the train component anomaly detection model. S5: When it is necessary to detect anomalies in train target components, acquire the image of the train to be detected, and use the trained target detection model and train component anomaly detection model to locate, crop, and detect anomalies in the train target components.

[0013] Furthermore, step S1 includes: S1.1, Based on HSV nonlinear color space conversion and double closed interval mask segmentation strategy, generate the initial full-scene component abnormality image anomaly annotation box mask map; S1.2, based on the mathematical morphology joint filtering of partial differential equation evolution, performs closing and dilation operations on the initial anomaly labeling box mask to obtain the final anomaly labeling box mask; S1.3, Based on the image inpainting algorithm that considers pixels in the neighborhood and pixel gradients, the abnormal images of all-scene components with abnormal annotations are repaired using the final abnormal annotation box mask image, resulting in the unannotated abnormal images of all-scene components.

[0014] Furthermore, S1.3 includes: Starting from the mask boundary of the final anomaly annotation box mask image, the process proceeds gradually into the interior of the mask region in an increasing order of distance function. For the pixels to be repaired that are being calculated within the mask area Its color prediction value depends not only on its surrounding small radius known neighborhood. pixels within It is more strictly controlled by the pixel gradient within that neighborhood of the image, pixel Final color composite value It is a weighted integral of the estimated values ​​of all known pixels in the neighborhood: ; In the formula, I(q) represents the weight; I(q) represents the pixel. The color value; To approximate the color values ​​of a local region using a first-order Taylor series expansion, This represents a linear estimate of the amount of color change in an image caused by spatial location differences; After repairing all pixels within the masked area, the final unannotated image of the abnormal parts in the entire scene is obtained.

[0015] Furthermore, the target detection model in step S2 adopts the YOLOv11x model architecture, specifically including: S2.1, a backbone network based on the C3k2 operator is used to extract deep features from the input unlabeled full-scene component anomaly image to obtain the low-level visual semantic information of the unlabeled full-scene component anomaly image; S2.2, a neck network based on the C2PSA spatial attention mechanism is used for feature weighting and fusion to output a sequence of component prediction boxes; S2.3 employs a lossless ROI cropping mechanism based on coordinate inverse mapping, mapping the coordinates of the predicted bounding boxes of the parts back to the coordinate system of the annotated full-scene abnormal parts image, and using the tensor slicing operator to crop the abnormal parts ROI image.

[0016] Furthermore, step S3, before the standardization process, also includes: Data augmentation based on probabilistic triggered affine transformation and color reconstruction: In the spatial dimension, a random scale dynamic scaling and random mirror inversion with a scaling factor in the range of [0.8, 1.0] are introduced; in the color dimension, Gaussian white noise is adaptively injected, and random contrast stretching and nonlinear gamma correction of brightness are performed in the pixel domain.

[0017] Furthermore, step S4 includes: S4.1, the training sample dataset is input into the reconstructed deep residual network, and forward high-dimensional feature extraction, GAP space compression and backpropagation weight iteration based on Adam momentum gradient are performed to obtain a parameter evolution model with fine-grained fault perception capability. S4.2 performs gradient-free forward inference with fixed optimal weights and F1-Score multidimensional index verification on the test sample dataset to obtain a train component anomaly detection model that balances recall and precision.

[0018] Furthermore, the reconstructed deep residual network is built on ResNet50, and the fully connected layers and flattening operations at the tail of the original ResNet50 architecture are removed and replaced with a global average pooling layer, a single-neuron linear classification head, and a sigmoid activation function.

[0019] Furthermore, step S5 includes: S5.1 When it is necessary to perform anomaly detection on target components of a train, acquire an image of the train to be detected; S5.2, use the trained target detection model to perform ROI localization and cropping in the train image to be detected, and determine the ROI image and component category of the train target parts; S5.3 standardizes the ROI image of the target train component, then inputs it into the train component anomaly detection model, and outputs the anomaly detection results of the target train component.

[0020] Furthermore, prior to step S1, the following steps are also included: S0: Obtain historical visual cue information for each train component from the train inspection database. The historical visual cue information includes component location and name cue data and full-scene component anomaly images with anomaly annotations.

[0021] Furthermore, step S0 includes: S0.1, Establish a cx_Oracle streaming data mining interface based on the Python environment with the train inspection database; S0.2, use pre-set semantic filtering logic to determine keywords corresponding to the names of train parts or the abnormality types of parts; S0.3, Based on keywords, LOB field streaming technology is used to export component location and name prompt data and full-scene component anomaly images with anomaly annotations from the train inspection database to form historical visual prompt information.

[0022] The bus component anomaly detection method based on deep residual networks provided by this invention has the following beneficial effects: (1) Based on image processing technology and computer graphics processing of anomaly images of all-scene components with anomaly annotations, the unannotated anomaly images of all-scene components are obtained. This achieves the seamless removal of manual annotations from anomaly images of components with annotations from multiple perspectives, corrects the feature learning direction from the source, avoids the model from falling into the learning error, effectively reduces the false negative rate, and improves the generalization performance of the trained model in the actual production environment. (2) Scene pruning is performed using the target detection model to simplify the panoramic complete component anomaly image into a regional ROI image. Then, the train component anomaly detection model based on deep residual network is used to detect component anomalies. Since the strategy of ROI pruning can maintain the original pixel density, it effectively improves the ability to identify extremely small targets such as small cracks in the sleeper beam and slight oil leakage in the shock absorber, thus greatly reducing the false alarm rate. (3) Randomly mix and stratified sample the ROI images of abnormal parts and normal parts to construct a sample dataset for training and testing the train parts anomaly detection model. This ensures that the extremely rare fatal defect samples still have an equal exposure rate in the test set. This scientific sampling framework effectively avoids the illusion of accuracy caused by the "loss" of rare categories in traditional evaluation, and ensures that the F1-Score shown in the subsequent model test can directly and honestly reflect the real early warning effectiveness on the production line. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 The flowchart illustrates the bus component anomaly detection method based on deep residual networks provided in this embodiment of the invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0026] like Figure 1 As shown, this embodiment of the invention provides a method for detecting anomalies in bus components based on deep residual networks, including the following steps: S0: Obtain historical visual cue information for each train component from the train inspection database. The historical visual cue information includes component location and name cue data and full-scene component anomaly images with anomaly annotations.

[0027] The upper limit of industrial-grade deep learning is determined by a high-quality data foundation. The core task of this step (stage) is to extract "historical visual cue information" of each train component from the massive TVDS train inspection history database, both in structured and unstructured form. This historical visual cue information includes not only the absolute physical location and category name of the component in the panoramic image, but more importantly, it contains global component anomaly images with anomaly annotations from the inspector.

[0028] In this embodiment, the train inspection database is the Oracle database of the Train Vehicle Detection System (TVDS). TVDS obtains historical inspection data of all train components stored in the Oracle database. S0.1, Establish a Python-based streaming data mining interface with an Oracle-type train detection database; Python-based streaming data mining uses cx_Oracle to establish a high-concurrency connection pool, which can directly access the Oracle database on the TVDS backend. S0.2, use pre-set semantic filtering logic to determine the keywords corresponding to each train component; For example, in core production tables such as TV_FAILURE_BAK_2024, SQL operators (such as LIKE) and regular expressions are used to match keywords in the S_ANALYSTREMARK (analysis notes) field (such as "pillow beam crack", "severe leakage of shock absorber", "loose bolts"). This joint extraction of text semantics and visual data endows the images with accurate expert prior knowledge from the very beginning. S0.3, based on keywords, the LOB (Large Object) field streaming technology is used to export component location prompt data and multi-view labeled component anomaly images from the train inspection database to form historical visual prompt information.

[0029] The ultra-high resolution array images of the train's underside are typically persisted in Oracle databases as binary large objects. Considering that a single uncompressed industrial-grade array image can be tens of megabytes in size, loading it into memory all at once using the conventional cursor.fetchone() and simple read() methods would cause extremely serious memory leaks and database network I / O congestion.

[0030] This embodiment employs streaming reading technology for LOB fields. In the underlying Python script implementation, by overriding the cursor's `outputTypeHandler`, the extraction of large binary object data is redirected to iterative memory mapping. Specifically, a safe memory block threshold (e.g., 65536 bytes in `num_bytes_in_chunk`) is set. A dynamic offset pointer `offset` is used in conjunction with `blob.read(offset, num_bytes_in_chunk)` to iteratively retrieve data blocks, and the image tensor is reconstructed losslessly in the host machine's memory by concatenating the byte streams. Simultaneously, the bounding box coordinates and name dictionaries of components stored in other relational columns of the database are extracted synchronously. The precise guidance of text coordinates, component name prompts, and images containing red and blue highlighting constitute a multimodal set, forming a historical visual cue information database.

[0031] S1, based on image processing technology and computer graphics, processes the anomaly images of all-scene components with anomaly annotations to obtain anomaly images of all-scene components without annotations.

[0032] In this embodiment, existing full-scene component anomaly images with anomaly annotations generally use red or blue markings. Before inputting the image into any neural network with a weight update mechanism, these contaminated pixels must be thoroughly cleaned at the physical level using image processing and computer graphics processing techniques. This step (stage) is precisely to output an anomaly-free full-scene component anomaly image that reflects the true physical chassis appearance. To accurately capture and remove manual red / blue annotations under different lighting conditions, the following steps are required; S1.1, Based on HSV nonlinear color space conversion and double-closed interval mask segmentation strategy, generate an anomaly annotation box mask map of the initial full-scene component anomaly image.

[0033] The red / blue markers commonly used by train inspectors exhibit drastic and non-linear shifts in pixel values ​​within the standard RGB (red, green, blue) color space under varying train arrival times, weather conditions, and undercarriage spotlight intensity. To address this challenge, this embodiment employs an HSV (Hue, Saturation, Value) color space segmentation strategy, which is insensitive to brightness and reflects the dynamics of human visual color perception. Because color channels in the RGB space are highly coupled, they are easily affected by complex lighting and shadows under the train; however, in the HSV space, hue (H) is highly robust to changes in lighting conditions.

[0034] Let the matrix representation of the image after conversion to HSV space be: An adaptive double-closed interval mask extraction logic was constructed by utilizing the periodicity of the H component. Its mask generation logic is mathematically expressed as follows: ; ; in, This is the mask value of the red anomaly marker box at pixel (x, y); The threshold for the red hue is a two-range threshold (because red crosses the starting point of the color wheel). This represents the mask value of the blue anomaly marker box at pixel (x, y). The threshold for the blue hue range; H(x,y), S(x,y), and V(x,y) represent the hue, saturation, and brightness of pixel (x,y), respectively; These are the lower limits for saturation and brightness, respectively.

[0035] For example, in OpenCV, if H∈[0,179], then: .

[0036] S1.2, based on the mathematical morphology joint filtering of partial differential equation evolution, performs closing and dilation operations on the initial anomaly labeling box mask to obtain the final anomaly labeling box mask.

[0037] Limited by image compression algorithms (such as JPEG blockiness) and the uneven pressure of the brush used by the inspector, the initial mask obtained by direct mapping from the color space is often riddled with tiny holes, and its edges cannot completely encompass the transition zone of color diffusion. If this mask is used directly for repair, the image will produce disastrous grid-like artifacts. To address this, a joint filtering based on mathematical morphology, derived from partial differential equations, is introduced.

[0038] This embodiment introduces a 3×3 elliptical structural element K. ell Morphological joint operations: ; in, and These represent the enhanced anomaly bounding box mask and the original anomaly bounding box mask, respectively. Corresponding to or ; "Represents morphological closing operation (expansion followed by erosion), used to bridge tiny holes inside the connector strip; This represents a morphological dilation operation used to enclose the edge transition area where the color diffuses outward, ensuring that the mask to be repaired completely covers the contaminated pixels.

[0039] The main function of this step is to non-destructively bridge the fibrous gaps inside the marker lines; the subsequent pure dilation operation forces the mask boundary to expand outward in a radial pattern by 1 to 2 pixel levels. This redundant boundary design provides sufficient texture transition zone for subsequent color filling.

[0040] S1.3, Based on the image inpainting algorithm that considers pixels in the neighborhood and pixel gradients, the abnormal images of all-scene components with abnormal annotations are repaired using the final abnormal annotation box mask image, resulting in the unannotated abnormal images of all-scene components.

[0041] After accurately identifying the region to be repaired and its known edges based on the anomaly bounding box mask image, an image inpainting algorithm considering neighborhood pixels and pixel gradients is introduced for repair. Unlike simple mean interpolation or uncontrollable generation based on deep generative models (GANs), the core advantage of the image inpainting algorithm considering neighborhood pixels and pixel gradients lies in its strict adherence to the structure-preserving property of image gradients, which can perfectly restore the surface texture of steel parts.

[0042] In this embodiment, S1.3 includes: Starting from the mask boundary of the final anomaly annotation box mask image, the process proceeds gradually into the interior of the mask region in an increasing order of distance function. For the pixels to be repaired that are being calculated within the mask area Its color prediction value depends not only on its surrounding small radius known neighborhood. pixels within It is more strictly controlled by the pixel gradient within that neighborhood of the image, pixel Final color composite value It is a weighted integral of the estimated values ​​of all known pixels in the neighborhood: ; In the formula, the weight function It is designed as a product of the following three terms: exponential decay of the geometric distance factor, contour direction factor, and level set normal factor; I(q) is the pixel. The color value; To approximate the color values ​​of a local region using a first-order Taylor series expansion, This represents a linear estimate of the amount of color change in an image caused by spatial location differences; After repairing all pixels within the masked area, the final unannotated image of the abnormal parts in the entire scene is obtained.

[0043] Through the deduction of image inpainting algorithms that take into account pixels in the neighborhood and pixel gradients, the red and blue lines in the image are "erased" and seamlessly replaced by real textures such as surrounding oil stains, mud, and metal rust. The restored, unannotated, full-scene abnormal component images achieve a purity level that is indistinguishable to the naked eye as to any previous human intervention, providing an absolutely objective physical benchmark for deep learning.

[0044] S2 uses a trained object detection model to locate and crop abnormal parts in the unlabeled full-scene abnormal part images, achieving scene pruning and obtaining abnormal part ROI images.

[0045] After obtaining a physically clean global image, the core challenge arises: resolving the contradiction between a massive 8192px scene and extremely small 0.5% lesions. Due to the dual limitations of GPU memory and the loss of information in the pooling layer, directly inputting the panoramic image into the fault classification network is destined to fail. Therefore, the object detection model in this step (stage) employs a spatial attention compression strategy of "scene pruning".

[0046] Unlike traditional machine learning's "model pruning" (or unstructured pruning, which removes redundant neuron weights by setting weights to zero using the L1 norm or magnitude threshold to accelerate the inference process), the innovative "scene pruning" in this embodiment refers to using the coordinate regression capability of the detection network at the spatial image input end to cut off and discard more than 90% of the worthless pipelines, dirt, and railway track backgrounds, retaining and extracting only the core region of interest (ROI).

[0047] In this embodiment, the YOLOv11x model architecture is used as an example for the target detection model, which specifically includes: S2.1, a backbone network based on the C3k2 operator is used to extract deep features from the input unlabeled full-scene component anomaly image to obtain the low-level visual semantic information of the unlabeled full-scene component anomaly image.

[0048] The C3k2 operator performs channel-level separation of the input feature matrix and extracts features alternately using two convolutional kernels with small receptive fields. Its underlying mathematical logic follows: ; in, The low-level visual semantic feature map output by the C3k2 operator of the backbone network; The two sub-feature matrices obtained after channel separation from the input feature matrix; Convolution operations with smaller receptive fields; : The concatenation operation of feature tensors along the channel dimension; : Weight matrix used for feature fusion.

[0049] This module significantly improves inference speed while reducing parameter redundancy. It enhances the network's nonlinear mapping capability, enabling high-density extraction of complex low-level visual semantic information such as mechanical contours and metallic reflections from panoramic images.

[0050] S2.2 employs a neck network based on the C2PSA spatial attention mechanism for feature weighting and fusion, outputting a sequence of component prediction boxes.

[0051] To address the drastic changes in light and shadow on the train chassis and the positioning drift caused by occlusion, the Neck network integrates C2PSA (Cross-Stage Local Module with Spatial Attention) and PANet.

[0052] Visual Focusing: C2PSA uses parallel branches to process feature maps Local pooling and convolution activation are performed to generate a spatial weight matrix. : ; in, : The generated spatial attention weight matrix; The Sigmoid activation function is used to normalize the weights to the [0, 1] interval. : Convolution operations are used for cross-channel feature fusion and dimensionality reduction. and : Represents the max pooling and average pooling operations performed along the channel dimension, respectively.

[0053] pass During operation, the system precisely locks onto the edges of components like a "spotlight". Enhanced feature map after spatial attention weighting. : Represents the element-wise multiplication operation of a matrix.

[0054] Fusion mechanism: The attention-weighted features are fed into PANet (path aggregation network). Through bidirectional path interweaving from top to bottom and bottom to top, deep semantic and shallow location information are deeply coupled, and a highly robust anchor box prediction sequence is output.

[0055] PANet introduces a bottom-up path enhancement structure, iteratively backpropagating fine-grained features from shallow layers to higher layers. Let the backbone input features of the i-th layer be... PANet's feature fusion involves more than just top-down computation: ; A parallel bottom-up enhancement path is also introduced, which fuses high-resolution features from lower levels with those from the current level after downsampling. The mathematical expression for this is supplemented as follows: ; in, : The feature map of the i-th layer after top-down path fusion; Upsampling operation enlarges deep, low-resolution feature maps; : Top-down feature map of the (i+1)th layer (higher semantic levels); : The final enhanced feature map of the i-th layer output by PANet; Downsampling reduces the size of shallow, high-resolution feature maps; : The final enhanced feature map of the (i-1)th layer (shallower layer).

[0056] S2.3 employs a lossless ROI cropping mechanism based on coordinate inverse mapping, mapping the coordinates of the predicted bounding boxes of the parts back to the coordinate system of the annotated full-scene abnormal parts image, and using the tensor slicing operator to crop the abnormal parts ROI image.

[0057] After the YOLOv11x model training converges, the system uses this model to perform automated, large-scale inference and screening of anomaly images of all scene components output in step S1. To ensure that each physical component outputs only one absolutely certain spatial bounding box, an extremely strict confidence threshold (e.g., ...) is set at the end of the detection head. ), and an extremely low crossover ratio threshold for nonmaximum suppression (NMS) ( ( ), to eliminate duplicate and redundant anchor frames.

[0058] At this point, the model outputs a size relative to its compressed input size (e.g., ...). Normalized relative coordinates (scaled mesh). To obtain lossless, high-resolution defect slices, a coordinate inverse mapping matrix algorithm was constructed. The coordinates of the predicted bounding boxes output by the network were then... The absolute coordinates are obtained by accurately mapping back to the original, unannotated, full-scene component anomaly image coordinate system. .

[0059] ; ; in, The normalized relative coordinates of the network output prediction box correspond to the X-axis and Y-axis coordinates of the center point and the width and height of the prediction box, respectively, with values ​​ranging from [0, 1]. : The absolute physical width and height of the original unannotated full-scene component anomaly image; : Map back to the original unannotated full-scene component anomaly image coordinate system, including the X-axis coordinates of the center point of the predicted bounding box, the Y-axis coordinates, and the width and height of the predicted bounding box.

[0060] After obtaining the absolute coordinates, the system uses tensor slicing to extract the original image matrix. Non-destructive region of interest extraction to generate ROI images of abnormal components. : ; ; ; in : The horizontal starting and ending coordinates of the extracted region on the original image; : The vertical starting and ending coordinates of the cropped area on the original image; : The pixel matrix of the original unannotated full-scene component anomaly image. : The pixel matrix of the region of interest for the abnormal components output after cropping.

[0061] This process deconstructs the panoramic image into regions of interest (ROIs) stored in compartments categorized by type. These ROI images not only have significantly reduced file size and cleaner backgrounds, but also maintain 100% of the original pixel density of the acquisition device at the physical level. Cracks and blemishes, which occupy only 0.1% of the width of the panoramic image, appear extremely clear and conspicuous in the slices, removing the fundamental spatial dimension obstacle for the depth analysis in stage five.

[0062] S3 involves randomly mixing and stratified sampling of ROI images of abnormal components and normal component images, followed by standardization to obtain a sample dataset for training and testing the train component anomaly detection model.

[0063] After obtaining a large number of categorized images of abnormal component ROIs, these images need to be merged with normal ROI images of homogeneous components extracted from normal trains. However, fault datasets in industrial scenarios generally suffer from extreme class imbalance. There may be tens of thousands of normal component images, while high-risk abnormal components such as leaking shock absorbers or cracked sleeper beams often have only a few hundred images. If the network is trained directly, the gradient descent process will be hijacked by the massive number of negative samples (normal components), causing the classifier to collapse into a constant predictor during hyperplane partitioning. This step (stage) aims to synthesize a robust dataset with extremely high generalization potential through rigorous statistical data governance and augmentation techniques.

[0064] To avoid inflated or distorted evaluation results caused by data segmentation, this embodiment employs an extremely rigorous stratified random sampling protocol. When segmenting the entire multidimensional ROI image library into training, validation, and test sets, it is mandatory that all segmentations be performed independently and proportionally within each "subdivided fault subtype category".

[0065] Assuming the category set includes Normal, Abnormal, etc., in the training set... and test set The distribution is strictly constrained by the absolute proportion function: ; Where C represents the set of all image categories (e.g., in this embodiment, it includes categories such as Normal and Abnormal). : Represents any specific category in the category set C. D: Represents the entire original dataset. : Represents the training set partitioned from the full dataset D. : Represents the set of all samples in the full dataset D whose true labels belong to category c. : Represents the partitioned training set In this context, the set consists of all samples whose true labels belong to category c.

[0066] This constraint ensures that even the extremely rare, critically flawed samples still have a proportional exposure rate in the test set. This scientific sampling framework effectively avoids the illusion of accuracy caused by the "loss" of rare categories in traditional evaluations, ensuring that the F1-Score shown in subsequent model testing directly and honestly reflects the real early warning effectiveness on the production line. Furthermore, the entire data construction process uses the operating system's symbolic link technology to map the virtual storage tree, eliminating storage fragmentation and I / O congestion caused by the physical copying of massive amounts of images.

[0067] In some preferred embodiments, to enable the model to cope with unknown lighting shifts and shooting perspective distortions that may occur when actual trains enter stations in the future, a data augmentation pipeline based on probability-triggered affine transformation and color reconstruction is deployed before standardization: In the spatial dimension, random scale dynamic scaling and random mirror reversal with a scale coefficient range of [0.8, 1.0] are introduced, which profoundly simulates the lens perspective error caused by the dynamic deformation of the train on the rails and the physical mirror properties of the train's left and right symmetrical structural components; In the color dimension, Gaussian white noise is adaptively injected, and random contrast stretching and nonlinear gamma correction of brightness are performed in the pixel domain, accurately replicating various extreme and harsh working conditions such as continuous rain, low light in tunnels, and strong reflections from rusted steel under the train.

[0068] The final step is the crucial standardization mapping operation. This utilizes pre-computed or globally applicable ImageNet distribution parameters—the mean tensor. (Typically [0.485, 0.456, 0.406]) and the standard deviation tensor ([0.229, 0.224, 0.225]), performing high-dimensional Z-score normalized projection on each RGB image in the recombined batch. This underlying mathematical transformation not only completely eliminates the ill-conditioned distortion of the error surface caused by the difference in dimensions of each color channel in the feature space, but also ensures that the gradient vector can slide rapidly toward the global optimum in the smoothest optimization space when subsequent ultra-deep networks (such as ResNet50) perform backpropagation updates by forcibly pulling the feature distribution back into a hypersphere centered at zero.

[0069] S4. Train and evaluate the train component anomaly detection model based on deep residual network according to the sample dataset to obtain the train component anomaly detection model.

[0070] For high-quality ROI image tensors, a powerful feature classifier capable of extracting fine-grained "lesions" in a deep feature space is needed. Given that features such as imperceptible cracks on metal surfaces and minute oil seepage at threaded joints exhibit extremely weak local high-frequency responses in the image frequency domain, a deep residual network (using ResNet50 as an example in this embodiment) was chosen, combined with transfer learning, to construct the final train component anomaly detection model.

[0071] When convolutional neural networks have dozens of layers, they are highly susceptible to vanishing gradients or gradient explosions due to the chain rule (multiplication of numerous derivatives less than 1) during backpropagation, causing the loss function to stagnate. ResNet50 innovatively introduced a skip connection / shortcut topology. In the residual block, the network is no longer forced to learn the complete underlying mapping function. Instead, it fits the residual function. At this point, the original mapping becomes: ; in, For input features, These are the weights of the convolutional layer. Representative process The feature residuals after bottleneck convolution computation. (End of section) This is called a skip connection. When calculating the gradient during backpropagation, according to the chain rule: ; in The total partial derivative of the loss function L with respect to the input feature x represents the overall gradient signal transmitted to the current shallow network (i.e., the input end of this layer) during backpropagation gradient calculation. The partial derivative of the loss function L with respect to the current residual block output H(x) represents the gradient information passed from backpropagation in the deeper network. : The local partial derivative of the complete mapping function H(x) with respect to the input feature x. : The nonlinear gradient inside the residual function, that is, the partial derivative of the residual branch with respect to the input feature x.

[0072] Due to constant The existence of deep nonlinear gradients Approaching The gradient signal can also be directly and losslessly transmitted to the shallow network through the constant channel.

[0073] This short-circuit direct connection directly transmits the input signal across layers to the output, providing a non-attenuating "gradient highway" for backpropagation. This allows the model to maintain extremely high feature extraction and representation capabilities even at depths of up to 50 layers, enabling it to resolve critical minute defects such as bolt thread deformation from high-dimensional latent space.

[0074] In the engineering practice for the current binary classification (normal / abnormal) discrimination task, the system follows the transfer learning mechanism, loads pre-trained weights based on ImageNet, and performs deep reconstruction and cutting on the original ResNet50 architecture.

[0075] Freeze & Fine-tune strategy: In the early stages of training, the system freezes the low-level convolutional base containing shallow edge and texture recognition operators to protect the strong generalization features learned on ImageNet from being destroyed. As training enters a stable period, the deep residual blocks are unfrozen to perform fine-tuning based on the physical characteristics of the train chassis.

[0076] Spatial Redundancy Suppression and Global Average Pooling (GAP): The 1000 fully connected layers and redundant flattening operations at the end of the original architecture are removed, and replaced with a global average pooling layer. The GAP layer directly compresses and reorganizes the 7×7 high-dimensional feature map matrix into a compact 2048-dimensional vector representation in the spatial dimension.

[0077] The GAP layer provides a feature map matrix for each two-dimensional space of the tensor. The formula for calculating the global mean is as follows: ; in, : No. Each feature channel in coordinates The activation response value at the location; : No. Global scalar features after channel compression; : Spatial height of the feature map tensor (actually taken as 7 here). : Spatial width of the feature map tensor (actually taken as 7 here).

[0078] Through GAP operation, originally The high-dimensional tensor is extremely condensed into a 2048-dimensional eigenvector. This not only reduces the number of parameters in this part to 0, completely eliminating the risk of overfitting, but also establishes a clear semantic mapping between feature channels and the final detection category. This fundamentally eliminates tens of millions of redundant parameters, strongly blocks the path of model overfitting, and significantly enhances the model's tolerance to component viewpoint translation.

[0079] Deploying a linear classification decision head: This involves obtaining feature vectors. Subsequently, this embodiment deploys a customized single-neuron linear classification head to perform logical judgment for fault binary classification: ; in, For the weight vector, For scalar bias, This represents the logits output of the linear classification layer, which are the raw prediction scores without activation function mapping.

[0080] The input is fed into the Sigmoid activation function, and the output is the final probability that the component has a physical anomaly. : ; in, This represents the Sigmoid activation function.

[0081] In the backpropagation and weight evolution stages of the loss function, the cross-entropy loss function is used to measure the divergence penalty between the predicted distribution and the true label. Based on this, the Adam optimizer, which integrates first-order and second-order moment estimates, is selected to dynamically schedule the global learning rate. In each iteration update of Adam, at the [missing information] stage of each training round... At each time step, let the target loss function L be adjusted against the network parameters. The gradient is The Adam algorithm tracks the first moment of the gradient. (Mean, similar to momentum) and second moment (Uncentered variance) is used to update parameters: ; ; in, : The exponential decay rate of the first moment estimate (usually set to 0.9 by default), used to control the weight of the influence of historical gradients on the current momentum; The exponential decay rate of the second-order moment estimate (usually set to 0.999 by default) is used to control the weighting of the influence of historical gradient squares on the current variance; the bias-corrected matrix is ​​used to guide the weights. The network steadily slides down the most directional curve towards the bottom of the loss canyon. The adaptive nature of the Adam optimizer gives the network the ability to capture extremely weak gradients during the fine-tuning stage. The final model parameter iteration formula is: ; Explanation of variable meanings: Global base learning rate; The corrected first moment provides momentum for the smoothing trend of the gradient, which helps the model overcome gentle saddle points. The corrected second moment, as the denominator, can be used to assign a larger learning step size to those defect feature parameters that rarely appear but are extremely critical in industrial inspection. : To prevent the denominator from being a small positive constant (usually taken as ) This dynamic mechanism enables ResNet50 to quickly converge to the global optimal neighborhood in the early stages when facing complex components at the bottom layer of a bus, and to capture extremely weak gradient signals of physical defects in the middle and later stages by fine-tuning the weights.

[0082] Regarding the evaluation and model preservation mechanism, for train fault diagnosis, the risk of train derailment due to missing a single serious crack is far greater than the cost of dispatching maintenance personnel for one hundred manual re-inspections due to system false alarms. Therefore, this embodiment abandons the traditional accuracy metric and adopts a monitoring mechanism with the F1 score as the absolute core. Based on the confusion matrix, precision P and recall R are defined as follows: ; ; In this system, true positives (TP) represent correctly detected faulty components, false positives (FP) represent false alarms, and false negatives (FN) represent missed detections. The F1 score is the harmonic mean of the two scores. ; Automatic checkpointing strategy: Throughout the system's training process, the loss function is not only used as the direction of gradient descent, but the system also calculates the macro-average F1 score for all classes on the test set every fixed number of iterations using the current weights. The model architecture has a built-in monitoring script that triggers a weight-saving instruction to persist the network parameters only when the F1 score generated in the current epoch is significantly greater than the historical highest F1 record. This innovation ensures that the classification model finally delivered to the production environment perfectly balances recall and precision, minimizing false negatives while avoiding excessive false positives caused by oversensitivity.

[0083] S5: When it is necessary to detect anomalies in train target components, acquire the image of the train to be detected, and use the trained target detection model and train component anomaly detection model to locate, crop, and detect anomalies in the train target components.

[0084] Once all stages in the offline environment have been iterated and closed, a complete "cascaded visual large model" inference engine is finally built. The technical process then formally enters the production line environment, executing the automated inference stage for online anomaly detection of trains in all-weather service. This step (stage) aims to achieve millisecond-level automated operation for incoming trains, from panoramic image acquisition to multi-dimensional judgment reports.

[0085] Specifically, it includes: S5.1 When it is necessary to detect abnormalities in target components of a train, the TVDS optoelectronic equipment deployed on the track can quickly acquire images of the train to be inspected. S5.2, using the trained target detection model to perform ROI localization and cropping in the train image to be detected, determine the ROI image and component category of the train target parts, and effectively filter out interference information such as the base plate, mud, and light and shadow that are irrelevant to the task in the background; S5.3 standardizes the ROI image of the target train component, then inputs it into the train component anomaly detection model, and outputs the anomaly detection results of the target train component.

[0086] In some embodiments, after cross-validation between the spatial dimension target detection model and the deep semantic dimension train component anomaly detection model, the output endpoint data of each model are aggregated. The model identifier of the target component, its absolute spatial position (horizontal / vertical coordinate system) at the bottom of the carriage, the high-definition close-up ROI slices, and the final anomaly detection results including confidence scores are serialized and reassembled according to structured communication protocols such as JSON.

[0087] If the system's inference result determines the problem to be "abnormal" and the confidence level exceeds the safety threshold, the intelligent distribution system will immediately activate the alarm link, automatically carrying a comprehensive diagnostic report containing the aforementioned multimodal data, and accurately pushing it to the handheld mobile industrial control device of the on-site maintenance personnel or the command center monitoring platform. Because the system has accurately located the defect and eliminated background noise that interferes with the view, maintenance technicians can use detailed, high-resolution structured prompts to perform directional verification and physical intervention repairs under the train.

[0088] This six-stage high-dimensional visual inference architecture, starting with automated historical annotation and cleaning, proceeding through panoramic scene pruning and spatial isolation, and finally completing the judgment using a deep residual feature matrix, achieves an end-to-end technological leap from massive, disordered industrial images to accurate, reliable, and auditable intelligent fault judgment. This process effectively solves the problems of human noise interference and redundant computation in traditional machine vision systems, laying a solid technical foundation for building a next-generation, unmanned, fully covered intelligent rail transit operation and maintenance equipment system.

[0089] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting anomalies in bus components based on deep residual networks, characterized in that, Includes the following steps: S1, based on image processing technology and computer graphics, process the full-scene component anomaly image with anomaly annotations to obtain the unannotated full-scene component anomaly image; S2, the trained target detection model is used to locate and crop abnormal parts in the unlabeled full-scene abnormal part images to achieve scene pruning and obtain abnormal part ROI images; S3, randomly mix and stratify the ROI images of abnormal parts and normal parts, and then standardize them to obtain a sample dataset for training and testing the train parts anomaly detection model. S4. Train and evaluate the train component anomaly detection model based on deep residual network according to the sample dataset to obtain the train component anomaly detection model. S5: When it is necessary to detect anomalies in train target components, acquire the image of the train to be detected, and use the trained target detection model and train component anomaly detection model to locate, crop, and detect anomalies in the train target components.

2. The method for detecting anomalies in bus components based on deep residual networks according to claim 1, characterized in that, Step S1 includes: S1.1, Based on HSV nonlinear color space conversion and double closed interval mask segmentation strategy, generate the initial full-scene component abnormality image anomaly annotation box mask map; S1.2, based on the mathematical morphology joint filtering of partial differential equation evolution, performs closing and dilation operations on the initial anomaly labeling box mask to obtain the final anomaly labeling box mask; S1.3, Based on the image inpainting algorithm that considers pixels in the neighborhood and pixel gradients, the abnormal images of all-scene components with abnormal annotations are repaired using the final abnormal annotation box mask image, resulting in the unannotated abnormal images of all-scene components.

3. The method for detecting anomalies in bus components based on deep residual networks according to claim 2, characterized in that, S1.3 includes: Starting from the mask boundary of the final anomaly annotation box mask image, the process proceeds gradually into the interior of the mask region in an increasing order of distance function. For the pixels to be repaired that are being calculated within the mask area Pixels Final color composite value It is the neighborhood All known pixels within Weighted integral of the estimated value: ； In the formula, I(q) represents the weight; I(q) represents the pixel. The color value; To approximate the color values ​​of a local region using a first-order Taylor series expansion, This represents a linear estimate of the amount of color change in an image caused by spatial location differences; After repairing all pixels within the masked area, the final unannotated image of the abnormal parts in the entire scene is obtained.

4. The method for detecting anomalies in bus components based on deep residual networks according to claim 1, characterized in that, The object detection model in step S2 adopts the YOLOv11x model architecture, which specifically includes: S2.1, a backbone network based on the C3k2 operator is used to extract deep features from the input unlabeled full-scene component anomaly image to obtain the low-level visual semantic information of the unlabeled full-scene component anomaly image; S2.2, a neck network based on the C2PSA spatial attention mechanism is used for feature weighting and fusion to output a sequence of component prediction boxes; S2.3 employs a lossless ROI cropping mechanism based on coordinate inverse mapping, mapping the coordinates of the predicted bounding boxes of the parts back to the coordinate system of the annotated full-scene abnormal parts image, and using the tensor slicing operator to crop the abnormal parts ROI image.

5. The method for detecting anomalies in bus components based on deep residual networks according to claim 1, characterized in that, Step S3, before the standardization process, also includes: Data augmentation based on probabilistic triggered affine transformation and color reconstruction: In the spatial dimension, a random scale dynamic scaling and random mirror inversion with a scaling factor in the range of [0.8, 1.0] are introduced; in the color dimension, Gaussian white noise is adaptively injected, and random contrast stretching and nonlinear gamma correction of brightness are performed in the pixel domain.

6. The method for detecting anomalies in bus components based on deep residual networks according to claim 1, characterized in that, Step S4 includes: S4.1, the training sample dataset is input into the reconstructed deep residual network, and forward high-dimensional feature extraction, GAP space compression and backpropagation weight iteration based on Adam momentum gradient are performed to obtain a parameter evolution model with fine-grained fault perception capability. S4.2 performs gradient-free forward inference with fixed optimal weights and F1-Score multidimensional index verification on the test sample dataset to obtain a train component anomaly detection model that balances recall and precision.

7. The method for detecting anomalies in bus components based on deep residual networks according to claim 6, characterized in that, The reconstructed deep residual network is built on ResNet50, and the fully connected layers and flattening operations at the tail of the original ResNet50 architecture are removed and replaced with a global average pooling layer, a single neuron linear classification head, and a sigmoid activation function.

8. The method for detecting anomalies in bus components based on deep residual networks according to claim 1, characterized in that, Step S5 includes: S5.1 When it is necessary to perform anomaly detection on target components of a train, acquire an image of the train to be detected; S5.2, use the trained target detection model to perform ROI localization and cropping in the train image to be detected, and determine the ROI image and component category of the train target parts; S5.3 standardizes the ROI image of the target train component, then inputs it into the train component anomaly detection model, and outputs the anomaly detection results of the target train component.

9. The method for detecting anomalies in bus components based on deep residual networks according to claim 1, characterized in that, Before step S1, the following is also included: S0: Obtain historical visual cue information for each train component from the train inspection database. The historical visual cue information includes component location and name cue data and full-scene component anomaly images with anomaly annotations.

10. The method for detecting anomalies in bus components based on deep residual networks according to claim 9, characterized in that, Step S0 includes: S0.1, Establish a cx_Oracle streaming data mining interface based on the Python environment with the train inspection database; S0.2, use pre-set semantic filtering logic to determine keywords corresponding to the names of train parts or the abnormality types of parts; S0.3, Based on keywords, LOB field streaming technology is used to export component location and name prompt data and full-scene component anomaly images with anomaly annotations from the train inspection database to form historical visual prompt information.

Images