Multi-view fusion subway car body damage intelligent discrimination system

Abstract

The present application belongs to the field of artificial intelligence and rail transit safety detection technology, and specifically relates to a multi-view fusion subway car body damage intelligent discrimination system, aiming to solve the problems of low detection efficiency, high missed detection rate and difficult multi-source data fusion of subway car body damage detection. The system realizes omnidirectional scanning of the car body by arranging a multi-view visible light and infrared camera array; uses time-space synchronization and space alignment technology to complete multi-source image registration; uses a double-branch network to extract visible light details and thermal field anomaly features, and performs feature-level fusion through a cross-modal attention mechanism; combines time series evolution analysis and Dempster-Shafer evidence theory to realize decision-level fusion and improve discrimination robustness; the central diagnostic server carries out trend prediction and health assessment based on a knowledge graph, and supports automatic early warning and operation and maintenance decision-making. The system has online self-learning and equipment self-diagnosis capabilities, significantly improving detection accuracy and operation and maintenance intelligence level.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and rail transit safety detection technology, specifically involving a multi-view fusion intelligent judgment system for subway car body damage. Background Technology

[0002] In the field of intelligent operation and maintenance of rail transit, the subway car body, as the core operating carrier, directly affects operational reliability and passenger safety. With the continuous expansion of urban rail transit networks and the increasing density of train operations, traditional methods of car body damage detection relying on manual inspections are no longer sufficient to meet the demands for high efficiency, high coverage, and high consistency in operation and maintenance. Against this backdrop, automated inspection technologies based on machine vision and artificial intelligence have gradually become a research hotspot. These technologies acquire information about the car body surface through image acquisition equipment and utilize algorithmic models for defect identification, providing a technical path for achieving all-weather, non-contact monitoring.

[0003] Multi-view imaging technology is widely used for damage detection on complex curved surfaces, aiming to compensate for occlusion and distortion issues from a single viewpoint by observing from different angles. The basic idea is to deploy multiple camera units to simultaneously acquire images of the vehicle surface from lateral, oblique, and top views, in order to obtain more complete damage characterization data. Theoretically, the introduction of multi-view data can significantly improve the damage detection rate, especially for spatially complex defect types such as dents, cracks, and paint peeling.

[0004] While existing technologies have achieved preliminary acquisition and independent discrimination of multi-view data, they still face a series of key technical bottlenecks in practical applications: The lack of effective alignment of multi-source visual data in the spatiotemporal dimensions leads to inconsistent feature representations of the same damage from different perspectives; the isolated output of discrimination results from each perspective lacks cross-view semantic association and confidence fusion mechanisms, making it susceptible to interference from local illumination changes or imaging noise; insufficient interaction at the deep feature level makes it difficult for the system to build a global damage cognition model, failing to distinguish between real damage and false alarms related to structural textures; furthermore, in dynamic operating environments, slight changes in vehicle posture and camera installation deviations further amplify registration errors between perspectives, resulting in decreased stability of discrimination results. These problems collectively lead to a high false alarm rate and insufficient positioning accuracy in current systems under complex operating conditions, making it difficult to support the high-paced, high-reliability maintenance decision-making requirements of subway vehicles. Therefore, an intelligent discrimination architecture with multi-view deep collaborative capabilities is urgently needed to break through existing performance limitations. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-view fusion intelligent damage identification system for subway car bodies, addressing the technical problems of existing subway vehicle inspection processes, such as reliance on manual visual inspection, low detection efficiency, high missed detection rate, strong subjectivity in judgment standards, and difficulty in achieving continuous monitoring around the clock. Currently, with the continuous expansion of urban rail transit networks and the increasing density of train operations, the traditional model based on periodic shutdowns for manual inspection can no longer meet the demands for real-time and refined management of car body structural safety. Although some lines have introduced automatic detection equipment based on single vision or infrared imaging, limitations such as fixed viewing angles, changes in ambient lighting, occlusion interference, and the lack of a multi-modal information collaborative analysis mechanism result in insufficient accuracy in identifying typical damage types such as micro-cracks, local deformation, and loose connectors, especially exhibiting instability under complex backgrounds and dynamic operating conditions.

[0006] The technical solution of this invention is to construct an intelligent discrimination system integrating multi-source sensing, spatiotemporal alignment, and dual fusion of feature-level and decision-level technologies. This system is deployed along the track and on the vehicle platform in a subway depot, capable of omnidirectional scanning and damage assessment of the exterior surface of subway trains in motion or at rest. The system includes a front-end sensing subsystem, a data transmission and synchronization unit, an edge computing processing center, and a central diagnostic server. The front-end sensing subsystem consists of a first high-resolution industrial camera array deployed along both sides of the track, a top-down camera module, a bottom-up scanning camera assembly, and surrounding infrared thermal imaging sensors. Each sensor is installed according to a preset spatial geometry, covering key areas such as the side walls of the car body, the bogie area, the undercarriage suspension components, and the pantograph connection area on the roof. The data transmission and synchronization unit uses timestamps and trigger pulse signals to achieve precise frame-level synchronization of multiple video streams and thermal image data, ensuring strict temporal consistency of information collected from different perspectives at the same time. The edge computing processing center is deployed in a field cabinet and receives all raw data streams from the front end. It first performs image enhancement preprocessing, including dynamic range compression, dehazing correction, non-uniformity compensation, and motion blur suppression algorithms, to improve the amount of usable information under harsh imaging conditions.

[0007] Furthermore, the edge computing processing center incorporates a multi-view joint calibration model. Utilizing known sensor position and attitude parameters, it maps images from various perspectives to a unified three-dimensional reference coordinate system for the vehicle body, generating spatially aligned multi-channel feature maps. As one embodiment of the invention, the feature extraction stage employs a dual-branch convolutional neural network architecture. The first branch performs local detail enhancement convolution operations on the visible light image sequence, focusing on capturing spatial domain anomalies such as edge abrupt changes, texture breaks, and rust patches. The second branch performs temperature gradient field modeling on the infrared thermal image, identifying abnormal temperature rise areas caused by frictional overheating, poor electrical contact, or internal structural fatigue by calculating the heat flow direction divergence and heat accumulation rate within the neighborhood of each pixel. The high-level semantic features output from the two branches are concatenated and fed into a cross-modal attention fusion module. This module learns the correlation weight distribution between the visible light feature map and the thermal image, adaptively weighting and integrating the two types of heterogeneous information to generate a comprehensive damage confidence heatmap.

[0008] Furthermore, the system introduces a temporal evolution analysis mechanism to track and detect changes in the fusion results of multiple consecutive frames. For any candidate damage region, the system establishes its state transition sequence in the time dimension, predicts its spatial location evolution trend using Kalman filtering, and calculates the slope of the change in characteristic response intensity within the region. When a region exhibits a monotonically increasing abnormal response and a stable displacement trajectory for more than three consecutive frames, it is identified as a potential dynamic damage point. As one embodiment of the invention, the decision-level fusion unit receives preliminary discrimination results from various independent perspectives and a comprehensive score from the multi-perspective fusion output. It uses the Dempster-Shafer evidence theory framework for uncertainty reasoning, sets a basic probability assignment function to quantify the evidence strength based on three dimensions: local support, neighborhood consistency, and historical matching degree, and fuses multiple pieces of evidence through synthesis rules to finally output the globally optimal damage category determination result and its confidence level.

[0009] Furthermore, the central diagnostic server receives the judgment reports uploaded by all edge nodes and performs big data analysis across trains and time periods. The server has a built-in damage knowledge graph that compares the current detection results with data from historical archives on similar train models, operating conditions, and mileage segments to identify damage development trends and common patterns. Knowledge graph nodes contain metadata such as train model, manufacturing batch, service life, cumulative mileage, maintenance records, and climate exposure history. Edge relationships are defined as statistically significant association rules, such as "a certain type of aluminum alloy sidewall is prone to weld microcrack clusters after 5 years of operation in northern winters." Based on this, the system generates personalized health assessment reports and pushes them to the operation and maintenance management platform. As one embodiment of the invention, when the system detects damage confidence at the same location twice consecutively exceeding 15%, or when a single detection reaches the red warning threshold, it automatically triggers a maintenance work order generation process and suggests a priority inspection level and expected handling cycle.

[0010] Furthermore, the system possesses online self-learning capabilities, continuously optimizing model performance through a closed-loop feedback mechanism. Each manual verification result is fed back to the training database as a real label, used to incrementally update the weight parameters of the convolutional network. Simultaneously, the system monitors the working status and data quality of each sensor. When a camera's field of view is found to be contaminated or infrared thermography drifts, the system automatically adjusts the weight ratio of that input in the fusion process and issues a device maintenance reminder. As one embodiment of the invention, the entire system's discrimination logic operates within a hierarchical response framework. The first level is millisecond-level real-time screening, used to quickly filter out abnormal areas; the second level is second-level fine analysis, focusing on suspicious targets for multi-scale feature mining; and the third level is minute-level expert-mode reasoning, calling upon knowledge graphs and historical data for deep source tracing analysis to form a final conclusion.

[0011] Compared with the prior art, the advantages and positive effects of the present invention are as follows: This system overcomes the blind spots and occlusion limitations of single-view imaging by constructing a multi-view perception network covering the omnidirectional space of the vehicle body, achieving comprehensive scanning of damage on the outer surface of the subway car body. By introducing spatial alignment and temporal synchronization mechanisms, it ensures accurate registration of multi-source heterogeneous data in a unified coordinate system, providing a reliable data foundation for subsequent fusion analysis. Employing a dual fusion strategy at the feature level and decision level, it not only achieves complementary advantages between visible light details and infrared thermal features at the lower level but also processes uncertain information through evidence theory at the higher level, significantly improving the robustness and accuracy of damage identification in complex scenarios. The temporal evolution analysis mechanism endows the system with dynamic recognition capabilities, effectively distinguishing between instantaneous interference and real scalability defects, reducing false alarm rates. Cross-data association analysis based on knowledge graphs upgrades damage assessment from isolated event judgment to lifecycle trend prediction, greatly enhancing the foresight and scientific nature of operation and maintenance decisions. The entire system supports online model updates and equipment self-diagnosis, possessing long-term operational stability and adaptability, comprehensively solving the core pain points of traditional detection methods such as low efficiency, strong subjectivity, and incomplete coverage, providing strong technical support for the safe and efficient operation of urban rail transit. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the overall technical solution architecture proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of the dual fusion of feature level and decision level in this invention. Detailed Implementation

[0013] Example 1 Please refer to Figure 1 and Figure 2 This invention presents a multi-perspective fusion intelligent damage identification system for subway car bodies, aiming to address the technical shortcomings of traditional subway vehicle inspections, which rely on manual labor, are inefficient, have high false negative rates, suffer from subjective judgment criteria, and struggle to achieve continuous 24 / 7 monitoring. The system constructs a multi-source sensing network covering the omnidirectional space of the car body, combining a spatiotemporal alignment mechanism, a feature-level and decision-level dual fusion architecture, and a temporal evolution analysis model. This enables high-precision, low-false-report automatic identification of typical damage types on the outer surface of subway car bodies, such as micro-cracks, local deformation, loose connectors, and abnormal temperature rises. It also supports cross-cycle trend prediction and closed-loop self-learning optimization. In this embodiment, the system is deployed along the tracks of urban rail transit depot entry and exit lines and on the onboard auxiliary platform, serving high-intensity operation lines that operate for more than 20 hours per day, and possessing 24 / 7 uninterrupted operation capability.

[0014] The system's overall technical flow begins with multimodal data acquisition from the front-end perception subsystem. This data is then synchronized at the frame level via a data transmission and synchronization unit, before entering the edge computing processing center for image preprocessing, spatial coordinate mapping, dual-modal feature extraction, and cross-view fusion analysis, generating preliminary damage assessment results. These results, along with intermediate outputs from each independent viewpoint, are uploaded to the central diagnostic server. There, at a larger spatiotemporal scale, knowledge graph association and comparison, health trend modeling, and maintenance strategy generation are performed, ultimately forming a structured diagnostic report and triggering the operation and maintenance response mechanism. The entire system employs a layered, progressive processing framework, ensuring controllability and interpretability throughout the entire process, from millisecond-level real-time screening to minute-level expert reasoning.

[0015] The front-end sensing subsystem is the physical foundation for system information input, responsible for omnidirectional scanning of the exterior surface of the subway train, whether in motion or stationary. The subsystem consists of a high-resolution industrial camera array deployed along both sides of the track, a top-down camera module, a bottom-up scanning camera assembly, and surrounding infrared thermal imaging sensors. All sensors are fixedly installed on the trackside protective supports and maintenance platform according to pre-calibrated spatial geometry, ensuring blind-spot-free coverage of key structural areas such as the side walls of the train body, bogie areas, undercarriage suspension components, and the pantograph connection area on the roof. The first high-resolution industrial camera array is symmetrically distributed along both sides of the track, with each side containing six CMOS area array cameras with a resolution of 5120×3840 pixels and a frame rate of 30 frames per second. The lenses have a zoom range of 12mm to 50mm, an aperture of F2.8, and operate in the visible light band from 400nm to 700nm, capturing surface texture details in areas such as the side walls of the train body, door and window seals, and welded joints. The camera array uses a line-by-line trigger shooting mode, and uses the laser displacement sensor to detect the time when the wheel passes as the initial trigger signal to ensure that the first frame image accurately corresponds to the position of the front of the car.

[0016] The top-view camera module is installed on the gantry structure above the track and consists of three wide-angle fisheye cameras, each with an effective pixel count of 3072×2048 and a field of view of 180 degrees, covering the entire top plane of the train. It focuses on monitoring conditions such as pantograph wear, aging of high-voltage cable insulation, damage to the air conditioning unit casing, and blockage of drainage holes. The bottom-view scanning camera assembly is located in the maintenance pit below the track. It uses a linear scan method and is equipped with two sets of eight linear scanning cameras, each with a resolution of 1024×1 pixel and a scanning frequency of 20,000 lines per second. With the help of a high-frequency pulsed LED light source, it can acquire continuous high-definition strip images of hidden areas such as the undercarriage structure, brake lines, traction motor mounts, and gearbox housing when the train passes at low speed. After stitching, these images form a complete panoramic view of the undercarriage.

[0017] Nine observation points are set up for the infrared thermal imaging sensor, located at the center of both sides of the track, the four corners of the top gantry, and both ends of the bottom maintenance pit. It employs an uncooled vanadium oxide microradiometer detector, operating in the 8-14 micrometer long-wave infrared band, with a spatial resolution of 640×512 pixels, a temperature sensitivity better than 0.05 degrees Celsius, and a temperature measurement range of -20 degrees Celsius to +300 degrees Celsius. The sensor collects real-time data on the surface temperature distribution of various parts of the vehicle body, paying particular attention to the thermal behavior of heat-generating components such as bogie axle box bearings, brake disc friction surfaces, electrical terminal blocks, and traction inverter heat sinks. All sensors are equipped with IP68-rated protective housings and built-in constant-temperature heating and compressed air purging devices to prevent rain, snow, and dust from affecting image quality. They are connected to the field cabinet via electromagnetic interference-resistant shielded cables.

[0018] The data transmission and synchronization unit is responsible for the aggregation, encapsulation, and time consistency assurance of multi-source heterogeneous data streams. The core task of this unit is to ensure strict temporal alignment of data from different locations and types of sensors as the train passes through the detection zone at speeds from 0 to 80 km / h, preventing misalignment in subsequent fusion analysis due to differences in acquisition delays. To this end, the system establishes a unified time reference system, with all sensor nodes connected to a network time synchronization system based on the IEEE 1588 precision time protocol, and clock synchronization accuracy controlled within ±1 microsecond. Each video frame or heatmap data packet is immediately timestamped upon generation, in the format of year-month-day-hour-minute-second-millisecond-microsecond, and stored in the metadata field of the data header.

[0019] In addition, the system employs a hardware-level triggering mechanism. A laser photoelectric switch installed at the track start point detects the arrival time of the train's first axle, generating a global synchronization trigger pulse signal. This signal is broadcast to all front-end devices via a dedicated coaxial cable, forcibly initiating the synchronous acquisition process. For area scan cameras and thermal imagers, the trigger signal directly controls the start of exposure; for line scan cameras, the scanning frequency is dynamically adjusted based on vehicle speed feedback to ensure a strict match between the scanning rate and the train's speed, guaranteeing images without stretching or distortion. All raw data is uploaded to the edge computing processing center via a gigabit Ethernet interface, using the RTSP real-time streaming protocol for transmission. Simultaneously, the UDP protocol's forward error correction coding mechanism is enabled, maintaining a data packet reception integrity rate of over 99.9% even under network jitter conditions.

[0020] The edge computing processing center is deployed in a field rack near the detection area. It employs an industrial-grade ruggedized server cluster architecture, comprising two main control computing nodes and one redundant backup node. Each node is equipped with dual Intel Xeon Silver 4310 processors, 512GB of DDR4 memory, four NVIDIA A40 GPU accelerator cards, and a total of 80TB of solid-state storage array, supporting concurrent processing of 16 high-definition video streams and 9 infrared thermal image streams. After receiving all raw data streams from the front end, the center first performs image enhancement preprocessing to increase the amount of usable information under harsh imaging conditions and eliminate the influence of environmental interference factors.

[0021] The preprocessing module integrates multiple adaptive algorithms. For visible light images, dynamic range compression technology is applied, employing a hybrid method combining bilateral filtering and histogram equalization to suppress overexposure in areas of strong light reflection while enhancing detail visibility in shadow areas. The dehazing correction algorithm, based on dark channel prior theory, estimates scene transmittance and atmospheric light values ​​to reconstruct a clear image, particularly suitable for the white fog effect caused by sudden changes in illumination at tunnel exits. Motion blur suppression uses a Wiener deconvolution and deep learning joint deblurring network, taking multiple frames as input to reconstruct a sharp instantaneous imaging result, effectively mitigating the motion blur problem caused by high-speed passage. For infrared thermal images, a non-uniformity compensation algorithm is executed, using a blackbody reference source to periodically calibrate the response differences of each pixel, eliminating fixed pattern noise caused by manufacturing tolerances of the detector array. Simultaneously, an environmental temperature drift compensation model is introduced, dynamically correcting the temperature measurement benchmark based on real-time temperature data provided by outdoor weather stations to ensure consistency across day and night and across seasons.

[0022] After preprocessing, the system enters the multi-view joint calibration stage. The goal of this stage is to uniformly map images from all views onto the vehicle's three-dimensional reference coordinate system, achieving spatial alignment. The system pre-determines the precise position and attitude parameters of all sensors relative to the track centerline using an offline calibration program, including three-dimensional coordinates, pitch angle, yaw angle, and roll angle. The calibration process uses a checkerboard calibration board combined with the Levenberg-Marquardt nonlinear optimization algorithm, controlling the reprojection error to within 0.5 pixels. During online runtime, the system calls the stored intrinsic and extrinsic parameter matrices to construct a projection transformation model, mapping each pixel back to the world coordinates of the vehicle's surface. For occluded areas that cannot be directly observed (such as the wheel-occluded portion of the vehicle's undercarriage), the system employs a hole-filling strategy based on a digital twin model, using historical unoccluded data and geometric constraints to infer the potential surface morphology and generate a completed multi-channel feature map.

[0023] The feature extraction stage employs a dual-branch convolutional neural network architecture, processing visible light image sequences and infrared thermal image data separately to mine complementary damage features in the two modalities. The first branch performs local detail enhancement convolution operations on the visible light images. The network structure is evolved from an improved ResNet-50 backbone network. The backbone retains the first four residual stages, while the fifth stage is replaced with three sets of dilated convolutional modules with dilation rates of 2, 4, and 6, respectively, to expand the receptive field without sacrificing spatial resolution. A pyramid pooling structure is added at the end of the network to fuse multi-scale contextual information, along with an edge-aware attention submodule. This submodule pre-calculates the image gradient magnitude and direction distribution using the Sobel operator and histogram of oriented gradients, guiding the network to focus on spatial domain anomalies such as edge abrupt changes, texture breaks, and rust patches. During training, a transfer learning strategy is used. The initial weights are loaded with an ImageNet pre-trained model, and then fine-tuned using a subway car image database labeled with defect samples such as cracks, dents, and paint peeling. The loss function is jointly optimized using weighted cross-entropy and Dice coefficients to improve the detection sensitivity for small targets.

[0024] The second branch is dedicated to infrared thermal image processing, with its core focus on temperature gradient field modeling. The input image is the original thermal image after non-uniformity compensation, and the pixel values ​​represent the actual temperature readings. The network structure adopts a U-Net encoder-decoder architecture, where the encoder extracts global heat distribution features, and the decoder gradually recovers spatial details. Temperature gradient field analysis is performed on the feature map, specifically calculating the heat flow direction divergence and heat accumulation rate in the neighborhood of each pixel. The heat flow direction divergence reflects the divergence or convergence characteristics of local heat sources, and the calculation formula is as follows: in, Indicates the location The heat flow vector at that point is determined by the temperature field. The negative gradient is defined as follows: , This represents the material's thermal conductivity. A positive value indicates heat diffusion outwards, commonly seen in areas of normal heat dissipation; a negative value indicates heat accumulation inwards, suggesting potential internal friction or increased contact resistance at heating points. The heat accumulation rate is obtained by comparing the average temperature change rate of the same area in three consecutive thermal images, measured in degrees Celsius per second, used to identify urgent hazards caused by rapid temperature rise. The high-level semantic feature maps output by both branches are 256×192×128 pixels in size, representing the anomalous response intensity distribution in the visible and infrared domains, respectively.

[0025] The two feature maps are fed into the cross-modal attention fusion module for feature-level fusion. This module is designed as an adaptive weighted integration mechanism that learns the semantic association weight distribution between the visible light feature map and the heatmap, thereby generating a comprehensive damage confidence heatmap at the pixel level. The module structure includes two sub-units: channel attention and spatial attention. The channel attention sub-module compresses the spatial dimension through global average pooling, obtaining two 1×1×128 feature vectors. These vectors are then reduced to 64 dimensions by a shared fully connected layer and then increased back to 128 dimensions, outputting channel weight coefficients to adjust the importance of each feature channel. The spatial attention sub-module performs max pooling and average pooling along the channel axis on the concatenated 256×192×256 feature map, obtaining two 256×192×1 feature maps. These are then concatenated and convolved with a 7×7 kernel to generate a spatial attention map, highlighting the location of key regions.

[0026] Finally, the original feature map is multiplied element-wise with the weighted attention map to obtain the fused feature representation. This representation not only preserves the clear geometric contour information in the visible light image but also incorporates potential thermal anomaly clues revealed in the infrared image. For example, a loose bolt connection may not have caused obvious deformation yet, but it has generated a local temperature rise under continuous vibration and friction. In this case, the fused result will show a higher comprehensive confidence peak at that location than any single mode. The fused feature map is further normalized to the 0-1 interval using the sigmoid activation function to form an initial comprehensive damage confidence heatmap, which serves as the basic input for subsequent time-series analysis.

[0027] The system incorporates a temporal evolution analysis mechanism to track trajectories and detect changes in the fused results of multiple consecutive frames, endowing the discrimination logic with dynamic recognition capabilities. For any candidate damage region, the system establishes its state transition sequence in the time dimension, predicts its spatial location evolution trend using Kalman filtering, and calculates the slope of change in the characteristic response intensity within the region. Specifically, the system maintains a dynamic target list, recording the location, size, shape, and response value of newly discovered anomalous regions with a confidence level exceeding 0.7 in each frame. Cross-frame target matching is achieved through the Hungarian algorithm, and the association cost matrix is ​​calculated based on a weighted average of three indicators: center distance, IoU (Intersection over Union), and response similarity, achieving stable tracking.

[0028] A successfully matched target enters the state update process, where a Kalman filter based on a discrete constant velocity motion model is applied to estimate and predict its two-dimensional position and velocity vectors. The filter state vector is defined as follows: The observation input is the center coordinates of the bounding box detected in the current frame. The prediction step calculates the current position based on the state of the previous moment, while the update step merges the predicted value with the actual observation value to reduce the impact of random fluctuations. For targets that have been successfully tracked for more than 3 consecutive frames, the system calculates the time series first-order difference mean of its confidence response intensity, i.e., the slope of change. When the slope is greater than 0.05 and the standard deviation of the displacement trajectory is less than 2 pixels, it is identified as a potential dynamic damage point, indicating that the anomaly is continuously expanding and its position is fixed, excluding instantaneous interference such as birds flying by or water reflections.

[0029] The decision-level fusion unit receives preliminary judgment results from various independent perspectives and a comprehensive score from the multi-view fusion output. It employs the Dempster-Shafer evidence theory framework for uncertainty reasoning, outputting the globally optimal damage category determination and its confidence level. The unit treats each perspective as an independent source of evidence, including six basic inputs: left camera group, right camera group, top camera, bottom line scan, infrared array, and multi-view fusion channel. For each candidate damage target, the system assigns a recognition framework Θ={crack, deformation, loosening, temperature rise, corrosion, other}, representing the set of all possible damage types.

[0030] The basic probability assignment function quantifies the strength of evidence based on three dimensions: local support, neighborhood consistency, and historical matching. Local support measures the similarity between the target region and various damage templates from this perspective, obtained by transforming the softmax probability distribution output by a pre-trained fine-grained classification network. Neighborhood consistency assesses the response consistency of pixels surrounding the target, calculating the reciprocal of the local variance of its confidence map; a smaller variance indicates a clearer boundary and more uniform internal structure of the anomalous region, resulting in higher evidence credibility. Historical matching queries the detection records of this location over the past 7 days; if similar anomalies have occurred multiple times, they are assigned higher weights, reflecting the continuity of damage. The three factors are weighted, summed, and normalized to form the basic probability allocation. ,in Indicates the first One source of evidence, It is a non-empty subset of Θ.

[0031] The synthesis rule employs orthogonal summation to fuse multiple pieces of evidence, iteratively calculating until all evidence is combined. The final basic probability assignment is interpreted using the centroid method to determine the most probable single injury type, and its confidence level is calculated. A confidence level below 0.6 is marked as "pending review," above 0.85 as "confirmed," and between these levels as "suspected." All judgment results, accompanied by spatiotemporal labels, confidence values, a list of evidence sources, and original image slices, are packaged into a structured message and uploaded to the central diagnostic server.

[0032] The central diagnostic server receives diagnostic reports uploaded by all edge nodes and performs big data analysis across trains and time periods. Deployed in the depot's data center, the server employs a distributed Kubernetes container orchestration architecture, running multiple microservice instances to collaboratively process massive amounts of diagnostic data. The server has a built-in damage knowledge graph, serving as the core carrier for long-term memory and the reasoning engine. The knowledge graph is stored using an attribute graph model, with node types including "vehicle model," "manufacturing batch," "service years," "cumulative mileage," "maintenance records," "climate and environmental exposure history," "part number," and "damage mode." Edge relationships are defined as statistically significant association rules, such as "a certain type of aluminum alloy sidewall is prone to weld microcrack clusters after 5 years of operation in northern winters."

[0033] The graph construction process is divided into two stages: static initialization and dynamic evolution. The initialization stage imports technical files, historical maintenance records, and environmental service data of all in-service trains across the entire railway network to form an initial topology. The dynamic evolution stage continuously updates the graph content through an online learning mechanism. Whenever a new diagnostic result is entered, the system executes the Node2Vec graph embedding algorithm to vectorize the relevant nodes and calculates their semantic similarity to existing damage patterns. If the similarity exceeds a threshold of 0.82, the association discovery process is triggered, calling the Pearson correlation coefficient and Granger causality test to analyze whether a statistical dependency exists between the two. Once verified, a weighted directed edge is added to the graph, with the weight value equal to the logarithmic transformation of the reciprocal of the p-value, reflecting the association strength.

[0034] The system generates a personalized health assessment report based on this information and pushes it to the operation and maintenance management platform. The report includes a current detection summary, historical trend curves, comparisons with similar cases, risk level assessment, and handling recommendations. When the system detects damage at the same location twice consecutively with a confidence level increase exceeding 15%, or when a single detection reaches the red alert threshold (confidence level ≥ 0.92), it automatically triggers the maintenance work order generation process. The work order includes the three-dimensional coordinates of the fault location, recommended inspection methods (such as magnetic particle testing or ultrasonic testing), recommended priority inspection level (Level 1: immediate shutdown, Level 2: after the last shift of the day, Level 3: within the weekly plan), estimated handling cycle (in man-hours), and a spare parts requirement list. After manual review and confirmation, the work order is issued to the corresponding work team for closed-loop management.

[0035] The system possesses online self-learning capabilities, continuously optimizing model performance through a closed-loop feedback mechanism. Each manual review and confirmation result is fed back to the training database as a true label, used for incrementally updating the weight parameters of the convolutional network. The specific process is as follows: Every morning, the system automatically extracts all samples marked "pending review" and "confirmed" from the previous day, along with their manually corrected labels, to form an incremental training set. An elastic weight solidification algorithm is used for model fine-tuning, absorbing new sample features while retaining existing knowledge, preventing catastrophic forgetting. After the new model is trained and its performance is verified through A / B testing, it is gradually replaced with the old online version through a canary release strategy.

[0036] Simultaneously, the system monitors the operating status and data quality of each sensor. When a camera's field of view is found to be contaminated or infrared thermography drifts, the system automatically adjusts the weight ratio of that input in the fusion process and issues a device maintenance reminder. The quality assessment module calculates the image sharpness (based on the Tenengrad gradient function), contrast, signal-to-noise ratio, and effective coverage of each video stream in real time. If 10 consecutive frames are below a set threshold, it is considered degraded. For degraded channels, the system reduces their attention weight to 30% of its original value during feature-level fusion and lowers their evidence credibility rating during decision-level fusion. At the same time, it sends an SNMP trap alarm to the device management system, prompting the scheduling of cleaning or calibration operations.

[0037] The entire system's discrimination logic operates within a hierarchical response framework. The first level is millisecond-level real-time screening, used to quickly filter out anomaly-free regions. This level deploys a lightweight YOLOv5s model on a GPU inference engine, scanning the entire image at 200 frames per second, retaining only candidate regions with a confidence level higher than 0.5 for the next level of processing, reducing unnecessary computational load. The second level is second-level fine-grained analysis, focusing on suspicious targets for multi-scale feature mining, calling the aforementioned dual-branch network and cross-modal fusion module for deep inference, taking approximately 1.2 seconds per target. The third level is minute-level expert-mode inference, calling the knowledge graph and historical data for deep source tracing analysis to form a final conclusion, with an average response time of 3.5 minutes per train. This three-level linkage mechanism ensures both real-time requirements and high-precision discrimination, meeting the needs of large-scale deployment and application.

[0038] This embodiment, through the systematic integration of the aforementioned technical means, constructs a complete closed-loop discrimination system encompassing data acquisition, preprocessing, feature extraction, multimodal fusion, temporal analysis, and knowledge reasoning. The system is no longer limited to isolated judgments of single-frame images, but comprehensively utilizes redundant information from multiple spatial perspectives, temporal evolution patterns, and historical experience, achieving a leap in damage identification from "visible" to "accurate," and from "fast judgment" to "deep thinking." Actual deployment data shows that during a 6-month trial operation, the system achieved a 98.7% detection rate for surface cracks longer than 2 mm, and an average early warning time of 14.3 days for early loosening and temperature rise-related hazards. This represents a 76% reduction in missed detection rates compared to traditional manual inspections, and the inspection time per train was reduced from 45 minutes to 8 minutes, significantly improving the automation, intelligence, and scientific level of subway vehicle maintenance.

[0039] Existing technologies for subway car inspection generally employ single-view visual inspection or independent infrared temperature measurement schemes, lacking a multi-source information coordination mechanism. Such systems are vulnerable to complex background interference, partial occlusion, or weak contrast defects, easily generating false alarms or missed alarms. The fundamental difference in this solution lies in constructing a multi-view perception network covering the omnidirectional space of the car body and designing a dual-fusion architecture at the feature level and decision level. The former achieves complementary advantages between visible light details and infrared thermal features at the lower level, while the latter processes uncertain information through evidence theory at the higher level, forming a robust joint discrimination capability. More importantly, the system introduces a temporal evolution analysis mechanism, which can distinguish between instantaneous interference and real scalability defects, fundamentally solving the high false alarm problem caused by static snapshot-based detection in traditional methods. This series of technological innovations constitutes the essential characteristics and technological advancements that distinguish this invention from existing technologies.

[0040] The Dempster-Shafer evidence theory framework adopted by the decision-level fusion unit provides a powerful mathematical tool for handling uncertainty and incomplete information. Unlike traditional Bayesian methods that require pre-setting prior probabilities, this framework allows "unknown" to be represented as an explicit state, making it more suitable for integrating multi-source heterogeneous evidence in complex industrial scenarios. In this embodiment, the framework is implemented as follows: First, an identification framework Θ is defined, encompassing all possible damage types; then, a basic probability assignment function is constructed for each evidence source, reflecting its support for each hypothesis; finally, evidence is synthesized through orthogonal sums and rules to derive the global confidence distribution.

[0041] The design of the basic probability assignment function fully considers the practical constraints of industrial inspection. Local support is derived from the output probability of the deep learning model, but it is not directly based on the softmax result. Instead, it undergoes reliability correction through temperature scaling and Platt calibration to make it closer to the true confidence level. Neighborhood consistency is obtained by calculating the inverse of the standard deviation of the confidence level of pixels within the candidate region. The smaller the standard deviation, the more concentrated the response and the more reliable the evidence. Historical matching degree queries whether a similar anomaly has occurred at this spatial location in the past period. If so, its basic probability value is increased to reflect the continuous pattern of damage development. The three factors are weighted and summed with weights of 0.5, 0.3, and 0.2, and then normalized to form the final basic probability assignment.

[0042] The evidence synthesis process employs the classic Dempster combinatorial rule: in, For the proposition after synthesis The basic probability distribution, and These represent the first and second sources of evidence for their respective propositions. and The basic probability distribution, In the summation term and There are two sets whose intersection is exactly equal to the target set. , As the normalization factor, and Basic probability assignment for two independent sources of evidence. The conflict coefficient represents the degree of contradiction between pieces of evidence, and is calculated using the following formula: .when When the value is too large, it indicates serious conflict of evidence. The system will reject simple synthesis and instead activate the conflict arbitration mechanism, calling on a third-party perspective or historical data for adjudication. The synthesized basic probability assignment is interpreted by the centroid method into the most probable single damage type, and its confidence level is calculated as the final output.

[0043] By introducing the Dempster-Shafer evidence theory, this embodiment constructs a robust reasoning mechanism capable of handling ambiguous, conflicting, and incomplete information. The system can not only integrate positive evidence from different perspectives but also identify and manage contradictory information, avoiding overall judgment errors caused by false alarms from individual sensors. This design significantly improves the system's fault tolerance and decision reliability in complex real-world environments, providing a novel technological paradigm for rail transit safety monitoring.

[0044] Example 2 The specific implementation of the temporal evolution analysis mechanism is as follows: establish a time series state model of the candidate damage area, apply Kalman filtering for trajectory prediction, and calculate the changing trend of characteristic response intensity to distinguish between real damage and transient disturbance.

[0045] In this embodiment, the temporal evolution analysis mechanism is elaborated in detail. The aim is to provide a damage evolution identification method based on dynamic modeling, enabling the system to distinguish between short-term disturbances and long-term development trends, thereby significantly reducing the false alarm rate.

[0046] The core logic of the mechanism is to construct a state-space model of the candidate target. Whenever a new anomalous region with a confidence level exceeding a threshold is detected in a frame, the system creates a target instance for it and initializes its state vector. The state vector includes position coordinates, velocity, acceleration, bounding box size, average confidence level, and timestamp. The system maintains a scrolling window with a length of 10 frames to store the historical state records of the target. Cross-frame association uses an improved IOU matching algorithm, combining appearance feature descriptors (generated from the pooling output of the last layer of ResNet-18) with motion prediction results for joint judgment, improving tracking stability under occlusion conditions.

[0047] Successfully matched targets enter the Kalman filter processing flow. The filter employs a two-dimensional constant acceleration motion model, and the state vector is defined as follows: The process noise covariance matrix is ​​set based on statistical data of train operation stability, while the observation noise covariance is configured based on the measured results of image detection accuracy. During each frame processing, a prediction step is first executed, calculating the current position based on the previous state and dynamic equations; then an update step is executed, using the actual detected bounding box center as the observation input to correct the prediction error. The filtered output position trajectory is smooth and continuous, effectively suppressing detection jumps.

[0048] For stable targets that exist continuously for more than 5 frames, the system extracts their average confidence time series, applies linear least squares to fit a trend line, and calculates the slope s. If s > 0.05 and the coefficient of determination R² > 0.8, it is judged as a significant increasing trend, supporting the "dynamically expanding damage" hypothesis. Simultaneously, a trajectory stability index is calculated, defined as the standard deviation of the position prediction residual. If it is less than 3 pixels, the target position is considered fixed, excluding interference from external objects moving with the vehicle.

[0049] When both of the above conditions are met simultaneously, the system prioritizes the target, adds it to the key attention list, and automatically extracts five frames of image sequences before and after it to generate a dynamic diagnostic package for manual review. In actual operation, this mechanism effectively identified multiple early bearing wear events, characterized by a continuous temperature rise trend in the bearing box area over several days, but the daily change did not reach the alarm threshold. Traditional static detection methods cannot capture this slow evolution process.

[0050] By employing an evolutionary identification mechanism based on Kalman filtering and time-series trend analysis, this embodiment enables the system to possess "proactive" diagnostic capabilities. Unlike existing technologies that rely solely on passive response modes based on single-frame threshold judgments, this solution can proactively detect potential faults in their nascent stages, achieving a shift from "post-event handling" to "pre-event prevention." This improvement significantly enhances the scientific rigor and proactiveness of subway vehicle health management, demonstrating substantial engineering application value.

Claims

1. A multi-view fusion intelligent damage identification system for subway car bodies, characterized in that, include: The front-end sensing subsystem is used to simultaneously collect multimodal sensing data of the subway car body surface from multiple preset spatial orientations; The data transmission and synchronization unit is used to apply a timestamp to the multimodal sensing data and achieve frame-level time synchronization based on a hardware-triggered pulse signal to generate a spatiotemporally aligned data stream. The edge computing processing center is used to receive the spatiotemporally aligned data stream, perform image enhancement preprocessing, and map the data from each viewpoint to a unified vehicle body three-dimensional reference coordinate system based on the pre-stored sensor pose parameters to generate a spatially aligned multi-channel feature map. The edge computing processing center has a built-in dual-branch feature extraction network and a cross-modal attention fusion module. The dual-branch feature extraction network includes a first branch and a second branch. The cross-modal attention fusion module is used to adaptively and weightedly integrate the high-level semantic features output by the first branch and the second branch to generate a comprehensive damage confidence heatmap. The decision-level fusion unit is used to receive the preliminary judgment results from various independent perspectives and the comprehensive damage confidence heatmap, and to perform uncertainty reasoning using the Dempster-Shafer evidence theory framework. It measures the evidence strength of each evidence source based on local support, neighborhood consistency and historical matching, and outputs the global optimal damage category judgment result and its confidence level through evidence synthesis rules. The central diagnostic server receives the discrimination reports uploaded by all edge nodes, performs big data analysis across trains and time periods, and generates personalized health assessment reports based on the built-in damage knowledge graph.

2. The multi-view fusion intelligent damage identification system for subway car bodies according to claim 1, characterized in that, The first branch is used to perform local detail enhancement convolution operation on the visible light image sequence to capture spatial domain anomalous features. The second branch is used to perform temperature gradient field modeling on the infrared thermal map to identify anomalous temperature rise regions. The edge computing processing center also has a built-in temporal evolution analysis module, which is used to track and detect changes in the comprehensive damage confidence heat map of multiple consecutive frames. When a candidate damage region exhibits a monotonically increasing anomalous response and a stable displacement trajectory in multiple consecutive frames, it is determined to be a potential dynamic damage point. The front-end sensing subsystem includes a high-resolution industrial camera array deployed along both sides of the track, a top-down camera module, a bottom-up scanning camera assembly, and an infrared thermal imaging sensor arranged around it. The high-resolution industrial camera array, the top-down camera module, and the bottom-up scanning camera assembly are used to acquire visible light images, and the infrared thermal imaging sensor is used to acquire infrared thermal maps.

3. The multi-view fusion intelligent damage identification system for subway car bodies according to claim 2, characterized in that, The data transmission and synchronization unit is connected to a network time synchronization system based on the IEEE 1588 precision time protocol, and generates a global synchronization trigger pulse signal through a laser photoelectric switch installed at the starting point of the track.

4. The multi-view fusion intelligent damage identification system for subway car bodies according to claim 2, characterized in that, The image enhancement preprocessing includes dynamic range compression, dehazing correction, and motion blur suppression for visible light images, as well as non-uniformity compensation and ambient temperature drift compensation for infrared thermal images.

5. The multi-view fusion intelligent damage identification system for subway car bodies according to claim 2, characterized in that, The cross-modal attention fusion module includes a channel attention subunit and a spatial attention subunit. The channel attention subunit is used to adjust the importance weights of each feature channel, and the spatial attention subunit is used to generate a spatial attention map that highlights the location of key regions.

6. The multi-view fusion intelligent damage identification system for subway car bodies according to claim 2, characterized in that, The temporal evolution analysis module is specifically used for: creating target instances for newly discovered candidate damage regions in each frame; achieving cross-frame target matching using the Hungarian algorithm; and applying a Kalman filter to perform state estimation and trajectory prediction for successfully matched targets. Calculate the slope of the characteristic response intensity change and trajectory stability index of the target region within multiple consecutive frames.

7. The multi-view fusion intelligent damage identification system for subway car bodies according to claim 2, characterized in that, Under the Dempster-Shafer evidence theory framework, the decision-level fusion unit regards each independent perspective and the comprehensive damage confidence heatmap as independent evidence sources, and defines an identification framework that includes all possible damage types for each candidate damage target.

8. The multi-view fusion intelligent damage identification system for subway car bodies according to claim 7, characterized in that, The local support is obtained by correcting the reliability of the output probability of the fine-grained classification network. The neighborhood consistency is obtained by calculating the reciprocal of the standard deviation of the confidence of pixels within the candidate region. The historical matching degree is obtained by querying the historical detection records of the spatial location within a preset time period.

9. The multi-view fusion intelligent damage identification system for subway car bodies according to claim 2, characterized in that, The damage knowledge graph built into the central diagnostic server is stored using an attribute graph model. Its nodes include vehicle model, manufacturing batch, service life, cumulative mileage, maintenance records, and climate and environmental exposure history. Its edge relationships are defined as association rules with statistical significance.

10. The multi-view fusion intelligent damage identification system for subway car bodies according to claim 2, characterized in that, The system also has online self-learning capabilities. Each manual verification result is sent back to the training database as a real label to incrementally update the weight parameters of the dual-branch feature extraction network. At the same time, the system monitors the data quality of each sensor. When data degradation is detected, the system automatically adjusts the weight ratio of that input in the cross-modal attention fusion module and the decision-level fusion unit, and issues a device maintenance reminder.

Images