A rail transit defect detection method based on multi-source perception and causal decoupling

Abstract

The application discloses a kind of track traffic defect detection methods based on multi-source perception and causal decoupling. By synchronously acquiring high-resolution RGB image and structural micro-vibration signal of track inspection scene, four-dimensional perception data field containing component dynamic mechanical response characteristics is constructed, and dynamic mechanical response field representing the stiffness of structure at each spatial position is generated;Instance segmentation is performed on the static topography component in the RGB image and the four-dimensional data field, and a topological relationship graph is constructed with components as nodes and functional coupling relationships as edges;According to the topological relationship graph, double abnormal reasoning at node level and edge level is performed, and combined with generative deduction based on causal decoupling and cross-modal topological consistency verification, the final comprehensive defect judgment result is generated.The application improves the recognition accuracy and robustness of track traffic defect detection on subtle defects such as micro-looseness and functional disorder.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection of track defects, and in particular to a method for detecting track defects based on multi-source sensing and causal decoupling. Background Technology

[0002] As a backbone mode of modern urban and intercity transportation, rail transit's operational safety directly impacts public safety and socio-economic efficiency. During long-term service, core rail components such as rails, fasteners, and track beds endure the dual effects of repeated dynamic loads from trains and environmental erosion, inevitably leading to various defects such as cracks, spalling, wear, loosening, and settlement. Accurate and timely detection of these defects is a critical technical requirement for ensuring the safe operation of rail transit.

[0003] Currently, track defect detection technology is evolving in two main directions. The first direction is based on two-dimensional vision-based detection methods. Early solutions relied on manual inspection and simple image processing algorithms to identify obvious cracks and spalling on the rail surface through edge detection, threshold segmentation, and other means. In recent years, deep learning models, represented by convolutional neural networks and visual Transformers, have been introduced. Supervised learning is used to train defect classifiers and object detectors on large-scale labeled datasets, which has improved detection accuracy to some extent. Large-scale vision-language pre-trained models, represented by CLIP, are also being explored for application in industrial defect detection, giving the models a certain zero-shot recognition capability through image-text semantic alignment. However, pure vision methods have inherent limitations: the information provided by two-dimensional images is limited to the surface texture and color of components, and cannot perceive three-dimensional geometric deformations; when faced with functional defects characterized by abnormal spatial pose, such as loose fasteners, structural offsets, and base surface depressions, pure vision solutions are often powerless. In addition, complex working conditions such as uneven lighting in tunnels, oil stains, and periodic fish-scale patterns on the rail surface make it easy for discrimination based solely on texture information to generate a large number of false alarms and missed detections.

[0004] The second direction is 3D detection methods that integrate depth information. The introduction of 3D sensors such as structured light cameras and LiDAR enables the system to acquire high-precision depth maps or point cloud data of the track surface. Existing technical solutions typically perform simple channel stitching of RGB images and depth maps, or use dual-stream convolutional networks to extract features separately and then stitch and fuse them to achieve joint utilization of appearance and geometric information. Some solutions use depth maps for background filtering to remove interference from non-interested areas such as the track bed. However, these methods generally remain at the level of using depth information as an auxiliary filtering tool or simple feature stacking, without elevating the depth geometric modality to an independent semantic modeling status equivalent to the visual texture modality. They lack a unified cross-modal alignment mechanism among RGB texture, depth geometry, and defect text semantics, and cannot use natural language descriptions to guide the model's semantic understanding of defects and zero-shot generalization.

[0005] More importantly, both of the above-mentioned schemes are based on a static perception paradigm, which extracts static texture and geometric features from single frames or independent multi-frame data. This paradigm ignores the essential attributes of track defects, especially hidden defects such as loose fasteners, unsupported sleepers, and early microcracks: these defects often exhibit abnormal dynamic responses that differ from normal components under the micro-vibration excitation before and after train passage. Existing static detection methods cannot capture the structural health information contained in these dynamic responses, resulting in extremely limited detection capabilities for early-stage, invisible defects without significant appearance changes.

[0006] Furthermore, the scarcity of track defect annotation data and the dynamic growth of defect types constitute another practical bottleneck. In actual operating lines, real samples of certain defect types are extremely rare, while new defects may appear at any time as the operating environment changes. Existing deep learning methods heavily rely on large-scale labeled data; when faced with new defects with only a few samples, the model must be retrained, making rapid and low-cost incremental adaptation impossible.

[0007] In summary, existing technologies struggle to simultaneously cover both surface texture defects and three-dimensional geometric deformation defects, lack a structured understanding of the functional coupling relationship of components, have not incorporated dynamic mechanical response information into the perception dimension, and exhibit insufficient generalization ability when faced with scarce annotations and open expansion of defect types. Therefore, a novel detection method is urgently needed that integrates dynamic mechanical perception, topological relationship reasoning, and generative cognitive deduction. Summary of the Invention

[0008] The purpose of this invention is to provide a rail transit defect detection method based on multi-source sensing and causal decoupling, so as to solve the problems existing in the prior art.

[0009] Technical solution: A method for detecting defects in rail transit based on multi-source sensing and causal decoupling, comprising the following steps: Acquire multi-source synchronous sensing data in the track inspection scenario, including RGB images and synchronously acquired structural micro-vibration signals; Based on multi-source synchronous sensing data, a four-dimensional sensing data field containing the dynamic mechanical response characteristics of components is constructed, and a dynamic mechanical response field characterizing the structural stiffness at each spatial location is generated. Instance segmentation is performed on the static shape components in RGB images and four-dimensional perception data fields to construct a topological relationship graph representing the functional coupling relationship between components. The nodes of the topological relationship graph store the multimodal absolute attributes of the components, and the edges store the relative pose and connection tightness relationship attributes between the components. Based on the topology graph, dual anomaly reasoning at the node and edge levels is performed, and combined with generative inference based on causal decoupling and cross-modal topology consistency verification, the final comprehensive defect judgment result is generated.

[0010] The beneficial effects are as follows: By expanding static visual perception into an active cognitive paradigm that integrates dynamic mechanical response, topological reasoning and causal deduction, this invention achieves highly sensitive detection of hidden functional defects such as loose fasteners and zero-sample generalization identification of unknown combination defects. Attached Figure Description

[0011] Figure 1 This is a flowchart of the overall solution of the present invention.

[0012] Figure 2 This invention is a flowchart of constructing a four-dimensional sensing data field containing the dynamic mechanical response characteristics of components and generating a dynamic mechanical response field characterizing the structural stiffness at each spatial location.

[0013] Figure 3 This is a flowchart of the topological relationship diagram that represents the functional coupling relationship between components in this invention.

[0014] Figure 4 This is a flowchart of the dual anomaly reasoning at the node and edge levels in this invention.

[0015] Figure 5 This is a flowchart of the generative deduction based on causal decoupling in this invention.

[0016] Figure 6 This is a flowchart of the cross-modal topology consistency verification process of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] like Figures 1-6 As shown in this embodiment, a detailed description of the data processing flow of a rail transit defect detection method based on multi-source sensing and causal decoupling is provided, specifically including: S1: Acquire multi-source synchronous sensing data in the track inspection scenario, including RGB images and synchronously acquired structural micro-vibration signals.

[0019] This step is the front-end perception foundation of the entire defect detection process. It is used to acquire multi-dimensional raw information containing the appearance texture, three-dimensional shape and dynamic mechanical properties of track components at one time, providing a complete data source for the subsequent construction of a four-dimensional data field.

[0020] A rail transit defect detection system based on multi-source sensing and causal decoupling includes: A mobile data acquisition platform, configured to simultaneously acquire multi-source sensing data while moving along a track; the mobile data acquisition platform includes: The visual imaging unit is used to acquire RGB images and three-dimensional shape data of the orbital scene; Motion measurement unit: configured to be installed on the mobile platform body, used to sense and output the pose change data and motion state parameters of the mobile platform in space in real time; preferably, a three-axis inertial measurement unit (IMU) is used for platform motion measurement; The dynamic response sensing unit includes at least one sensor that forms a following contact or non-contact coupling with the rail surface, for directly acquiring the micro-vibration response signal of the rail surface under preset or environmental excitation. Optionally, the excitation unit is used to apply controllable micro-vibration excitation to the rail surface.

[0021] When the mobile platform travels along the track at a preset speed, the central control module generates a unified external hardware trigger signal, which is simultaneously connected to the frame synchronization input of the visual imaging unit and the data acquisition gating of the dynamic response sensing unit.

[0022] At the rising edge of the trigger signal, the visual imaging unit performs an exposure and structured light projection to acquire a frame of high-density 3D point cloud data with sub-millimeter precision, as well as a 2D RGB texture image that is strictly registered pixel by pixel. The two constitute a spatiotemporally aligned RGB-D data pair.

[0023] Simultaneously, the sensors in the dynamic response sensing unit perform analog-to-digital conversion on the micro-vibration response signals they currently sense, coupled to the rail surface, and assign them to the depth of the current frame. Figure 1 Output after a specified timestamp. The response signal includes at least the vertical and lateral vibration displacement or acceleration components of the rail surface.

[0024] Preferably, the motion measurement unit uses the same trigger signal as a reference to synchronously record the pose data of the mobile platform in space at the current moment, which is used for subsequent spatial registration and motion compensation of the depth map.

[0025] Preferably, all acquired multi-source data streams (RGB-D data from the visual imaging unit, micro-vibration response signals from the dynamic response sensing unit, and pose data from the motion measurement unit) are encapsulated in real time into data packets with a unified global timestamp and transmitted to the memory buffer of the edge computing unit via a high-speed data bus for queuing and processing, ensuring that each modal data has completed nanosecond-level precise time alignment before entering the subsequent algorithm pipeline.

[0026] S2: Based on multi-source synchronous sensing data, a four-dimensional sensing data field containing the dynamic mechanical response characteristics of components is constructed, and a dynamic mechanical response field characterizing the structural stiffness at each spatial location is generated.

[0027] This step breaks through the limitations of traditional three-dimensional static perception by structurally integrating and frequency-domain transforming continuous multi-frame depth data and micro-vibration signals in four-dimensional spacetime, thereby extracting dynamic mechanical response characteristics that can reflect the differences in stiffness of the internal structure of the component.

[0028] Furthermore, in one embodiment of this application, N frames of depth maps with spatial-timestamp alignment are first continuously retrieved from the memory buffer, where N is a preset frame number constant. The pose trajectory data of the inspection platform, synchronously recorded by the inertial measurement unit, is extracted. Based on this pose data, rigid body space transformation and pixel reprojection operations are performed on each frame of depth map, uniformly mapping the depth values ​​of all frames to a global reference coordinate system with the first frame's camera coordinate system as the origin. In this embodiment, after spatial registration, the depth values ​​at each identical global coordinate position in these N frames of depth maps are extracted and arranged according to the chronological index of the time frames, thereby constructing a three-dimensional data structure, defined as a spatiotemporal depth cube with dimensions H×W×N, where H and W are the spatial height and width of the depth map, respectively. This cube simultaneously encodes the static three-dimensional contour information of the track surface, as well as the implicitly recorded surface micro-deformation fluctuation history in the time dimension. For example, for a fastener that has loosened early on, the surface of the rail above it will produce specific micro-undulations related to its natural frequency under micro-vibration excitation, and this undulation pattern is fully recorded in the time axis signal corresponding to that location.

[0029] In a preferred embodiment, a three-dimensional fast Fourier transform is then performed on the spatiotemporal depth cube to map it from the spatiotemporal domain to the frequency-wavenumber domain, obtaining a three-dimensional spectrum. In this spectrum, the spatial wavenumber axis corresponds to the periodic structure of the physical space, and the time frequency axis corresponds to the vibration response rate at each point. According to one aspect of this application, based on the inherent mechanical properties of the rail and fastener system, a set of learnable frequency domain filter kernels is designed to segment and filter the three-dimensional spectrum into subbands, focusing on extracting specific frequency-wavenumber subbands that are highly correlated with structural stiffness degradation. Energy spectral density calculation and mode distribution analysis are performed on the selected subbands, ultimately generating a two-dimensional image corresponding to the original two-dimensional spatial coordinates, defined as the dynamic mechanical response field. The value of each pixel in this response field characterizes the degree to which the structural response mode at the corresponding spatial location deviates from the normal fastening state under micro-vibration conditions.

[0030] S3: Perform instance segmentation on the static shape components in the RGB image and the four-dimensional perception data field, and construct a topological relationship graph representing the functional coupling relationship between components. The nodes of the topological relationship graph store the multimodal absolute attributes of the components, and the edges store the relative pose and connection tightness relationship attributes between the components.

[0031] This step organizes multimodal sensing information from a pixel-level or point-level disordered state into a highly structured graph data representation, characterizing the multimodal attributes of each physical component in the orbital system, as well as the functional connections between them that have engineering significance.

[0032] Specifically, the static morphology components are first extracted from the spatiotemporal depth cube, i.e., the steady-state depth map obtained by averaging over the time axis and eliminating micro-vibration fluctuations. In a two-stream instance segmentation architecture, the first encoding stream takes the acquired RGB image as input and extracts feature maps such as appearance texture, color, and edges; the second encoding stream takes the steady-state depth map as input and extracts 3D geometric shape and spatial position feature maps. In this embodiment, a cross-modal cross-attention module is embedded in the intermediate layer of the network, enabling each pixel query in the RGB feature map to adaptively weight and aggregate geometric information from the corresponding spatial neighborhood in the depth feature map. At the same time, the depth feature map can also query and fuse the texture information of RGB to generate a tightly coupled fused feature map. The output head of the fused feature map predicts the component semantic category and its instance center offset of each pixel in parallel. After clustering post-processing algorithm, physical components such as "rail", "fastener", "elastic bar", "sleeper", and "ballast" are independently instantiated, and the RGB sub-image, 3D point cloud block and dynamic mechanical response field sub-region corresponding to each component instance are extracted.

[0033] Furthermore, a topological graph of the track scene is constructed based on instantiated components. Each node in the graph corresponds one-to-one with a component instance. The edges are not based on simple spatial Euclidean nearest neighbor edges, but rather follow a hierarchical engineering logic: forcibly, for each node identified as a "fastener" or "elastic bar," an edge is established connecting it to its spatially associated "rail" node and "sleeper" node. In an optional implementation, for each topological edge, the relative pose vector and connection tightness scalar between the two associated nodes are calculated. The relative pose vector describes the translation and rotation relationship between the local coordinate systems of the two components, while the connection tightness scalar is obtained by calculating the average spacing and normal consistency of the adjacent surface point clouds. These original relationship features are mapped by a dedicated edge feature encoder to generate a normalized relationship attribute embedding vector for that edge, completing the graph construction.

[0034] S4: Based on the topological relationship graph, perform dual anomaly reasoning at the node and edge levels, and combine generative inference based on causal decoupling with cross-modal topological consistency verification to generate the final comprehensive defect judgment result.

[0035] This step is the core of the entire defect detection process. On top of the constructed structured graph, multiple complementary reasoning and verification mechanisms are executed in parallel to achieve a leap in defect identification capabilities from "passive matching" to "active deduction".

[0036] According to one aspect of this application, dual anomaly inference is accomplished by two parallel graph neural network branches. The first branch is a node anomaly inferencer, with a graph attention network at its core. Its input is the absolute attribute embedding vector of each node after multimodal fusion, which integrates three types of information: appearance texture, static geometry, and dynamic mechanical response. Through inter-layer attention message passing, each node integrates its own attributes and those of its functional neighbors to generate a context-aware node anomaly score, used to identify defects such as rail cracks and surface spalling, which are mainly due to node attribute anomalies. The second branch is an edge anomaly inferencer, which processes the relationship attribute embedding vectors of all functional edges of the same type as a statistical population, learning and fitting a probabilistic model of their normal relative pose distribution. In this embodiment, the probabilistic model is exemplarily a Gaussian mixture model. During inference, the Mahalanobis distance of the relative pose vector of each edge to be tested relative to the normal distribution is calculated. Functional relationships that deviate significantly from the normal distribution are judged as edge anomalies, such as loose fasteners or installation pose offsets. This edge-level anomaly inference enables the system to detect hidden defects where the appearance texture is completely normal, but the functional relationships are disordered.

[0037] In a preferred embodiment, to further enhance adaptability to scenarios with small or even zero samples, a generative inference mechanism based on causal decoupling is introduced. Multi-source data from each node instance is fed into a causally-inspired variational autoencoder, forcibly decoupling three independent latent factor spaces: a defect content semantic space, a defect geometric morphology space, and a contextual style space. When it is necessary to discriminate novel combined defects with only a very small number of labeled samples, content, geometric, and style factors are sampled from the posterior factor distributions of relevant basic defect categories, and a fictitious combined defect feature sample is actively generated by the decoder. The ternary features of the actual query sample are compared with the fictitious sample using high-level semantic similarity to determine whether it belongs to the unseen combined defect category.

[0038] Furthermore, to fundamentally suppress high false alarms caused by visual artifacts on the track surface such as oil stains, rust, and periodic fish-scale patterns that do not alter the structural topology, a cross-modal topology consistency check is introduced into the process. Feature maps are extracted from the intermediate layers of the RGB encoding branch and the geometric encoding branch, respectively, and the first and second derivatives are calculated to obtain a binary topology feature map. Its response is only sensitive to real structural edges, ridges, and fractures. By forcibly maximizing the response consistency of real defect regions across cross-modal topology feature maps while minimizing the response consistency of false interference regions, the model is trained to physically distinguish between topology breaks and metric perturbations. Finally, the node anomaly score, edge anomaly score, causal inference confidence score, and local consistency score calculated based on topology consistency are weighted and fused. When the comprehensive score exceeds a preset global judgment threshold, the final defect alarm signal and information on defect type, location, and impact range are output.

[0039] According to another aspect of this application, a four-dimensional sensing data field containing the dynamic mechanical response characteristics of components is constructed, and a dynamic mechanical response field characterizing the structural stiffness at each spatial location is generated, including: Spatial registration is performed on the depth maps of N consecutive frames, and the depth values ​​of each frame are uniformly registered to the global reference coordinate system and stacked along the time axis to construct a spatiotemporal depth cube with dimensions H×W×N; where H and W are the spatial height and width of the depth map, and N is the number of frames.

[0040] This step is used to eliminate the inter-frame spatial offset caused by the movement of the inspection platform, and to organize the multi-frame depth data in the time series in a unified global coordinate system to form a three-dimensional data structure that simultaneously contains spatial morphology and temporal evolution information.

[0041] Specifically, in one embodiment of this application, firstly, N frames of time-aligned depth maps are sequentially read from the memory buffer of the high-speed data bus or edge computing unit, in chronological order of timestamps, where N is a preset frame number constant based on the inspection speed and defect detection sensitivity requirements. Each frame of the depth map stores the distance values ​​of each pixel within the field of view relative to the camera's optical center at that moment. Simultaneously, the six-degree-of-freedom pose data of the inspection platform, precisely corresponding to the timestamp of each frame of the depth map, is extracted from the data stream of the inertial measurement unit (IMU), including three-dimensional spatial translation and three-dimensional rotation Euler angles or rotation matrices. In this embodiment, the camera coordinate system of the first frame in the consecutive N frames is selected as the global reference coordinate system. For the subsequent k-th frame of the depth map, a rigid body transformation matrix from the k-th frame camera coordinate system to the first frame camera coordinate system is constructed using its corresponding pose data. For each valid pixel in the depth map of the k-th frame, backproject it from the image coordinates to the 3D point in the camera coordinate system of the k-th frame according to the camera intrinsic parameters, then multiply it by the rigid body transformation matrix to map it to the global reference coordinate system, and finally reproject it onto the virtual global image plane according to the camera intrinsic parameters to complete the spatial registration.

[0042] Furthermore, after all N frames of depth maps have completed the aforementioned spatial registration and reprojection operations, for each valid coordinate position (x, y) on the global reference plane, N depth values ​​are extracted sequentially along the time frame index k direction, forming a one-dimensional time series signal of length N. These time series signals for all spatial positions are rearranged according to their spatial coordinates (x, y), ultimately constructing a three-dimensional matrix structure, defined as a spatiotemporal depth cube, with a spatial height of H, a spatial width of W, and a temporal depth of N. For example, when N is 128 and the depth map resolution is 640×480, the cube's scale is 640×480×128. Each horizontal slice of this cube (with a fixed k value) corresponds to one frame of registered global depth map, while each vertical time series (with fixed x, y values) records the sub-millimeter depth value fluctuation history of that spatial point within N consecutive frames.

[0043] The spatiotemporal depth cube is transformed into the frequency-wavenumber domain using a three-dimensional fast Fourier transform to obtain a three-dimensional spectrum.

[0044] This step is used to perform a frequency domain transformation on the constructed spatiotemporal three-dimensional data structure, converting it from an intuitive spatiotemporal representation into a frequency domain representation that can reveal the dynamic response modes and spatial periodic structure of the components.

[0045] In a preferred embodiment, a complete three-dimensional discrete fast Fourier transform is performed on a spatiotemporal depth cube of size H×W×N. This transform is performed simultaneously in three dimensions: along the spatial x-axis (corresponding to spatial wavenumber u), along the spatial y-axis (corresponding to spatial wavenumber v), and along the time axis (corresponding to time frequency ω). The output after the transform is a three-dimensional complex matrix with the same size as its input, defined as the three-dimensional spectrum F(u, v, ω). In this spectrum, each element contains an amplitude component and a phase component. According to one aspect of this application, the low-frequency region (low u, low v, low ω) in the spectrum mainly corresponds to large-scale trends such as the gentle undulations and slow settlement of the track as a whole; the mid-frequency region corresponds to structural features with specific spatial periods, such as rail corrugation wear and sleeper spacing; and the high-frequency region corresponds to detailed information such as micro-cracks and surface roughness. The energy peak appearing at a specific spatial position (u, v) along the ω-axis in the spectrum directly reflects the vibration response intensity of the corresponding spatial structure at that frequency.

[0046] Furthermore, the three-dimensional spectrum is not used entirely in subsequent processing. Based on prior knowledge of the inherent mechanical properties of the rail and fastener system—such as the inherent bending frequency range of the rail under specific constraints and the typical vibration modes of the fastener spring clip under normal tightening force—spatiotemporal frequency subbands highly correlated with the structural health status can be selected from the three-dimensional spectrum for analysis. For example, in a fastener experiencing early micro-loosening, the contact stiffness between the spring clip and the rail base has degraded. Under micro-vibration excitation, the two will produce nonlinear rubbing motion. This rubbing is manifested in the three-dimensional spectrum as a family of high-order harmonics along the ω-axis at specific spatial wavenumber positions and a broadband noise floor bulge. By pattern recognition of these spectral features, non-contact sensing of dynamic anomalies such as loosening can be achieved.

[0047] Subband segmentation and energy spectral density analysis are performed on the three-dimensional spectrum to extract the response modes within a specific frequency-wavenumber range caused by structural micro-vibration, and a two-dimensional dynamic mechanical response field corresponding to the spatial coordinates is generated.

[0048] This step involves selecting spectral components directly related to the structural health status from the generated original three-dimensional spectrum based on physical priors, and mapping them back to the spatial domain to generate a two-dimensional feature map that can intuitively reflect the dynamic response characteristics of the structure at each location.

[0049] In this embodiment, one or more sets of three-dimensional frequency domain filter kernels are first designed based on a pre-established prior knowledge base of the mechanical properties of track components. This knowledge base stores the statistical range of frequency-wavenumber response characteristics of components such as rails, fasteners, and spring clips under normal and abnormal conditions. Each filter kernel defines a specific region in the uv-ω space, for example, an ellipsoidal subband corresponding to the P2 resonant frequency of the rail, or a specific high-frequency broadband region corresponding to fastener loosening. In an optional implementation, these filter kernels are not fixed but are implemented as learnable parametric tensors, capable of automatically fine-tuning their passband range and shape based on the statistical distribution of actual defect data during end-to-end network training, thereby adapting to differences in structural response under different lines and operating conditions.

[0050] Furthermore, each set of frequency domain filter kernels is sequentially multiplied element-wise with the three-dimensional spectrum F(u, v, ω) to extract the spectral components within that sub-band. For the filtered sub-band spectrum, its energy spectral density is calculated along the ω-axis, i.e., by integrating or weighted summing the squares of the amplitudes of each frequency component. Specifically, for a spatial location (x, y), its final energy response value is determined by the total energy contributed by all frequency components within the filter sub-band at that location. The calculated energy values ​​are rearranged according to their corresponding spatial coordinates (x, y) to generate a two-dimensional grayscale or pseudo-color image, defined as the dynamic mechanical response field M. dyn (x, y). For example, in this response field, normally tightened fastener areas exhibit a uniform low-energy background, while loose fastener areas, due to significant anomalous energy concentration within specific high-frequency subbands, appear as distinct bright areas. Preferably, during the generation of the dynamic mechanical response field, sub-millimeter deformation fluctuation information of the same order of magnitude as structural micro-vibrations is retained to ensure sensitivity to early, subtle loosening.

[0051] According to another aspect of this application, a topological relationship diagram characterizing the functional coupling relationship between components is constructed, including: Extract the static topographic components of the four-dimensional sensing data field, i.e., the mean map of the spacetime depth cube along the time axis.

[0052] This step decouples and extracts static components from the four-dimensional sensing data field, which only characterize the stable three-dimensional shape of the track components, providing high-quality geometric input that is not affected by micro-vibrations for subsequent instance segmentation.

[0053] Specifically, in one embodiment of this application, data is obtained from a constructed spatiotemporal depth cube. This cube has dimensions H×W×N, where the first two dimensions are spatial dimensions and the third dimension is a temporal frame dimension. For each spatial coordinate position (x, y) in the cube, a one-dimensional signal vector of length N is arranged along its corresponding time axis. This vector records the depth value change history of that spatial point over N consecutive frames. In this embodiment, an arithmetic mean operation is performed on this one-dimensional signal vector along the time axis, i.e., the arithmetic mean of these N depth values ​​is calculated. This mean effectively smooths out instantaneous depth value fluctuations caused by random factors such as micro-vibration noise and sensor thermal noise, while completely preserving the true steady-state morphological depth information of the track at that point. After performing the above time axis averaging operation on all spatial coordinate positions (x, y), a two-dimensional matrix of size H×W is obtained, defined as a static morphological component, or what may be called a mean depth map.

[0054] According to one aspect of this application, the static topography component is used as input to the geometric coding branch in the subsequent instance segmentation process. Compared to directly using the original depth map of a single frame, this mean map has significant advantages: First, depth holes or flying spots that may exist in the single-frame depth map due to instantaneous occlusion of structured light or sudden changes in reflectivity are effectively filled or suppressed after averaging across multiple frames, improving the integrity of the point cloud data; Second, sub-millimeter-level elastic deformation of components caused by micro-vibrations of the train passing by or the environment is smoothly filtered out, allowing the segmentation network to focus on the stable geometric contours of the components and avoiding the impact of edge blurring caused by vibration on the accuracy of instance segmentation. For example, for a section of rail with periodic fish-scale patterns on its surface, the depth values ​​of the grooves in its single-frame depth map fluctuate between frames due to the influence of illumination angle and micro-vibrations, while the mean map can output a high-quality geometric reference map with clear texture boundaries and stable depth values, laying a solid data foundation for downstream segmentation and recognition tasks.

[0055] The RGB image is input into the visual feature extraction network to obtain the appearance feature map, and the mean image is input into the point cloud feature extraction network to obtain the geometric feature map.

[0056] This step is the front-end feature extraction stage of the dual-stream instance segmentation architecture. Its purpose is to independently extract discriminative high-level semantic features from the appearance modality and the geometric modality, respectively, so as to provide complementary original feature expressions for subsequent cross-modal fusion.

[0057] In a preferred embodiment, the first branch is a visual feature extraction network, whose input is a two-dimensional RGB texture image. The main architecture of the network exemplarily employs a convolutional neural network pre-trained on a large-scale dataset such as ImageNet, or a visual Transformer architecture with stronger global context modeling capabilities. Through layer-by-layer convolution or self-attention operations, it progressively extracts multi-scale appearance feature maps, ranging from low-level local texture features such as edges and corners, to mid-level regional features such as surface damage morphology and material differences, and finally to high-level global features such as component category semantics. Specifically, the feature maps output by the network can be designed to have a spatial resolution of 1 / 8 or 1 / 16 of the input image, but expanded to 256 or 512 dimensions in the channel dimension, so that each feature map pixel vector forms a compact embedding encoding of the appearance attributes within its receptive field.

[0058] Furthermore, the second branch is a point cloud feature extraction network, whose input is the generated static shape component, i.e., the mean depth map. First, based on the camera intrinsics, each effective pixel in the mean depth map is back-projected from the image coordinates to 3D space, generating a dense 3D point cloud. Subsequently, this point cloud is fed into a feature extraction network specifically designed for unordered point sets, such as PointNet++ or its improved version. Through farthest point sampling and multi-scale grouping, the relative geometric relationships of points in the local neighborhood are calculated, and then aggregated layer by layer through a multilayer perceptron with shared weights, finally outputting a fixed-dimensional local geometric embedding vector for each point. In this embodiment, in order to maintain spatial alignment with the visual feature map, the point cloud features are reprojected back onto the 2D image plane to form a geometric feature map with the same spatial resolution as the visual feature map. The geometric feature map encodes 3D structural information such as the curvature, normal vector direction, and elevation changes of the local surface where each pixel is located. It is not sensitive to changes in appearance texture, but has a natural sensitivity to structural anomalies such as geometric deformation, depressions, and displacements, thus complementing the appearance feature map of the first branch.

[0059] By employing a cross-modal attention mechanism, pixel queries in the appearance feature map and the geometric feature map adaptively aggregate each other's modal features to generate a fused feature map.

[0060] This step is the core fusion link of the entire dual-stream instance segmentation architecture. Its purpose is not simply to concatenate the two output feature maps channel by channel or add them element by element, but to achieve pixel-level, content-adaptive information exchange and selective aggregation through an attention mechanism, thereby generating a tightly coupled representation that can fully complement the advantages of their respective modalities.

[0061] According to one aspect of this application, a bidirectional, cross-modal cross-attention module is designed. The two inputs of this module receive appearance feature maps F respectively. rgb With geometric feature map Fgeo Both have the same spatial resolution. In the first attention direction, the appearance feature map F... rgb Each pixel position in the data is used as a query vector, with the geometric feature map F as the reference vector. geo The feature vectors of all pixels on the same horizontal and vertical line as the current pixel are used as the key and value vectors, respectively. A set of spatially adaptive attention weights is obtained by calculating the dot product similarity between the query vector and each key vector and then normalizing it using softmax. These weights are used to perform a weighted summation of the value vectors at the corresponding positions, generating the geometrically enhanced appearance features for that pixel position. In this embodiment, this cross-shaped attention window design achieves a good trade-off between computational efficiency and the global receptive field, avoiding the quadratic computational complexity of full-image self-attention.

[0062] Furthermore, in the second attention direction, using the geometric feature map F geo Using pixels in the image as queries, the reverse process is performed from the appearance feature map F. rgb The corresponding cross-shaped window aggregates appearance and texture information to generate geometric features that enhance appearance. The enhanced feature maps output from the two directions are then added element-wise or concatenated along the channel dimension with the original input feature map, and the information is integrated through a lightweight fusion convolutional layer to output the final cross-modal fusion feature map. For example, in a scenario where there is oil stains on the surface of a rail, appearance features alone can easily misjudge the dark areas of the oil stains as potential defects; while geometric features can accurately reflect that the area has no depth anomalies, they lack sufficient texture context. Through cross-modal attention, the high-response query vector of the oil stain area in the appearance branch will actively draw information from the low-response area (no depth anomaly) in the geometric branch, thereby effectively suppressing the interference of oil stains in the fusion feature map, while retaining the common high response of the surrounding real crack area in both modalities.

[0063] Pixel-level component category prediction and instance center offset prediction are performed on the fused feature map. After clustering, the mask and point cloud blocks of each component instance are obtained, which serve as nodes in the topology graph.

[0064] This step is responsible for transforming the generated information-rich fusion feature map into discrete component individual instantiation results, completing the structured analysis from continuous feature space to discrete physical component objects, and laying the node foundation for the construction of the topological relationship graph.

[0065] Specifically, in one embodiment of this application, two prediction head networks are set in parallel on the cross-modal fusion feature map. The first prediction head is a semantic category prediction head, which consists of several convolutional layers and pixel-wise softmax activation functions, outputting a multi-channel semantic probability map with the same spatial resolution as the input feature map. Each channel corresponds to a predefined track component category, such as rail top surface, fastener, elastic clip, sleeper, ballast, etc. The second prediction head is an instance center offset prediction head, which outputs a two-channel offset vector map. Each pixel position outputs a two-dimensional vector pointing to the geometric center position of the component instance to which the pixel belongs. In this embodiment, the definition of the instance geometric center can be flexibly selected according to the component shape. For example, for a long and narrow rail, its center can be defined as its cross-sectional central axis; for a compact fastener, its center can be defined as the centroid of its point cloud block.

[0066] Furthermore, after obtaining the two prediction outputs, a density-based clustering post-processing procedure is executed. First, based on the pixel-level category determination results of the semantic probability map, all pixels belonging to the same foreground component category are initially divided into their respective coarse mask regions. Subsequently, for each coarse mask region, the instance center offsets predicted by each pixel within it are aggregated, and using mean shift or density-based spatial clustering algorithms, pixels with offsets pointing to similar center positions are grouped into the same instance, thereby achieving the separation of different individual components within the same category. For example, in a scene where adjacent fasteners are closely arranged, this offset clustering mechanism can effectively distinguish two fastener individuals with similar appearances and spatial proximity, assigning them different instance identifiers. After clustering is completed, the pixel mask of each instance, the corresponding two-dimensional RGB sub-image region, and the independent three-dimensional point cloud block obtained through back projection are uniquely identified and packaged for output, and formally registered as nodes in the scene topology graph to be constructed. According to one aspect of this application, the corresponding sub-regions of each node in the dynamic mechanical response field are extracted during this process, providing a data source for the dynamic mechanical channel in the subsequent embedding of node absolute attributes.

[0067] Based on the category of component instances, topological edges are forcibly established between nodes with functional coupling relationships; and the relative pose vector and connection tightness scalar between the two ends of each edge are calculated as the original features of the edge. After being mapped by the edge feature encoder, a topological relationship graph is generated.

[0068] This step organizes the instantiated isolated component nodes into a topological graph structure with clear physical meaning according to the engineering functional logic of the rail transit facility, and quantitatively describes the spatial assembly relationship and fitting state between components.

[0069] In one alternative implementation, the construction rules for topological edges are not based on simple Euclidean nearest neighbor distances in the point cloud, but are deeply integrated with the engineering semantic constraints of rail transit lines. The system maintains a component function association rule table, defining which categories of components have mandatory functional coupling relationships. For example, the rules explicitly require that each instantiated node classified as either a "fastener" or a "elastic strip" must establish a topological edge with its spatially associated "rail" node and "sleeper" node, respectively. The determination of the affiliation relationship can be comprehensively determined by calculating the minimum spatial distance between component masks, the relative centroid positions of point cloud blocks, and ray intersection tests. In this embodiment, if the centroid of a fastener node's point cloud block falls within the bounding box of a sleeper node in the vertical projection direction, and the distance between its lateral boundary and the edge of a rail node is less than a preset threshold, then the fastener node establishes a functional edge with these two nodes.

[0070] Furthermore, for each established topological edge, two types of relational attributes are quantitatively calculated. The first type of attribute is the relative pose vector. Specifically, principal component analysis is performed on the 3D point cloud blocks of each of the two end nodes of the edge to extract the local reference coordinate system of each component, including its normal vector direction, tangent plane direction, and geometric center position. The relative pose vector encodes the relative translation (three components) and relative rotation (three Euler angles or rotation quaternion components) between these two local coordinate systems, collectively forming a high-dimensional vector. For example, for the fastener-rail edge, this vector accurately describes the fastener's installation height, horizontal position, and attitude angle relative to the rail top surface. The second type of attribute is the connection tightness scalar, calculated by weighted fusion of three indicators: the average nearest point distance between the two node point cloud blocks in the spatially adjacent region, the percentage of overlapping point cloud volume, and the consistency of the normal vector direction. This numerically quantifies the tightness of the fit between the two components. According to one aspect of this application, the calculated original relative pose vector is concatenated with the connection tightness scalar to form a fixed-length original edge feature vector, which is then input into an edge feature encoder composed of shallow multilayer perceptrons. After nonlinear mapping, a normalized edge relation attribute embedding vector is output and stored in the edge data structure of the topological relation graph, thus completing the complete construction of the graph.

[0071] According to another aspect of this application, performing dual anomaly reasoning at the node and edge levels includes: The multimodal absolute attributes of each node are input into the node anomaly inferencer. Through the graph attention network, attention weights are dynamically calculated based on the absolute attributes of the node itself and its neighboring nodes, and a context-aware node anomaly score is output for each node.

[0072] This step utilizes the message passing mechanism of graph neural networks to enable the anomaly judgment of each component to integrate the state information of other components in its functional neighborhood, avoiding misjudgment or omission caused by isolated judgment.

[0073] Specifically, in one embodiment of this application, the core of the node anomaly inferencer is a multi-layer graph attention network. Its input is the fused absolute attribute embedding vector A of each node in the topological graph. i This system integrates three types of heterogeneous information: appearance texture, static geometry, and dynamic mechanical response. In each layer of the graph attention network, an asymmetric attention coefficient is calculated for each pair of nodes connected by a topological edge in the graph. In this embodiment, the attention coefficient is calculated by first multiplying the current layer feature vectors of the source node and the target node by their respective linear transformation matrices, concatenating them, and then mapping them to a scalar value through a single-layer feedforward network and the LeakyReLU activation function. Finally, softmax normalization is performed on the target node's set of all neighboring nodes. This attention coefficient quantitatively characterizes the informational importance of the source node's features for determining the current state of the target node.

[0074] Furthermore, the updated features of the target node in the next layer are the linearly transformed feature vectors of all its neighboring nodes (including itself), weighted by the corresponding attention coefficients. Through multiple iterations, the feature vector of each node gradually incorporates the contextual information of its multi-hop functional neighbors. For example, a fastener node located near a rail joint will naturally have its attention coefficients assigned higher weights to adjacent rail nodes and fastener nodes on the same side, because the state of these components is closely related to the structural integrity of the joint area. Finally, a classification prediction head is connected above the last layer, and a sigmoid activation function is added to a nonlinear mapping layer to output a scalar between 0 and 1 for each node in the graph, namely the node anomaly score s. node (v i The node anomaly score comprehensively considers the node's own multimodal attributes as well as the overall state of its functional subgroup, exhibiting global context awareness. For example, when a fastener node has only a slight anomaly in its absolute attributes, but multiple of its neighboring fastener nodes also show similar attribute shifts, the fastener's node anomaly score will be significantly amplified by the message passing mechanism of the graph network, thereby achieving early group perception of systemic degradation trends.

[0075] The relative pose vectors of all functional edges of the same type are grouped into a single distribution. Through the edge anomaly inferencer, the statistical distribution model of the relative pose of normal edges is learned. The Mahalanobis distance of the relative pose vector of each edge to be tested relative to the statistical distribution model is calculated as the edge anomaly score.

[0076] This step uses statistical learning to accurately identify hidden defects that appear normal in texture and geometry but have functional imbalances in the assembly relationships between components.

[0077] In a preferred embodiment, the operation of the edge anomaly inferencer is based on a core premise: on a normally operating rail transit line, the relative pose vectors between the two endpoints of the vast majority of similar functional edges (e.g., all "fastener-rail" functional edges) should conform to a centralized and stable statistical distribution, which characterizes the normal installation state determined by the line's design, construction, and maintenance standards. The primary task of the edge anomaly inferencer is to learn a probabilistic model of this distribution from a large number of normal samples during the training phase or the initial normal phase of online operation. In this embodiment, the probabilistic model is exemplarily selected as a Gaussian mixture model, which can flexibly fit possible multimodal distributions, such as different fastener models or different line sections potentially corresponding to different pose cluster centers. The parameters of the Gaussian mixture model, including the mean vector, covariance matrix, and mixture weights of each Gaussian component, are obtained by training from the generated set of normal edge relationship attribute embedding vectors using maximum likelihood estimation or expectation-maximization algorithms.

[0078] Furthermore, after learning the statistical model of the normal edge distribution, for any input topological edge to be tested, its functional edge type is first determined based on the component categories of its two endpoints, and the corresponding trained Gaussian mixture model is retrieved. Then, the relational attribute embedding vector of the edge to be tested, mapped by the encoder, is extracted, and the Mahalanobis distance of this vector relative to the Gaussian mixture model is calculated. The advantage of Mahalanobis distance in this scenario is that by standardizing and decorrelating each dimension through the inverse of the covariance matrix, it effectively eliminates the influence of scale differences and correlations between different components of the relative pose vector, allowing the distance metric to accurately reflect the degree of deviation of the pose of the edge to be tested from the normal distribution manifold. For example, if in a fastener-rail edge model, the normal variance of the installation height is small while the tolerance range of the horizontal position is large, the Mahalanobis distance will automatically assign higher anomaly weights to small deviations in the height direction. The calculated Mahalanobis distance, after a preset scaling and nonlinear mapping, is normalized to an edge anomaly score s between 0 and 1. edge (e ij The higher the anomaly score, the more serious the deviation of the actual assembly relationship between the components from the normal engineering standards, and the greater the possibility of functional abnormalities such as loosening, displacement, and installation errors.

[0079] According to another aspect of this application, generative deduction based on causal decoupling includes: The multi-source data of each node instance is input into the causal decoupling network and mapped to three independent latent spaces: defect content semantic space, defect geometric morphology space, and context style space.

[0080] This step extracts independent latent factors with physical causal meaning from the original high-dimensional perceptual data, providing a structured latent space representation basis for subsequent combinatorial generative deduction.

[0081] According to one aspect of this application, the architecture of the causal decoupling network is based on an improvement of the variational autoencoder paradigm, but the core design principle lies in introducing a causal structure prior to guide the decoupling of the latent space. The encoder part of the network receives multi-source data from a node instance, including RGB sub-image patches, 3D point cloud patches, and dynamic mechanical response field sub-regions. Instead of simply compressing this data into a single latent vector, the encoder outputs three independent posterior distribution parameters through three structurally identical but parameter-independent branch encoding networks. Each set contains a mean vector and a variance vector, collectively defining a Gaussian distribution in the three factor spaces. In this embodiment, the first factor is defined as the defective content semantic factor Z. content The first factor is responsible for encoding the semantic connotation of "what type of defect it is," such as abstract category-level concepts like "crack," "dent," and "peeling," which are unrelated to specific visual representations. The second factor is defined as the defect geometry factor Z. geometry This is responsible for encoding the specific three-dimensional shape features of defects, such as the length and direction of cracks, and the depth and contour of pits, as well as other spatial details. The third factor is defined as the context style factor Z. style It is responsible for encoding attributes that are unrelated to the category and form of the defect itself, but are related to the local environment and style of the defect, such as whether its surface is covered with a dry oxide layer, wet oil stains, or weeds.

[0082] Furthermore, to ensure true statistical independence among the three latent spaces, i.e., to satisfy the requirement of causal decoupling, a mutual information minimization regularization term is introduced into the network's training loss function. Specifically, the regularization term estimates the lower bound of the mutual information between any two factor posterior distributions through a discriminator network, and is added as a penalty term to the total loss. Minimizing this penalty term forces Z... content Z geometry With Z style They are mutually independent in a probabilistic sense. For example, after sufficient training, in Z... content Changing only the position of the sampling point in space will cause the generated decoding results to switch between different defect semantics such as cracks and dents, but will maintain their three-dimensional shape and surface style; in Z... geometry Moving through space, the semantic category of the defect remains unchanged, but its length and depth change continuously; while in Z... style Sampling in space showed that the defect fingerprint remained unchanged, but its surface exhibited a continuous transition from dry to oily and wet.

[0083] For novel defects with only a few supporting samples, the defect region of the supporting samples is decomposed into independent Gaussian posterior distributions in three latent spaces to achieve factorized memory. When comparing query samples, content factors, geometric factors, and style factors are actively sampled from the factor distributions of relevant defect categories, and the decoder generates ternary feature data of the fictional combined defect. The similarity between the real ternary feature data of the query sample and the ternary feature data of the fictional combined defect is calculated. If the similarity exceeds a threshold, the query sample is determined to belong to the combined defect.

[0084] This step applies causal decoupling capabilities to small-sample and zero-sample defect identification scenarios, and fully describes the entire generative defect cognition process from factorized memory storage to active inference generation and then to high-level semantic comparison.

[0085] In a preferred embodiment, when faced with a novel defect category with only 3 to 5 labeled samples during the operational phase, the factorization memory storage process is initiated first. For each of these few support samples, the multi-source data (RGB sub-image, point cloud patch, dynamic mechanical response field sub-region) corresponding to its defect region is extracted and mapped to posterior distribution parameters in three independent latent spaces using a trained and frozen causal decoupling encoder. Specifically, for all support samples, their Z-values ​​are calculated separately. content Z geometry and Z style The mean and variance in each dimension are fitted to three independent multidimensional Gaussian distributions, denoted as N(μ). c , Σ c ), N(μ) g , Σ g ) and N(μ s , Σ s These three distributions are stored in memory as factorized prototypes of the new defect category, requiring minimal storage space, with only a small number of distribution parameters to be stored.

[0086] Furthermore, when a defect determination is needed for a query sample, a mental deduction-based active generation operation is performed. Suppose we need to determine whether the query sample belongs to the combined defect "wrinkles covered in oil stains," which has never appeared in the training set. First, the geometric morphology factor distribution N(μ) of the "wrinkles" defect category is retrieved from the memory. g Waves, Σ g (Wavy), style factor distribution N(μ) from the context of "oil stain" style. s Oil stains, Σ s Random samples are taken from the oil stains to obtain a specific geometric factor vector Z. geometry and a style factor vector z style Simultaneously, a z-sample is taken from a general defect content distribution.content Vectors. In this embodiment, the three sampled latent factor vectors are fed together into the trained decoder network. The decoder generates a fictional multi-source data representation (generated ternary feature) of an "oil stain ripple" defect that the system has never directly observed in the training data. For example, the generated result presents the feature of "ripple" in terms of content semantics, has the wavelength and depth parameters of real ripple in terms of geometric morphology, but presents the dark and wet texture of oil stains in terms of appearance style.

[0087] Finally, the real ternary features obtained from the real multi-source data of the query sample through the same encoder are compared with the fictionally generated ternary features. The comparison is not performed in the original pixel or point cloud space, but rather in a high-level semantic space, where cosine similarity or Euclidean distance is calculated. If the similarity exceeds a preset threshold, it indicates that the feature combination pattern of the query sample highly matches the fictional combination defect expected through causal inference. Based on this, it is determined to be the combination type defect, and the specific content and style description of the defect are given in the output. This mechanism fundamentally breaks the dependence on a large number of labeled samples, endowing the system with incremental cognitive capabilities in an open world.

[0088] According to another aspect of this application, cross-modal topology consistency verification includes: Feature maps are extracted from the intermediate layers of the visual encoder and the geometric encoder respectively, and their first and second derivatives are calculated to generate binarized topological feature maps.

[0089] This step extracts an abstract topological skeleton diagram from the intermediate representations of deep networks that is sensitive only to structural topological changes (such as breaks and holes) and robust to metric perturbations (such as lighting, color, and texture).

[0090] Specifically, in one embodiment of this application, during the forward inference process of the visual encoder and the geometric encoder, instead of using only their final output layer features, an additional set of feature maps is extracted from specific intermediate layers of each of the two encoders. These intermediate layers are selected based on the fact that their receptive field size is sufficient to cover the typical scale of track defects while preserving sufficient spatial detail. In this embodiment, the output feature maps of the third downsampling stage of the visual encoder and the geometric encoder are exemplary selected, and after channel alignment and spatial resolution matching, they are denoted as Fi and Fj, respectively. rgb mid With F geo mid .

[0091] Furthermore, explicit computation of topological feature response maps is performed on these two intermediate feature maps. The computation process simulates a deep learning version of edge detection and corner detection operators in image processing. First, the first-order partial derivatives of the feature maps along the x and y directions in space are calculated, forming the spatial gradient map. The local maxima of the response correspond to edge positions in the feature space, i.e., the boundaries between different semantic or geometric regions. Second, based on the spatial gradient map, the second-order partial derivatives are further calculated to construct a local Hessian matrix. By calculating the combined response of its determinant and trace, the ridge and corner positions in the feature space are extracted, i.e., the locations where first-order edges turn or terminate. According to one aspect of this application, the first-order and second-order derivative response maps are binarized using a preset threshold to generate the final binarized topological feature map T. rgb With T geo In these binary images, pixels with a value of 1 constitute the topological skeleton of the modal sensing data in the high-dimensional feature space. For example, a real crack on a rail surface appears as a dark, thin line in the RGB image and as a micrometer-sized V-shaped groove in the depth image; both topological skeletons are continuous, high-response curves in the binary topological feature image. Conversely, fish-scale patterns on the surface only appear as dense, parallel, high-response lines in the RGB topological feature image, but exhibit a low response in the geometric topological feature image due to the lack of real depth variation.

[0092] A topology consistency regularization loss is constructed, which maximizes the response consistency of the real defect region between the visual and geometric topology feature maps and minimizes the response consistency of the false interference region. When generating the comprehensive defect judgment result, the node anomaly score, edge anomaly score, causal inference confidence score and the local consistency score calculated based on the topology consistency regularization loss are weighted and fused. When the weighted fused score exceeds the global judgment threshold, the final defect alarm, type and location information are output.

[0093] This step fully describes the entire chain from the training strategy of topological consistency constraints to the final multi-source score fusion and decision output, and is the core guarantee mechanism for this scheme to achieve the extremely low false alarm rate.

[0094] In a preferred embodiment, during the network training phase, a dedicated topology consistency regularization loss term L is constructed. topo And incorporate it as a component of the overall training objective function. topo The computation is divided into two adversarial directions. The first direction targets the real defect region (provided during training by a manually labeled mask), and computes the visual topological feature map T within that region. rgb With geometric topological feature map T geoThe pixel-wise response consistency is typically calculated by summing the element-wise products of the two regions and taking the negative value, thus maximizing the overlap of coverage. The second approach targets explicit spurious interference regions (for example, regions marked with fish scale patterns, oil stain boundaries, or rust edges but without actual structural damage). Pixel-wise response consistency is also calculated, but this term is included in the loss with a positive sign. That is, by minimizing their product, the topological feature maps of the two modalities are forced to be mutually exclusive within this region.

[0095] Furthermore, after training convergence, during the forward propagation of the model in the inference phase, the local topological consistency score is calculated in real time for each test region. In this embodiment, this score is defined as 1 minus L of the current region. topo Local calculated values ​​(after normalization). Therefore, a crack region that is highly consistent on both the visual and geometric topological maps will receive a high consistency score close to 1, while a fish-scale pattern region that responds only on the visual topological map and not on the geometric topological map will receive a low consistency score close to 0.

[0096] Finally, the comprehensive defect assessment and decision-making stage is entered. Independent scoring signals from multiple reasoning paths are summarized: node anomaly score s. node , edge abnormal score s edge The system combines a defect similarity score (i.e., a causal inference confidence score) with a local topological consistency score. These scores are linearly combined using a set of learnable weighted coefficients, which can be obtained on a validation set through grid search or learning. For example, in scenarios with poor lighting conditions, the reliability of appearance features decreases, and the weight of the topological consistency score can be dynamically increased. When the weighted composite score of a candidate defect location exceeds a preset global judgment threshold, a defect alarm record is officially generated. This record contains information such as the specific location of the component, the defect type, the degree of anomaly, and the impact range determined based on topological edge analysis. It is presented to inspection personnel through a human-machine interface or automatically triggers subsequent maintenance scheduling processes. This multi-source information cross-validation fusion decision architecture ensures that each alarm output undergoes multiple physical consistency checks, suppressing false alarms caused by appearance artifacts.

[0097] This invention enhances the defect detection capabilities of rail transit by constructing a three-pronged new technological paradigm: "dynamic mechanical perception, topological relationship reasoning, and causal generation deduction." First, in terms of perception, it breaks through the limitations of traditional methods that rely solely on static textures and geometric shapes. By simultaneously acquiring micro-vibration signals and constructing a spatiotemporal depth cube, it extracts a dynamic mechanical response field reflecting the structural stiffness at each spatial location. This allows for the sensitive detection of hidden functional defects without significant external changes, such as early micro-loosening of fasteners and concealed empty sleepers. Second, in terms of relational cognition, it expresses the track scene as a topological graph composed of functional coupling relationships between components. Relative poses between components replace absolute coordinates as the evaluation criterion, fundamentally solving the root causes of false alarms that have long plagued existing solutions, such as overall track settlement and inspection benchmark drift. This focuses anomaly detection on the coordination of functional relationships between components. Furthermore, at the defect generalization level, a causal decoupling network separates defect factors into three independent latent spaces: content semantics, geometric morphology, and appearance style. This endows the model with the ability to proactively imagine unknown combinations of defects, achieving zero-sample identification of new composite defect types with only a very small number of known factor samples, overcoming the heavy dependence of traditional deep models on large-scale labeled data. Finally, through cross-modal topological consistency verification, starting from the physical differences between structural topological breaks and surface metric perturbations, visual interferences such as oil stains, rust, and fish scale patterns, which have long led to high false alarm rates, are filtered out. The synergistic effect of these innovative mechanisms enables a single model to simultaneously cover two major categories of defects: appearance cracks and three-dimensional loosening. Detection accuracy, robustness, and generalization ability are all significantly improved, providing an open-world defect detection solution for intelligent operation and maintenance of rail transit that is incrementally scalable and can evolve over the long term.

[0098] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A method for detecting defects in rail transit based on multi-source sensing and causal decoupling, characterized in that, include: Acquire multi-source synchronous sensing data in the track inspection scenario, including RGB images and synchronously acquired structural micro-vibration signals; Based on multi-source synchronous sensing data, a four-dimensional sensing data field containing the dynamic mechanical response characteristics of components is constructed, and a dynamic mechanical response field characterizing the structural stiffness at each spatial location is generated. Instance segmentation is performed on the static shape components in RGB images and four-dimensional perception data fields to construct a topological relationship graph representing the functional coupling relationship between components. The nodes of the topological relationship graph store the multimodal absolute attributes of the components, and the edges store the relative pose and connection tightness relationship attributes between the components. Based on the topology graph, dual anomaly reasoning at the node and edge levels is performed, and combined with generative inference based on causal decoupling and cross-modal topology consistency verification, the final comprehensive defect judgment result is generated.

2. The method according to claim 1, characterized in that, A four-dimensional sensing data field containing the dynamic mechanical response characteristics of the components is constructed, and a dynamic mechanical response field characterizing the structural stiffness at each spatial location is generated, including: Spatial registration is performed on the depth maps of N consecutive frames, and the depth values ​​of each frame are uniformly registered to the global reference coordinate system and stacked along the time axis to construct a spatiotemporal depth cube with a size of H×W×N; where H and W are the spatial height and width of the depth map, and N is the number of frames; The spatiotemporal depth cube is transformed to the frequency-wavenumber domain using a three-dimensional fast Fourier transform to obtain a three-dimensional spectrum; Subband segmentation and energy spectral density analysis are performed on the three-dimensional spectrum to extract the response modes within a specific frequency-wavenumber range caused by structural micro-vibration, and a two-dimensional dynamic mechanical response field corresponding to the spatial coordinates is generated. Before generating the dynamic mechanical response field, a learnable frequency domain filter kernel is constructed based on the prior mechanical properties of the rail and fastener system to screen key spatiotemporal frequency subbands associated with structural stiffness anomalies.

3. The method according to claim 2, characterized in that, Spatial registration of depth maps for N consecutive frames includes: Acquire the inspection platform pose data output by the inertial measurement unit, synchronized with the timestamp of each frame of the depth map; Based on the pose data, rigid body transformation and pixel reprojection are performed on the depth map of each frame, and the depth information of all frames is uniformly registered under the global reference coordinate system based on the camera coordinate system of the first frame. The depth values ​​at the same global coordinate position in the registered N-frame depth map are extracted, arranged by time frame index, and a spatiotemporal depth cube is generated.

4. The method according to claim 1, characterized in that, Construct a topological relationship diagram representing the functional coupling relationships between components, including: Extract the static topography components of the four-dimensional sensing data field, i.e., the mean map of the space-time depth cube along the time axis. The RGB image is input into the visual feature extraction network to obtain the appearance feature map, and the mean image is input into the point cloud feature extraction network to obtain the geometric feature map; By using a cross-modal attention mechanism, pixel queries in the appearance feature map and the geometric feature map adaptively aggregate each other's modal features to generate a fused feature map. Pixel-level component category prediction and instance center offset prediction are performed on the fused feature map. After clustering, the mask and point cloud blocks of each component instance are obtained as nodes of the topology graph. Based on the category of component instances, topological edges are forcibly established between nodes with functional coupling relationships; and the relative pose vector and connection tightness scalar between the two ends of each edge are calculated as the original features of the edge. After being mapped by the edge feature encoder, a topological relationship graph is generated.

5. The method according to claim 1, characterized in that, Perform dual anomaly reasoning at both the node and edge levels, including: The multimodal absolute attributes of each node are input into the node anomaly inferencer. Through the graph attention network, attention weights are dynamically calculated based on the absolute attributes of the node itself and its neighboring nodes, and a context-aware node anomaly score is output for each node. The relative pose vectors of all functional edges of the same type are grouped into a whole distribution. Through the edge anomaly inferencer, the statistical distribution model of the relative pose of normal edges is learned. Calculate the Mahalanobis distance of the relative pose vector of each edge to be tested relative to the statistical distribution model, and use it as the edge anomaly score; Among them, the multimodal absolute attribute is a unified vector generated by projecting and fusing the visual texture embedding vector, the static geometric embedding vector, and the dynamic mechanical embedding vector.

6. The method according to claim 1, characterized in that, Generative deduction based on causal decoupling includes: The multi-source data of each node instance is input into the causal decoupling network and mapped to three independent latent spaces: defect content semantic space, defect geometric morphology space and context style space. For novel defects with only a few supporting samples, the defect region of the supporting samples is decomposed into independent Gaussian posterior distributions in three latent spaces to achieve factorized memory. When performing query sample comparison, the content factor, geometric factor and style factor are actively sampled from the factor distribution of relevant defect categories, and the decoder generates ternary feature data of fictional combined defects. Calculate the similarity between the real ternary feature data of the query sample and the ternary feature data of the fictional combined defect. If the similarity exceeds the threshold, the query sample is determined to belong to the combined defect.

7. The method according to claim 6, characterized in that, Perform cross-modal topology consistency verification, including: Feature maps are extracted from the intermediate layers of the visual encoder and the geometric encoder respectively, and their first and second derivatives are calculated to generate binarized topological feature maps. A topology consistency regularization loss is constructed, which maximizes the response consistency of real defect regions between visual and geometric topology feature maps and minimizes the response consistency of false interference regions. When generating the comprehensive defect judgment result, the node anomaly score, edge anomaly score, causal inference confidence score and local consistency score calculated based on topological consistency regularization loss are weighted and fused. When the weighted and fused score exceeds the global judgment threshold, the final defect alarm, type, and location information are output.

8. The method according to claim 1, characterized in that, The method also includes: In the topological graph, for edges judged as abnormal, analyze the absolute and relational attributes of the nodes at both ends to determine whether the abnormality stems from an absolute defect in the component itself or from a functional imbalance in the relative relationship between components. When a functional disorder is determined to be a relative relationship, all functionally coupled nodes connected to the abnormal edge are tracked and output to generate an impact range topology map.

9. The method according to claim 2, characterized in that, The method also includes: Extract features from the three-dimensional spectrum; in response to the detection of high-frequency modes and broadband noise floor that are inconsistent with the fundamental frequency of the rail at a certain spatial location, increase the feature response weight of the node corresponding to that spatial location when generating the dynamic mechanical embedding vector; In generating the two-dimensional dynamic mechanical response field, sub-millimeter deformation fluctuation information of the same order of magnitude as the micro-vibration of the structure is retained.

10. The method according to claim 5, characterized in that, Learn the statistical distribution model of the relative pose of normal edges, including: A Gaussian mixture model was used to fit the relative pose vector distribution of all normal functional edges of the same type; During continuous online inspection, the parameters of the Gaussian mixture model are incrementally updated based on the accumulated normal sample data to adapt to the overall slow changes in the track foundation under long-term operation. In calculating the Mahalanobis distance, the dynamic threshold used is set based on the probability density function of the Gaussian mixture model at a specific confidence level.

Images