Railway anomaly monitoring method and system based on unmanned aerial vehicle inspection

Abstract

The invention provides a railway anomaly monitoring method and system based on unmanned aerial vehicle routing inspection, and relates to the field of data processing, the method comprises the following steps: obtaining railway routing inspection data collected by an unmanned aerial vehicle, extracting a monitoring instruction in the railway routing inspection data, carrying out parameter analysis on the monitoring instruction, and obtaining an anomaly monitoring parameter set corresponding to the monitoring instruction; performing spatio-temporal feature extraction processing on the plurality of spatio-temporal frame sequences, generating a spatial feature vector and a time feature vector of each spatio-temporal frame sequence, screening out at least one abnormal region from the plurality of spatio-temporal frame sequences based on a dynamic matching relationship between the abnormal monitoring parameter set and the spatial feature vector and the time feature vector, and performing abnormal region extraction processing on the abnormal region. And calling an anomaly identification model to perform conjoint analysis on the spatial-temporal characteristics of the abnormal region, generating a railway anomaly monitoring result, and mapping the railway anomaly monitoring result to a physical coordinate system of the railway line to generate an anomaly positioning report. According to the invention, the accuracy and intelligent decision of railway inspection anomaly detection can be improved.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more specifically, to a method and system for monitoring railway anomalies based on unmanned aerial vehicle (UAV) inspection. Background Technology

[0002] In existing railway inspection technologies, traditional methods usually rely on manually preset fixed detection thresholds or single-frame image feature comparison. For example, structural deformation can be identified by segmenting track area images, or abnormal peak values ​​can be determined using time-series vibration data. This method is insufficient for handling the spatiotemporal correlation of multi-source heterogeneous data. Especially in continuous dynamic scenarios, parameter fixation leads to a high false alarm rate.

[0003] Based on this, some improvement schemes attempt to introduce convolutional neural networks to process spatial features, but they still fail to effectively integrate dynamic changes in the time dimension, resulting in problems such as feature drift. In addition, existing anomaly detection models usually use uniform parameter configurations, which particularly lack the ability to dynamically adapt to the parameters of UAV inspection tasks, and cannot accurately match the detection needs of different sections and different environmental conditions.

[0004] In the positioning process, most existing technologies directly convert geographic information using a two-dimensional image coordinate system, which often overlooks the problem of spatial error accumulation caused by UAV pose drift and multi-frame data fusion.

[0005] It is evident that the aforementioned technical deficiencies make it difficult for existing inspection systems to meet the real-time monitoring requirements of complex railway scenarios in terms of detection sensitivity and positioning accuracy. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for monitoring railway anomalies based on unmanned aerial vehicle (UAV) inspection. This invention is achieved as follows: In a first aspect, embodiments of the present invention provide a railway anomaly monitoring method based on unmanned aerial vehicle (UAV) inspection, comprising: acquiring railway inspection data collected by a UAV and extracting monitoring instructions from the railway inspection data, wherein the railway inspection data includes multiple continuously collected spatiotemporal frame sequences, each spatiotemporal frame sequence containing data in spatial and temporal dimensions; parsing the monitoring instructions to obtain an anomaly monitoring parameter set corresponding to the monitoring instructions; performing spatiotemporal feature extraction processing on the multiple spatiotemporal frame sequences to generate spatial feature vectors and temporal feature vectors for each spatiotemporal frame sequence; based on the dynamic matching relationship between the anomaly monitoring parameter set and the spatial feature vectors and the temporal feature vectors, selecting at least one anomaly region from the multiple spatiotemporal frame sequences; calling an anomaly identification model to perform joint analysis of the spatiotemporal features of the anomaly region to generate railway anomaly monitoring results, and mapping the railway anomaly monitoring results to the physical coordinate system of the railway line to generate an anomaly location report.

[0007] Secondly, the present invention provides a railway anomaly monitoring system based on unmanned aerial vehicle (UAV) inspection, comprising: one or more processors; a memory; one or more computer programs; wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and when the one or more computer programs are executed by the processors, they implement the method described above.

[0008] The beneficial effects of the present invention: The embodiments of the present invention achieve a breakthrough improvement in the accuracy and intelligent decision-making of railway inspection anomaly detection through multi-dimensional spatiotemporal data fusion and dynamic parameter adaptation mechanism.

[0009] First, a multi-dimensional data coupling analysis framework was constructed based on the spatiotemporal frame sequence continuously collected by UAVs. By extracting spatial structure features and temporal evolution features in parallel, it breaks through the limitations of traditional single-dimensional detection methods in representing complex scenes.

[0010] Secondly, an innovative dynamic matching relationship model is established, which nonlinearly correlates and maps monitoring command parameters with spatiotemporal feature vectors, enabling abnormal region screening to have parameter adaptive decision-making characteristics, effectively solving the problem of insufficient adaptability of fixed threshold detection to multiple types of anomalies.

[0011] Furthermore, through a spatiotemporal joint analysis model, collaborative reasoning of cross-modal features was achieved, capturing abnormal evolution patterns in the temporal dimension while preserving spatial topological relationships, significantly improving the early identification capability of progressive hidden dangers such as crack propagation and component loosening.

[0012] Finally, by combining the inverse mapping mechanism of the physical coordinate system, a two-way interpretable link from data features to entity location was constructed, providing technical support for railway infrastructure maintenance that combines spatial positioning accuracy and anomaly evolution trend analysis.

[0013] In summary, the embodiments of the present invention form an intelligent diagnostic closed loop with environmental perception capabilities through hierarchical feature interaction and dynamic matching mechanisms, which greatly improves the reliability and predictability of railway anomaly detection in complex operating environments.

[0014] Other features will be described in part in the following description. These features will be partially discovered by those skilled in the art upon examination of the following content and figures, or may be learned through production or application. The features of the present application can be implemented and obtained by practice or use of various aspects of the methods, tools, and combinations listed in the detailed examples described below. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below.

[0016] Figure 1 This is a flowchart of a railway anomaly monitoring method based on unmanned aerial vehicle (UAV) inspection, provided by an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the composition of a railway anomaly monitoring system based on unmanned aerial vehicle (UAV) inspection, provided by an embodiment of the present invention. Detailed Implementation

[0018] The embodiments of the present invention will now be described with reference to the accompanying drawings. The terminology used in the implementation section of the embodiments of the present invention is for illustrative purposes only and is not intended to limit the scope of the invention.

[0019] In this embodiment of the invention, the executing entity of the railway anomaly monitoring method based on UAV inspection is a railway anomaly monitoring system based on UAV inspection, including but not limited to servers, personal computers, laptops, tablets, smartphones, etc., which communicate with the UAV. Figure 1 As shown, the method includes the following steps: Step S100: Acquire railway inspection data collected by the UAV and extract monitoring instructions from the railway inspection data. The railway inspection data includes multiple spatiotemporal frame sequences collected continuously, and each spatiotemporal frame sequence contains data in spatial and temporal dimensions.

[0020] Railway inspection data refers to multimodal monitoring data collected in real time by drones equipped with sensors during inspections of railway lines. This includes, but is not limited to, lidar scanning data, high-resolution camera video stream data, infrared thermal imaging data, and flight attitude data recorded by inertial measurement units. Spatiotemporal frame sequences are the fundamental building blocks of railway inspection data. Each sequence consists of multidimensional data collected by the drone within fixed time intervals. Spatial dimension data includes geographic coordinates, 3D point cloud coordinates, image pixel distribution, and track structure geometric parameters—information representing physical spatial characteristics. Temporal dimension data includes data acquisition timestamps, sensor sampling frequency, and time synchronization signals—information reflecting the temporal correlation of the data. For example, if a drone collects railway track images at a rate of 30 frames per second, the spatial dimension data for each frame includes rail surface texture features, bolt connection status, and track geometric deformation parameters, while the temporal dimension data includes the acquisition timestamp for each frame (e.g., millisecond-level timestamps between 12:00:00.000 and 12:00:02.000) and the time interval data between consecutive frames. When extracting monitoring commands, it is necessary to parse the preset inspection task parameters from the metadata header of the railway inspection data. The monitoring commands are execution commands generated by the UAV flight control system based on the preset inspection plan, specifically including control information such as inspection path planning parameters, sensor acquisition modes, data transmission protocols, and anomaly monitoring trigger conditions. For example, a monitoring command might contain parameterized instructions such as "fly along the track centerline at an altitude of 50 meters, collect track surface images every 10 meters, and trigger a full-power scan of the infrared sensor when the detected temperature exceeds 60 degrees Celsius." It should be noted that when acquiring railway inspection data, it is essential to ensure that UAV operation complies with aviation regulations, and an airspace use permit must be obtained from the relevant departments before data collection. Furthermore, data involving privacy or sensitive areas must be anonymized.

[0021] Step S200: Parse the monitoring command to obtain the set of abnormal monitoring parameters corresponding to the monitoring command.

[0022] Parameter parsing refers to the process of transforming the natural language descriptions or binary encoded control parameters in monitoring instructions into a standardized set of parameters that can be recognized by machines, using semantic analysis, regular expression matching, or structured data decoding techniques. The anomaly monitoring parameter set is the core rule base guiding subsequent data processing and anomaly detection. It includes quantitative indicators such as temperature thresholds, vibration frequency ranges, geometric deformation tolerance, and image contrast difference thresholds, as well as the logical relationships between these parameters. For example, when a monitoring instruction includes "trigger an alarm when the number of missing bolts at the track joint exceeds 3," the parameter parsing process needs to extract the spatial location conditions of "track joint," the detection object of "number of missing bolts," and the threshold parameter of "more than 3," and transform these elements into structured fields in the anomaly monitoring parameter set. Furthermore, parameter parsing also needs to handle the priority and combination relationships between multiple parameters. For example, a complex condition such as "activate a level-two warning when the temperature exceeds 50 degrees Celsius and the track's lateral displacement exceeds 5 millimeters" needs to be decomposed into temperature threshold parameters, displacement threshold parameters, and logical AND relationship combinations. In some embodiments, if the monitoring command contains vaguely described or unquantified parameters (such as "significant deformation" or "abnormal vibration"), a preset default parameter library must be called for matching. For example, "significant deformation" can be mapped to a parameter value where the vertical height change of the track exceeds 10 mm / m. Furthermore, the parameter parsing process must also verify the validity of the parameters. For instance, if the monitoring command requires the infrared sensor sampling frequency to exceed the hardware limit, it must be automatically corrected to the maximum supported frequency and a log record generated.

[0023] Step S300: Perform spatiotemporal feature extraction processing on the multiple spatiotemporal frame sequences to generate spatial feature vectors and temporal feature vectors for each spatiotemporal frame sequence.

[0024] Spatiotemporal feature extraction refers to extracting key features that characterize the railway's state from the original spatiotemporal frame sequence using feature engineering methods or deep learning models. Spatial feature vectors are used to describe the static attributes and dynamic changes of the railway structure in three-dimensional space. For example, edge and texture features in track images are extracted using convolutional neural networks (CNNs), or geometric parameters such as track gauge, track height, and horizontal inclination are calculated using point cloud processing algorithms. Temporal feature vectors are used to characterize the regularity of the railway's state evolution over time. For example, the temporal variation trend of vibration signal frequency is analyzed using long short-term memory networks (LSTMs), or the periodic fluctuation features of temperature data are extracted using Fourier transforms. Specifically, for three-dimensional point cloud data acquired by lidar, spatial feature extraction may include: first, denoising and registering the point cloud to eliminate coordinate offsets caused by changes in UAV flight attitude; second, converting the point cloud into a regularized three-dimensional voxel representation using voxel meshing; and finally, using three-dimensional convolutional layers to extract spatial features such as track surface roughness and bolt distribution density from the voxel data. For video data from cameras, temporal feature extraction can include motion estimation of optical flow changes between consecutive frames and calculation of time-varying parameters such as displacement velocity and vibration amplitude of track components. Furthermore, the fusion of spatiotemporal features can be achieved by modeling the topological relationships between spatial nodes and state propagation in the temporal dimension using graph neural networks (GNNs). For example, stress data from adjacent sleepers can be constructed into a graph structure to analyze the propagation patterns of stress waves in time and space.

[0025] Step S400: Based on the dynamic matching relationship between the anomaly monitoring parameter set and the spatial feature vector and the temporal feature vector, at least one anomaly region is selected from the multiple spatiotemporal frame sequences.

[0026] Dynamic matching refers to identifying regions deviating from normal states based on real-time extracted spatial and temporal feature vectors, combined with threshold conditions and logical rules defined in the anomaly monitoring parameter set, through similarity calculation or pattern matching algorithms. For example, if the track gauge measurement value in the spatial feature vector is 1438 mm (standard track gauge is 1435 mm) and the temporal feature vector shows that the deviation continues to increase within 5 seconds, then according to the rule in the anomaly monitoring parameter set that "track gauge deviation exceeds 3 mm and shows an increasing trend," the region is marked as an anomaly. In specific implementation, the Dynamic Time Warping (DTW) algorithm can be used to match the temporal feature vector with the preset anomaly pattern template in the parameter set, or the degree of deviation between the spatial feature vector and the normal sample feature library can be calculated through cosine similarity. For multi-parameter composite conditions, a hyperplane decision boundary needs to be constructed in the multi-dimensional feature space. For example, a Support Vector Machine (SVM) classifier can be used to segment the three-dimensional feature space of "temperature-vibration-deformation," and the physical coordinates corresponding to data points falling in the anomaly region can be determined as the anomaly region. Furthermore, the dynamic matching process also needs to consider the confidence weight of the feature vector. For example, the spatial resolution of infrared thermal imaging data is relatively low, and the weight of its temperature features should be lower than that of the deformation features measured by high-precision strain sensors. The selected abnormal areas need to record their spatiotemporal coordinate range, abnormality type code, and severity level, such as "between K325+500 and K325+600, the lateral displacement of the track exceeds the limit, level II".

[0027] Step S500: Call the anomaly identification model to perform joint analysis on the spatiotemporal characteristics of the anomaly area, generate railway anomaly monitoring results, and map the railway anomaly monitoring results to the physical coordinate system of the railway line to generate an anomaly location report.

[0028] The anomaly identification model is an artificial intelligence model trained on multimodal data. It employs a spatiotemporal attention mechanism or a fusion network architecture to perform cross-modal correlation analysis on the spatial and temporal feature vectors of anomaly areas to distinguish between genuine anomalies and false alarms caused by sensor noise. For example, when the infrared signature of a certain area shows high temperature but the vibration signature does not show any anomalies, the model can combine historical data to determine whether the high temperature is caused by environmental factors such as direct sunlight, rather than a track structure failure. When generating railway anomaly monitoring results, the model outputs structured data including anomaly type (such as gauge overrun, missing bolts, roadbed settlement), anomaly confidence level (such as 92% probability of a genuine defect), and suggested remedial measures (such as speed limit operation or immediate shutdown). The process of mapping to the physical coordinate system requires converting the logical coordinates of the abnormal area (such as data frame index and offset relative to the UAV's starting point) into the absolute geographical coordinates of the railway line (such as latitude and longitude, and mileage markers). Specific methods include: establishing a correspondence table between spatiotemporal frame sequences and geographical coordinates using the fusion positioning results of UAV GNSS positioning data and inertial navigation systems (INS); and performing coordinate projection transformation using a 3D map of the railway environment constructed using the SLAM algorithm. The final anomaly location report uses a visual format to mark the spatial distribution of anomaly points, for example, highlighting deformed sections in the digital twin railway model, and generating a standardized document containing anomaly descriptions, coordinate information, on-site image attachments, and maintenance recommendations. This report is automatically distributed to relevant responsible units through the railway operation and maintenance management system, driving the generation of inspection work orders and the scheduling of maintenance resources.

[0029] As one implementation method, step S200 involves parsing the monitoring command to obtain a set of abnormal monitoring parameters corresponding to the monitoring command, including: Step S210: Perform multimodal data preprocessing on the monitoring command to generate preprocessed command text data; wherein, the multimodal data preprocessing includes noise filtering, time sequence alignment and format standardization.

[0030] Multimodal data preprocessing refers to the unified processing of various modal input data, such as text, voice, and gesture control signals, mixed in the original monitoring commands of UAVs. Noise filtering involves eliminating electromagnetic interference in sensor signals, environmental noise in voice commands, and non-standard characters or redundant control characters in text commands. For example, when the monitoring command contains JSON format text transmitted wirelessly, if some fields are garbled due to channel interference (such as "tempe6rature" in "{"tempe6rature_threshold": 50}"), noise filtering needs to identify and correct them to the valid key-value pair "temperature_threshold". Timing alignment is used to solve the timing misalignment problem caused by asynchronous clocks of acquisition devices or transmission delays in multimodal data. For example, if the UAV operator triggers a gesture to select the infrared scanning mode at the same time as the voice command "Take a photo of the track seam every 10 seconds from now on", the voice recognition text and gesture signal need to be bound to the same execution time starting point according to the timestamp. Format standardization converts heterogeneous data into a unified instruction text data format. For example, it converts voice commands into UTF-8 encoded text and maps gesture control signals to preset keywords (such as "infrared mode activation"), ultimately generating structured instruction text data whose fields include instruction type, parameter list, execution conditions, and priority identifier.

[0031] Step S220: Call the parameter encoder to perform hierarchical feature extraction on the preprocessed instruction text data to generate global parameter features and local parameter features of the monitoring instruction.

[0032] A parametric encoder is a feature extraction model based on deep neural networks. It separates global parametric features representing the overall monitoring task objective and local parametric features describing specific detection conditions from instruction text data through a multi-layered transformation structure. Global parametric features include macro-level control parameters such as the geographical scope of the inspection task, monitoring cycle, and sensor configuration. For example, the instruction text "continuous 24-hour multi-sensor joint monitoring of the section from K320+000 to K325+000" is encoded as global geographical coordinate boundaries (starting point K320+000, ending point K325+000) and a time span (24 hours). Local parametric features correspond to the details of detection rules for specific anomaly types, such as the displacement threshold (5 mm) and duration (10 minutes) in "track lateral displacement exceeds 5 mm for 10 minutes". Hierarchical feature extraction is achieved through cascaded operations of word embedding layer, convolutional layer and pooling layer: word embedding layer maps natural language words (such as "displacement" and "exceeding limit") in instruction text into high-dimensional vectors; convolutional layer extracts local semantic associations between adjacent words, such as the conditional relationship between "temperature" and "above" in "temperature is higher than 50 degrees Celsius"; pooling layer further aggregates key parameters to form a global feature vector.

[0033] Specifically, step S220, which involves calling a parameter encoder to perform hierarchical feature extraction on the preprocessed instruction text data to generate global and local parameter features of the monitoring instruction, may include: Step S221: Perform semantic segmentation on the instruction text data to generate multiple semantic units and contextual dependencies between semantic units.

[0034] Semantic segmentation uses natural language processing techniques to divide continuous instruction text into independent and complete semantic units. Each semantic unit corresponds to a monitoring task sub-objective or parameter condition. For example, the instruction text "Monitor track gauge changes, and record the position when the lateral displacement exceeds 5 mm and lasts for 10 minutes" is segmented into semantic unit 1 (monitoring object: track gauge change), semantic unit 2 (condition 1: lateral displacement > 5 mm), semantic unit 3 (condition 2: duration ≥ 10 minutes), and semantic unit 4 (action: record position). Contextual dependencies describe the logical connections between semantic units. For example, semantic unit 2 and semantic unit 3 form parallel conditions through "and", and both serve as the triggering premise for semantic unit 4. Semantic segmentation employs a combination of dependency parsing trees and named entity recognition to identify verb-parameter structures (such as "exceeds + 5 mm") and conditional branches guided by conjunctions.

[0035] Step S222: Perform time encoding processing on each semantic unit to generate a timestamp association vector for each semantic unit. The time encoding processing aims to assign execution order or effective time period information to the semantic units in the time dimension.

[0036] For example, the timestamp-associated vector encoding of the semantic unit "Activate HD camera daily from 08:00 to 18:00" is set to start time 08:00, end time 18:00, and cycle pattern "daily". Time encoding generates a timestamp vector containing fields such as start time, end time, duration, and cycle frequency by parsing explicit time descriptions (e.g., "every 2 hours" or "lasts 30 minutes") or implicit time associations in the instruction (e.g., mapping "immediately" to 0 delay in "take a picture immediately after detecting an anomaly"). For semantic units without explicit time information, the monitoring cycle in the global parameter features is inherited by default, or a pre-configured default value is used.

[0037] Step S223: Perform transform encoding on each semantic unit to generate a domain adaptation vector for each semantic unit.

[0038] Transform coding is used to convert semantic units described in general natural language into standardized parameter representations for the railway inspection domain. For example, the semantic unit "track surface crack" needs to be adapted to "track surface crack defect code T341" in the domain terminology library. Its domain adaptation vector includes defect type coding, detection method (visual inspection), and severity grading rules (crack length > 10 mm is Level III). Transform coding is achieved through domain dictionary matching and semantic similarity calculation: first, keywords in the semantic unit (such as "crack" and "displacement") are mapped to predefined railway terms; second, the parameter units are adjusted according to the context (such as "5 mm" does not need to be converted, while "5 inches" needs to be converted to 127 mm); finally, domain-specific detection logic is added, such as "temperature exceeds threshold" in track monitoring, which by default refers to the railhead surface temperature rather than the ambient temperature.

[0039] Step S224: Fuse the timestamp association vector and the domain adaptation vector to generate the global parameter features.

[0040] The fusion process integrates information from the time and domain dimensions through feature concatenation and weighted summation. For example, the "track gauge detection code G002" in the domain adaptation vector is concatenated with the "periodic pattern: once per hour" in the timestamp association vector, and then processed by a fully connected layer to generate global parameter features representing "track gauge detection is performed once per hour". The weight allocation strategy is dynamically adjusted according to the parameter type. For example, the timestamp vector of time-sensitive parameters (such as "continuous alarm until manual confirmation") has a higher weight, while the domain adaptation vector of spatially sensitive parameters (such as "monitoring section K320+000 to K325+000") has a higher weight.

[0041] Step S225: Based on the context dependency, attention weights are assigned to the multiple semantic units to generate the local parameter features.

[0042] The attention weight allocation module assigns different weights to each semantic unit based on the strength of logical dependencies between semantic units (such as logical "AND / OR" relationships in conditional branches) and execution priority. For example, in the parallel conditions consisting of semantic unit 2 (displacement > 5 mm) and semantic unit 3 (duration 10 minutes), both have an attention weight of 0.5; if the instruction is changed to "displacement > 5 mm or duration 10 minutes", the weights are adjusted to 0.3 and 0.7 to reflect the looseness of the logical "OR". The attention weights are dynamically determined by calculating the dependency distance between semantic units, the type of conjunctions (such as "AND" and "OR"), and the frequency of rules in historical tasks. The weighted semantic unit vectors are aggregated by an LSTM network to generate local parametric features describing the details of the detection conditions, such as the weighted combination features of "displacement threshold 5 mm" and "duration 10 minutes".

[0043] Step S230: Associate and map the global parameter features with a predefined railway anomaly type library to determine the target anomaly type of the monitoring instruction.

[0044] The predefined railway anomaly type library is a structured knowledge base that defines the attributes, detection methods, and associated parameters of anomaly types such as track geometric deformation, component loss, and subgrade settlement in ontology form. The association mapping process calculates the similarity between global parameter features and features of each category in the anomaly type library to select the target anomaly type with the highest matching degree. For example, if the global parameter features include "using a combination of LiDAR and high-definition cameras" and "monitoring period is 30 consecutive days during the rainy season," the association mapping module determines "subgrade settlement" as the target anomaly type based on the historical data showing that "subgrade settlement" anomalies often occur during the rainy season and rely on 3D deformation detection. In specific implementation, a cosine similarity algorithm or graph neural network can be used to match the global parameter features with the embedding vectors of each node in the anomaly type library. At the same time, a rule engine is used to verify logical consistency. For example, when the global parameter specifies "using only infrared sensors" but is associated with the "bolt loss" type that relies on visual detection, a conflict warning is triggered and the mapping result is recalculated.

[0045] Step S240: Based on the local parameter features and the spatiotemporal constraints of the target anomaly type, generate the anomaly monitoring parameter set, which includes a spatially sensitive area threshold, a temporal continuity weight, and an anomaly intensity determination threshold.

[0046] Spatiotemporal constraints refer to the regular restrictions on the spatial distribution and temporal evolution of target anomaly types. For example, track geometric deformation usually manifests as a continuous offset in a local area, while bolt loss is a sudden anomaly at discrete points. The spatially sensitive area threshold defines the spatial range within which anomalies may occur. For example, the spatially sensitive area for the "gauge deviation" anomaly is the set of measurement points within 16 mm of the center of the rail head on the inner side of the rail. Temporal continuity weights are used to quantify the impact of the duration of the anomaly on the detection results. For example, "subgrade settlement" requires a cumulative settlement exceeding 10 mm over three consecutive days to trigger an alarm, with a daily settlement weight of 0.4. Anomaly intensity judgment thresholds include numerical boundaries (such as a temperature threshold of 50 degrees Celsius) and logical conditions (such as "vibration frequency between 2-10 Hz and amplitude greater than 5 mm"). When generating an anomaly monitoring parameter set, the original values ​​in the local parameter features (such as "displacement exceeds 5 mm") need to be calibrated with the default parameters in the target anomaly type library. For example, when the duration of a local parameter is not specified, the minimum duration of the anomaly type is automatically filled in (such as "track gauge deviation" which needs to last for 30 minutes by default).

[0047] As one implementation, step S300 involves performing spatiotemporal feature extraction processing on the plurality of spatiotemporal frame sequences to generate a spatial feature vector and a temporal feature vector for each spatiotemporal frame sequence, including: Step S310: Divide each spatiotemporal frame sequence into spatial grid cells and temporal window segments.

[0048] Spatial grid cells refer to dividing a single frame of spatial data into multiple regular or irregular polygonal regions according to preset rules. Each region corresponds to a sub-region in the physical space of a railway line. For example, a track image captured by a drone can be divided into 1-meter × 1-meter grids, with each grid containing spatial information such as sleeper spacing, bolt status, and ballast distribution within that region. Time window segments, on the other hand, divide a continuous time-dimensional data stream into segments of fixed or variable duration. For example, infrared thermal imaging data can be extracted at 10-second intervals, with each segment containing the temporal changes in track temperature within that time period. During the division, it is essential to ensure that the spatial grid cells and time window segments are strictly aligned in the spatiotemporal coordinate system. For instance, when a time window segment corresponds to 12:00:00 to 12:00:10, the geographical coordinate range of its spatial grid cells must be consistent with the drone's flight trajectory and the sensor's field of view coverage area during that time period.

[0049] Step S320: Perform spatial encoding processing on the spatial grid cells to generate a spatial feature matrix for each spatial grid cell.

[0050] Spatial coding processes extract local structural features and global topological relationships within spatial grid cells using convolutional neural networks or geometric transformation algorithms. For example, for spatial grid cells constructed from LiDAR point cloud data, voxelization is first used to convert irregular point clouds into 3D raster representations. Subsequently, geometric parameters such as track gauge, track height, and horizontal tilt angle are extracted through 3D convolutional layers, generating a 64×64×32 spatial feature matrix, where each channel corresponds to a spatial attribute (such as height, density, or curvature). For spatial grid cells of image data, spatial coding may include edge detection, texture analysis, and object detection steps. For instance, a ResNet-50 model can be used to extract feature maps of cracks on the track surface, outputting a 256×256×1024 feature matrix, where the last dimension represents feature channels at different levels (such as edges, textures, and semantics).

[0051] Step S330: Perform time encoding processing on the time window segments to generate a time feature sequence for each time window segment.

[0052] Temporal coding aims to capture the dynamic evolution of data within a time window segment, such as modeling periodic or trend changes in parameters like temperature and vibration using recurrent neural networks or temporal convolutional models. Taking track lateral displacement monitoring as an example, the displacement data sequence within the time window segment is first differentially processed to eliminate baseline drift. Then, it is input into a bidirectional LSTM network to extract forward and backward temporal dependency features, outputting a 128-dimensional temporal feature sequence. The feature vector at each time step includes the displacement rate of change, acceleration, and correlation indicators with historical data. For data acquired simultaneously by multiple sensors, temporal coding requires multi-source temporal alignment and feature fusion. For example, after aligning the data streams from vibration sensors and strain gauges using millisecond-level timestamp interpolation, a multi-head self-attention mechanism is used to model cross-modal temporal correlations.

[0053] Step S340: Call the spatiotemporal encoder to perform cross-dimensional fusion of the spatial feature matrix and the temporal feature sequence to generate a joint spatiotemporal feature vector of the spatiotemporal frame sequence; wherein, the spatiotemporal encoder includes a three-dimensional convolutional kernel and a bidirectional recurrent network layer, used to synchronously capture spatial structure changes and temporal evolution patterns.

[0054] Cross-dimensional fusion constructs high-dimensional features that characterize the spatiotemporal correlation of railway conditions by jointly analyzing the static structural information of the spatial feature matrix and the dynamic evolution of the temporal feature sequence. For example, the three-dimensional convolutional kernel of the spatiotemporal encoder performs multi-scale feature extraction in the spatial dimension (e.g., a 3×3×3 convolutional kernel captures local geometric deformation, and a 5×5×5 convolutional kernel extracts regional ballast distribution patterns), while sliding along continuous time window segments in the temporal dimension to capture the propagation process of track deformation. The bidirectional recurrent network layer scans the temporal feature sequence in both forward and reverse directions to establish the cause-and-effect correlation of vibration signals (e.g., abnormal vibration in the first 10 seconds may lead to gauge deviation in the following 5 seconds). The fused joint spatiotemporal feature vector can be represented as a 512-dimensional dense vector, where each dimension corresponds to a cross-modal spatiotemporal feature combination, such as the correlation pattern of "abnormal displacement of sleeper region A at the 3rd second and increased vibration of region B at the 5th second".

[0055] As one implementation, step S340, which involves calling a spatiotemporal encoder to perform cross-dimensional fusion of the spatial feature matrix and the temporal feature sequence to generate a joint spatiotemporal feature vector of the spatiotemporal frame sequence, includes: Step S341: Input the spatial feature matrix into the three-dimensional convolution kernel to extract multi-scale spatial features and generate spatial pyramid features.

[0056] The 3D convolutional kernel processes the spatial feature matrix in parallel through convolutional layers of different scales. For example, 1×1×1, 3×3×3, and 5×5×5 convolutional kernels are used to extract pixel-level details, local structures, and region-level contextual features, respectively. These multi-scale features are upsampled and stitched together to form a spatial pyramid feature. For instance, feature maps output from the three scales (sizes of 256×256×64, 128×128×128, and 64×64×256, respectively) are adjusted to a uniform resolution and stitched along the channel dimension to generate a feature pyramid containing multi-granularity spatial information, with dimensions of 256×256×448. This feature pyramid can simultaneously characterize the microscopic state of the track bolts (such as the degree of corrosion) and their macroscopic distribution patterns (such as the clustering patterns of bolt-missing areas).

[0057] Step S342: Input the time feature sequence into the bidirectional recurrent network layer to perform temporal dependency modeling and generate time context features.

[0058] The bidirectional recurrent network layer consists of a forward LSTM and a backward LSTM. The forward LSTM progressively calculates the hidden states from the beginning to the end of the time window segment, capturing the impact of historical information on the current moment (such as the cumulative effect of orbital expansion caused by sustained high temperatures). The backward LSTM processes the time series in reverse, modeling the dependence of future states on the current moment (such as how subsequently detected deformation can correct misjudgments at the current moment). The hidden states from both are concatenated to generate a temporal context feature. For example, for a 128-dimensional time feature sequence, the bidirectional LSTM outputs a 256-dimensional feature vector, where the first 128 dimensions encode the historical context, and the last 128 dimensions encode the future context. This feature can identify the causal relationships of abnormal events within the time window segment, such as determining whether a sudden increase in vibration frequency is a precursor to a transient disturbance or a persistent fault.

[0059] Step S343: Perform feature concatenation on the spatial pyramid features and the temporal context features to generate initial fused features.

[0060] Feature concatenation links features from the spatial and temporal dimensions along the channel dimension. For example, concatenating spatial pyramid features (256×256×448) with temporal context features (256×256×256) generates an initial fused feature matrix with 704 channels. This process preserves the original spatial resolution and time step information, ensuring that subsequent processing can correlate anomalous patterns at specific spatial locations and time points. For instance, channel 500 at a location (x,y) in the initial fused features might correspond to "a joint anomaly of temperature gradient and vibration amplitude in region (x,y) between the 3rd and 5th seconds."

[0061] Step S344: The initial fusion features are dynamically weighted through a gating attention mechanism to generate the joint spatiotemporal feature vector; wherein, the weights of the gating attention mechanism are jointly controlled by the spatially sensitive region threshold and the temporal continuity weight in the anomaly monitoring parameter set.

[0062] The gated attention mechanism calculates the importance score of each feature channel based on spatially sensitive area thresholds (e.g., 0.9 for track joints and 0.3 for straight sections) and temporal continuity weights (e.g., 0.8 for anomalies lasting 10 minutes and 0.2 for transient fluctuations). Specifically, the spatially sensitive area threshold is converted into a spatial attention mask matrix, which is multiplied point-by-point with the initial fused features to enhance the feature response of key areas. The temporal continuity weights are then used to generate a temporal attention vector through the Sigmoid function, scaling the feature channels. For example, when the anomaly monitoring parameter set specifies "focusing on monitoring missing sleeper bolts," the gated attention mechanism will increase the weight of bolt location-related feature channels while suppressing the contribution of irrelevant features such as ballast distribution. The dynamically weighted features are compressed into a 512-dimensional joint spatiotemporal feature vector through global average pooling. Each dimension in this high-dimensional space encodes the spatiotemporal coupling pattern of the railway state, which can be directly input into the downstream anomaly classifier for decision-making.

[0063] As one implementation, step S400, based on the dynamic matching relationship between the anomaly monitoring parameter set and the spatial feature vector and the temporal feature vector, filters out at least one anomaly region from the plurality of spatiotemporal frame sequences, including: Step S410: Calculate the regional matching degree between the spatial feature vector and the spatial sensitive area threshold in the anomaly monitoring parameter set.

[0064] The spatially sensitive area threshold defines the range of physical areas in a railway line that require key monitoring and their corresponding allowable fluctuation range of characteristics. For example, the track gauge threshold at track joints is set to 1435±2 mm, and the horizontal inclination threshold for turnout areas is set to 0.5 degrees. The area matching degree is obtained by calculating the degree of conformity between the eigenvalues ​​of each spatial grid cell in the spatial feature vector and the threshold. Specifically, normalized Euclidean distance or cosine similarity algorithms are used to quantify the matching degree. Taking track geometric deformation detection as an example, if the spatial feature vector of a certain spatial grid cell contains a track gauge measurement of 1438 mm and a horizontal inclination angle of 0.7 degrees, then its matching degree with the "track gauge threshold of 1435±1 mm for straight sections" in the spatially sensitive area threshold is (1438-1436) / 1=2 (the matching degree is set to 0 when it exceeds the threshold range), while its matching degree with the "horizontal inclination threshold of 0.5 degrees for turnout areas" is 0.7-0.5=0.2 degrees (the matching degree is inversely proportional to the deviation). Furthermore, the matching degree calculation needs to be combined with the spatial weight distribution. For example, the matching degree weight coefficient at the track joint is 0.9, while the weight coefficient for the straight line segment is 0.3. The final regional matching degree can be expressed as the weighted average of the matching degree of each grid unit.

[0065] Step S420: Calculate the time matching degree between the time feature vector and the time continuity weight in the anomaly monitoring parameter set.

[0066] The temporal continuity weight is used to measure the impact of the persistence and evolution of abnormal events over time on the detection results. For example, for track gauge deviation anomalies that need to last for more than 10 minutes, the temporal continuity weight is set to 0.8, while the weight for instantaneous vibration events is 0.2. The temporal matching degree is obtained by analyzing the degree of fit between the duration and fluctuation frequency of the abnormal signal in the temporal feature vector and the temporal continuity weight. Taking temperature anomaly detection as an example, if the temporal feature vector shows that the temperature in a certain area exceeds 50 degrees Celsius within three consecutive time windows (30 minutes), and the temporal continuity weight requires "lasting for more than 20 minutes", then the temporal matching degree can be calculated as the ratio of the actual duration (30 minutes) to the required duration (20 minutes), which is 1.5. After being compressed to the 0-1 range by the Sigmoid function, a matching degree of 0.82 is obtained. For multi-parameter time constraints (such as "the temperature rise rate is >2℃ / min in the first 5 minutes and remains stable above the threshold in the following 5 minutes"), the time matching degree needs to be calculated in segments and then aggregated according to logical rules. For example, the sliding window method can be used to verify the condition satisfaction frame by frame and calculate the compliance rate.

[0067] Step S430: Generate a comprehensive anomaly score for each spatiotemporal frame sequence based on the weighted sum of the regional matching degree and the temporal matching degree.

[0068] During the weighted summation process, the weight coefficients of regional matching degree and temporal matching degree are dynamically adjusted by the priority of the detection targets in the anomaly monitoring parameter set. For example, in the roadbed settlement detection task, the weight of spatial features (such as settlement amount) is set to 0.7, and the weight of temporal features (such as continuous settlement rate) is set to 0.3; while in bolt missing detection, the weight of spatial features (accuracy of missing location) may be increased to 0.9, and the weight of temporal features may be reduced to 0.1. The formula for calculating the comprehensive anomaly score is: Score = Regional matching degree × Spatial weight + Temporal matching degree × Temporal weight. Taking a spatiotemporal frame sequence with a regional matching degree of 0.85 (spatial weight 0.7) and a temporal matching degree of 0.6 (temporal weight 0.3) as an example, its comprehensive anomaly score is 0.85 × 0.7 + 0.6 × 0.3 = 0.805. This score reflects the degree to which the frame sequence deviates from the normal state as a whole, and a high score indicates a high probability of the existence of target anomaly types. In addition, the weighting coefficients can be adaptively optimized based on feedback from historical detection results. For example, when a certain type of anomaly is frequently missed, the weighting coefficient of its corresponding feature is automatically increased.

[0069] Step S440: Compare the comprehensive anomaly score with the anomaly intensity judgment threshold, and select spatiotemporal frame sequences with scores exceeding the threshold as candidate anomaly regions.

[0070] The threshold for determining anomaly intensity is set based on railway safety standards and historical data statistics. For example, the threshold is divided into three levels: 0.7-0.8 is the observation level (generating an early warning record), 0.8-0.9 is the warning level (triggering manual review), and above 0.9 is the emergency level (immediate shutdown). If the comprehensive anomaly score of a spatiotemporal frame sequence is 0.85 and the threshold is set to 0.8, then the frame is marked as a candidate anomaly region for the warning level, and its spatiotemporal coordinates (e.g., K325+500 to K325+550, 12:00:00-12:10:00) and anomaly type code (e.g., track gauge over-limit code G001) are recorded in the candidate list. For candidate regions spanning multiple consecutive time window segments, their start and end timestamps must be retained to support subsequent time series analysis. The threshold comparison process also supports a dynamic adjustment mechanism, such as automatically lowering the temperature anomaly threshold in rainy or snowy weather to improve detection sensitivity.

[0071] Step S450: Perform cluster analysis based on the spatiotemporal continuity of the candidate abnormal regions, and merge the spatiotemporal frame sequences of adjacent regions to generate the at least one abnormal region.

[0072] Spatiotemporal continuity refers to the spatial proximity and temporal coherence of candidate anomaly regions. For example, in three consecutive time windows (12:00-12:10, 12:10-12:20, and 12:20-12:30), spatially adjacent gauge exceedance areas may belong to the same continuously expanding anomaly event and need to be merged into a unified anomaly region. Cluster analysis quantifies the spatiotemporal distance between candidate regions, aggregating scattered anomaly frame sequences into a complete anomaly event description, avoiding misjudgments or duplicate alarms caused by data fragmentation. The clustering results should output the precise boundaries, duration, and severity indicators of the anomaly region. For example, the merged anomaly region may cover the interval from K325+480 to K325+600, with a duration of 12:00-12:40 and a maximum gauge deviation of 8 mm.

[0073] As one implementation, step S450, which involves performing cluster analysis based on the spatiotemporal continuity of the candidate anomaly regions, may include: Step S451: Extract the spatial coordinates and timestamp information of the candidate anomaly region to construct a spatiotemporal distance matrix.

[0074] The spatiotemporal distance matrix is ​​an N×N symmetric matrix (N being the number of candidate regions), where each element represents the spatiotemporal distance between any two candidate regions. Spatial coordinates are represented by railway mileage markers (e.g., K325+500) and lateral offsets (left / right distance from the track centerline). Timestamp information includes start and end times. The formula for calculating the spatiotemporal distance is: Distance = α × Spatial Euclidean Distance + β × Time Interval Distance, where α and β are normalized weighting coefficients (usually α+β=1). For example, the spatial Euclidean distance between candidate region A (K325+500, 1.2 meters left, 12:00-12:10) and candidate region B (K325+520, 1.5 meters left, 12:10-12:20) is √[(520-500)]. 2 + (1.5-1.2) 2 =20.01 meters, the time interval is 12:10-12:10=0 minutes. If α=0.6 and β=0.4, then the spatiotemporal distance is 0.6×20.01 + 0.4×0=12.006.

[0075] Step S452: Calculate the region similarity based on the spatial Euclidean distance and time interval distance in the spatiotemporal distance matrix.

[0076] Region similarity is inversely proportional to the spatiotemporal comprehensive distance, and is usually mapped to the 0-1 range using a Gaussian kernel function. The formula is: Similarity = exp(-distance) 2 / (2σ 2 )), where σ is a scale parameter that controls the rate of similarity decay. For example, when σ=15, the similarity between regions A and B is exp(-(12.006)). 2 / (2×15 2 The similarity score is 0.72, indicating that the two have a high degree of similarity. For candidate regions with overlapping times (such as region C with a time of 12:05-12:15), the time interval between region C and region A is 12:05-12:10=5 minutes. The similarity calculation should consider both spatial proximity and temporal overlap to avoid misjudging different stages of the same event as independent anomalies.

[0077] Step S453: Group the regions based on their similarity using a density clustering algorithm to generate initial clusters.

[0078] Density clustering algorithms (such as DBSCAN) identify high-density regions by setting a neighborhood radius (eps) and a minimum number of samples (min_samples). For example, setting eps=0.5 (similarity threshold of approximately 0.6) and min_samples=3 means that a candidate region is only grouped into the same cluster if there are at least three other regions within its similarity neighborhood. This method can effectively distinguish between real anomalies (dense, continuous regions) and random noise (isolated regions). Taking four candidate regions as an example, their similarity matrix shows that the similarity between regions A, B, and C is all higher than 0.6, while the similarity between region D and other regions is lower than 0.3. Therefore, A, B, and C are clustered into the same initial cluster, and D is considered a noise point. The geometric center of the initial cluster can represent the core location of the anomaly. For example, the cluster center coordinates are K325+510, 1.4 meters to the left, with a time span of 12:00-12:30.

[0079] Step S454: Perform boundary optimization on each initial cluster to remove discrete noise points and expand the continuous coverage area.

[0080] Boundary optimization incorporates candidate points in edge regions into the cluster by calculating the convex hull or α-shape polygon of the cluster. For example, the initial cluster contains five candidate regions from K325+500 to K325+550, but its convex hull calculation shows that the actual coverage range can reach K325+495 to K325+555, so the boundary is extended to include these locations. Simultaneously, outlier detection algorithms (such as LOF) are used to identify and remove members with similarity below the cluster average. For instance, a candidate region may be close to the cluster center in spatiotemporal distance, but its orbital deviation value is significantly lower than other members, potentially indicating a false positive that needs to be removed. The optimized cluster exhibits smooth spatiotemporal boundaries and consistent anomaly intensity characteristics.

[0081] Step S455: Map the optimized clusters to the topology of the railway line to generate anomaly region boundaries with physical properties.

[0082] Railway topology includes geographical elements such as track alignment, turnout locations, bridges, and tunnels. The mapping process requires coordinate calibration of the geometric boundaries of clusters with the topological elements. For example, the original boundary of a cluster might be from K325+500 to K325+600, but the actual line in this section includes both curved and straight sections. During mapping, the boundary shape needs to be adjusted according to the track design drawings to ensure that the description of the abnormal area matches the actual physical structure. The final generated abnormal area boundaries are stored in GIS polygon format, with attribute fields including maximum anomaly intensity, average duration, and associated detection device number, supporting visualization and work order dispatch in the railway operation and maintenance system.

[0083] As one implementation method, step S500 involves calling an anomaly identification model to perform joint analysis of the spatiotemporal characteristics of the anomaly region, generating railway anomaly monitoring results, including: Step S510: Input the joint spatiotemporal feature vector of the abnormal region into the pre-trained spatiotemporal graph neural network.

[0084] The pre-trained spatiotemporal graph neural network is a deep learning model that integrates graph structure learning and temporal modeling. Its nodes represent the spatial locations of railway lines (such as sleepers, turnouts, and bridges), and edges represent spatial adjacency or physical connections (such as bolted connections between sleepers). The temporal dimension captures the evolution of anomalies through dynamic updates of node states. For example, for a joint spatiotemporal feature vector (dimension 512) of an anomaly region, the spatiotemporal graph neural network first maps it to initial features of graph nodes. Each node corresponds to one of the 10 sleeper locations between K325+500 and K325+600, and each node feature includes spatiotemporal aggregated values ​​of gauge, temperature, and vibration parameters. The model transmits node information through graph convolutional layers. For example, the anomalous vibration characteristics of sleeper A will affect the node state updates of adjacent sleepers B and C, thus modeling the spatial propagation effect of anomalies.

[0085] Step S520: Iteratively optimize the spatial features in the joint spatiotemporal feature vector through the node update layer of the spatiotemporal graph neural network to obtain the spatially optimized features.

[0086] The node update layer employs a gated graph attention mechanism, dynamically adjusting feature transfer weights based on the spatial distance and physical connection strength between nodes. For example, sleeper A is 0.6 meters away from sleeper B and directly connected by bolts, so its attention weight is set to 0.9; while sleeper A is not directly connected to sleeper C and is 2 meters away, so its weight is reduced to 0.3. In each iteration, node features are updated by aggregating the weighted features of neighboring nodes and its own historical state. For example, after the t-th iteration, sleeper A's feature vector incorporates the latest states of its three neighboring sleepers and its own features at time t-1. After three iterations of optimization, the spatially optimized features can highlight abnormal core areas (such as sleeper A's features significantly deviating from the normal range) and suppress edge noise (such as sleeper C's features tending towards normal values).

[0087] Step S530: The time features in the joint spatiotemporal feature vector are modeled by the time propagation layer of the spatiotemporal graph neural network to obtain the time transition features.

[0088] The time propagation layer employs gated cyclic units (GRUs) to capture the evolution of node states within a time window. For example, after inputting the feature sequences of sleeper A for five consecutive time slices (12:00-12:05 to 12:20-12:25) into the GRU, its hidden state encodes the cumulative process of the node's anomaly intensity (e.g., temperature gradually rising from 48℃ to 52℃). The time transition features are generated by linearly combining the node state at the current time step with historical states. For example, the time transition feature of sleeper A at time t = 0.7 × current feature + 0.3 × feature at time t-1, with weighting coefficients dynamically calculated by the GRU gating mechanism. This process can identify whether the anomaly is persistent (e.g., a continuous rise in temperature) or transient (e.g., a single vibration impact).

[0089] Step S540: Fuse the spatial optimization features and the temporal transition features to generate a semantic description vector for the abnormal region.

[0090] Feature fusion employs a channel attention mechanism, assigning adaptive weights to the spatial and temporal dimensions. For example, spatially optimized features (256 dimensions) and temporally transitional features (256 dimensions) are concatenated and input into a fully connected layer, generating an attention vector with a spatial weight of 0.6 and a temporal weight of 0.4. After weighted summation, a 512-dimensional semantic description vector is obtained. This vector integrates the spatial distribution pattern of anomalies (e.g., sleeper A as the core anomaly point) with temporal evolution characteristics (e.g., the anomaly intensity increases by 5% per hour), and can be directly mapped to a natural language description: "The track gauge in the section from K325+500 to K325+520 continues to widen, accompanied by a temperature increase of 2°C per hour. It is recommended to check the tightness of the sleeper bolts." Step S550: Match the semantic description vector with the railway equipment knowledge base to determine the anomaly type and impact level.

[0091] The railway equipment knowledge base is a structured database containing standard anomaly features, handling procedures, and historical cases. It uses graph embedding technology to encode heterogeneous data such as text, images, and parameters into a unified vector space. For example, the semantic description vector is matched with the embedded vectors of anomaly types such as "track gauge exceeding the limit," "missing bolts," and "subgrade settlement" in the knowledge base through cosine similarity calculation. If the similarity with "track gauge exceeding the limit" reaches 0.92 (threshold 0.85), the anomaly type is determined to be track gauge exceeding the limit, and the impact level is mapped to Level II (requiring repair within 48 hours) based on the exceeding value (e.g., 7 mm).

[0092] As one implementation method, step S550, which involves matching the semantic description vector with a railway equipment knowledge base to determine the anomaly type and impact level, may include: Step S551: Extract the embedded representation of standard anomaly features from the railway equipment knowledge base.

[0093] The embedding representations of standard anomaly features are generated through a pre-trained BERT model or graph autoencoder. For example, the embedding vector (dimension 512) for "track gauge exceeding limits" encodes the definition of the anomaly (track gauge deviation > 5 mm), typical spatial distribution (mostly occurring on curved sections), associated sensor parameters (LiDAR ranging anomaly), and historical handling records (average repair time 3 hours). The embedding vectors stored in the knowledge base are classified according to anomaly type. For example, the embedding vector for "missing bolts" includes the topological relationship of bolt positions, the threshold for the number of missing bolts (e.g., 3 consecutive missing bolts), and infrared image feature patterns.

[0094] Step S552: Calculate the cosine similarity between the semantic description vector and the embedded representation of each standard anomaly feature.

[0095] The cosine similarity formula is the dot product of two vectors divided by the product of their moduli, with a range of [-1, 1]. For example, if the semantic description vector has a similarity of 0.92 with the embedding vector for "track gauge exceeding the limit," 0.65 with "missing bolts," and 0.78 with "roadbed settlement," then the highest similarity type is preliminarily determined to be track gauge exceeding the limit. Before similarity calculation, L2 normalization of the vectors is required to ensure comparability between different anomaly types.

[0096] Step S553: ​​Select the standard anomaly feature with the highest similarity as the candidate matching result.

[0097] Candidate matching results must meet a preset threshold (e.g., ≥0.85); otherwise, they are marked as "unknown anomalies." For example, if the highest similarity of 0.92 (track gauge exceeding the limit) exceeds the threshold, a candidate result is generated: "track gauge exceeding the limit, confidence level 92%." If multiple anomaly types have similarities close (e.g., 0.89 and 0.87), a multi-candidate mechanism is triggered, and the first three results are output for the verification module to process.

[0098] Step S554: Verify the candidate matching results based on the target anomaly type in the anomaly monitoring parameter set.

[0099] The verification process checks whether the candidate results are consistent with the preset targets of the monitoring task. For example, if the target type specified in the abnormal monitoring parameter set is "track gauge exceeding the limit," and the candidate result is also "track gauge exceeding the limit," then the verification passes directly. If the target type is "missing bolts," but the candidate result is "roadbed settlement," then the spatiotemporal feature extraction process needs to be traced back to check whether sensor noise caused a mismatch. If the verification fails, the system will initiate a manual review process and record the reason for the mismatch to optimize the model.

[0100] Step S555: If the verification is successful, the anomaly type and impact level corresponding to the candidate matching result are written into the railway anomaly monitoring result.

[0101] The anomaly type is written according to the standard name specification in the knowledge base (e.g., "GJ-002 type track gauge over-limit"), and the impact level is matched with the grading rules in the knowledge base (5-8 mm is level II) based on the quantitative indicators in the semantic description vector (e.g., deviation value of 7 mm). The output is structured JSON data, including anomaly code, location, time, level, and handling suggestions. For example: json { "Abnormal type": "GJ-002", Location: K325+500 to K325+520 Time: 2023-10-05 12:00-12:40 Impact Level: Level II Recommended action: Adjust the track gauge and check the condition of the fasteners within 48 hours. } As one implementation method, step S500, mapping the railway anomaly monitoring results to the physical coordinate system of the railway line, includes: Step S501: Obtain the original position data of the abnormal area in the UAV acquisition coordinate system.

[0102] The UAV's data acquisition coordinate system uses the takeoff point as the origin (0,0,0) and records the relative positions of abnormal areas through the body coordinate system (X-axis for forward movement, Y-axis for rightward movement, Z-axis vertically downward). For example, the coordinates of an abnormal area in the body coordinate system are (X=1200 meters, Y=-3.5 meters, Z=50 meters), indicating that the abnormality was detected when the UAV was flying 1200 meters eastward, offset 3.5 meters to the left, and at a flight altitude of 50 meters.

[0103] Step S502: Perform coordinate transformation based on the UAV's positioning parameters and the railway line's GIS data to generate the latitude and longitude coordinates of the abnormal area.

[0104] Positioning parameters include WGS-84 latitude and longitude coordinates received by the UAV's GNSS (e.g., E116.3528°, N39.9067°), attitude angles recorded by the IMU (pitch angle 1.2°, roll angle 0.5°), and RTK correction data. Coordinate transformation employs a perspective projection model, converting three-dimensional points (X, Y, Z) in the body coordinate system to latitude, longitude, and elevation in the geographic coordinate system. For example, the coordinate transformation formula calculates the latitude and longitude of the center point of this anomalous area as E116.3535°, N39.9072°, with an elevation of 52 meters.

[0105] Step S503: Associate the latitude and longitude coordinates with the mileage markers of the railway track to generate mileage markers for abnormal locations.

[0106] Mileage marker association is achieved through a spatial indexing algorithm. Specifically, the track centerline closest to the anomaly point is located in the railway GIS data, and the curve distance (mileage) along the centerline is calculated. For example, if the anomaly point's latitude and longitude correspond to the track centerline K325+480, with a lateral offset of 2.3 meters to the left, then the mileage marker is "K325+480 left 2.3 meters". For curved sections, the projection correction of track superelevation and plane coordinates must be considered to ensure that the mileage calculation is accurate to the centimeter level.

[0107] Step S504: Generate an anomaly location report containing visual markers based on the mileage identifier and anomaly type.

[0108] Visual markers can be in SVG or GeoJSON format, highlighting the boundaries of abnormal areas on the digital railway map (e.g., a red polygon covering K325+480 to K325+520), and overlaying anomaly type icons (e.g., a yellow exclamation mark indicating a gauge overrun). The report also includes a text description table listing the mileage, deviation value, detection time, and a thumbnail of a photograph (e.g., a screenshot of the LiDAR point cloud of the gauge measurement point) for each abnormal point.

[0109] Step S505: Send the abnormal location report to the railway operation and maintenance terminal and trigger a priority scheduling command to start the maintenance process.

[0110] After receiving the report, the maintenance terminal automatically analyzes the anomaly type and level, generates a maintenance work order, and assigns it to the nearest maintenance team. For example, a Level II gauge over-limit work order will be pushed to work area A, requiring two operators to arrive at the site at a specified time with a gauge ruler and hydraulic adjuster. Simultaneously, the system dispatches resources such as track vehicles and lighting equipment based on the anomaly's impact range (e.g., 200 meters of continuous gauge over-limit), and updates the railway timetable to implement speed limits for that section (e.g., reducing speed from 120 km / h to 80 km / h). After maintenance is completed, on-site personnel upload repair confirmation information via the terminal, the system closes the anomaly event, and records it in the knowledge base for model iteration and optimization.

[0111] Based on the above-described inventive concept, in another preferred embodiment, step S502, which involves performing coordinate transformation based on the UAV's positioning parameters and the GIS data of the railway line to generate the latitude and longitude coordinates of the abnormal area, may include: Step S5021: Obtain the set of positioning parameters of the UAV when collecting the railway inspection data. The set of positioning parameters includes the UAV's global positioning system coordinates, the attitude angle data of the inertial measurement unit, and the intrinsic parameter matrix of the camera.

[0112] The positioning parameter set is the fundamental data source for performing coordinate transformations. Global Positioning System (GPS) coordinates are acquired in real-time by the Global Navigation Satellite System (GNSS) receiver onboard the UAV, recording the UAV's latitude, longitude, and altitude in the WGS-84 coordinate system. For example, the coordinates at the time of acquisition (12:00:00) are 116.3528 degrees East longitude, 39.9067 degrees North latitude, and 52.3 meters altitude. The attitude angle data from the inertial measurement unit (INS) includes the UAV's roll, pitch, and yaw angles, such as a roll angle of 0.5 degrees (right roll), a pitch angle of 1.2 degrees (forward roll), and a yaw angle of 182.3 degrees (0 degrees for true north). The camera's intrinsic parameter matrix describes its optical center, focal length, and pixel distortion parameters, such as focal length fx=1200 pixels, fy=1180 pixels, principal point coordinates (cx=960, cy=540), and distortion coefficients k1=0.12, k2=-0.03, which are used to map image pixel coordinates to physical space.

[0113] Step S5022: Extract the original pixel coordinates of the abnormal region in the UAV acquisition coordinate system. The original pixel coordinates are determined based on the center point of the target detection box in the image frame sequence of the railway inspection data.

[0114] The drone's coordinate system uses the top-left corner of the image as the origin (0,0), with the positive X-axis pointing to the right and the positive Y-axis pointing downwards. The target detection bounding box uses the YOLOv5 algorithm to locate abnormal regions in the image frame. For example, if the top-left corner of the detection bounding box for a track gauge anomaly is (320, 480) and the bottom-right corner is (640, 720), then the original pixel coordinates of the center point are (480, 600). These coordinates need to be normalized to relative values ​​(0.25, 0.555) based on the image resolution (e.g., 1920×1080) for subsequent coordinate transformations.

[0115] Step S5023: Based on the intrinsic parameter matrix of the camera, convert the original pixel coordinates into three-dimensional spatial coordinates in the camera coordinate system of the UAV to generate initial spatial coordinates.

[0116] The camera coordinate system has its origin at the lens optical center, with the Z-axis along the optical axis, the X-axis to the right, and the Y-axis downwards. The transformation process is achieved through inverse perspective transformation: first, the lens distortion is eliminated using an intrinsic parameter matrix, and the corrected pixel coordinates (480, 600) are converted into normalized planar coordinates (x'=(480-cx) / fx= (480-960) / 1200=-0.4, y'=(600-cy) / fy=(600-540) / 1180≈0.051). Assuming the target point is at a depth Z=50 meters from the camera (obtained through stereo vision or LiDAR ranging), the three-dimensional coordinates in the camera coordinate system are (X=Z×x'=50×(-0.4)=-20 meters, Y=Z×y'=50×0.051≈2.55 meters, Z=50 meters).

[0117] Step S5024: Construct a rotation matrix based on the attitude angle data of the inertial measurement unit, and transform the initial spatial coordinates to the global coordinate system of the UAV through the rotation matrix to generate intermediate geographic coordinates.

[0118] The attitude angles of the inertial measurement unit (IMU) are used to construct the rotation matrix from the camera coordinate system to the UAV body coordinate system. For example, the rotation matrix R corresponding to a roll angle of 0.5 degrees, a pitch angle of 1.2 degrees, and a yaw angle of 182.3 degrees is obtained by multiplying three Euler angle rotation submatrices. Multiplying the coordinates (-20, 2.55, 50) in the camera coordinate system by R and transforming them to the UAV body coordinate system yields the intermediate geographic coordinates (X_body = 19.8 meters, Y_body = -1.2 meters, Z_body = 49.6 meters). These coordinates represent the position of the anomalous area relative to the UAV's center of mass.

[0119] Step S5025: Combine the GPS coordinates with the intermediate geographic coordinates to perform a translation transformation, generating the uncorrected latitude and longitude coordinates of the abnormal area in the general geographic coordinate system.

[0120] Translation transformation overlays the intermediate geographic coordinates of the UAV's body coordinate system onto the UAV's GPS coordinates. For example, the GNSS coordinates of the UAV at the time of data collection are (E116.3528°, N39.9067°, altitude 52.3 meters). The intermediate geographic coordinates (X_body=19.8 meters, Y_body=-1.2 meters) are transformed using the Northeast Elevation (NED) coordinate system to obtain the uncorrected latitude and longitude of the anomalous area: Moving 19.8 meters north corresponds to a latitude increment Δlat=19.8 / 111319≈0.000178 degrees (N39.9067+0.000178= N39.906878°), and moving -1.2 meters east corresponds to a longitude increment Δlon=-1.2 / (111319×cos(39.9067°))≈-1.2 / 85300≈-0.000014 degrees (E116.3528-0.000014= E116.352786°), the altitude is 52.3 + 49.6 = 101.9 meters.

[0121] Step S5026: Call the digital elevation model in the geographic information system data of the railway line, extract the elevation offset corresponding to the uncorrected latitude and longitude coordinates, and generate elevation correction parameters.

[0122] The Digital Elevation Model (DEM) stores the surface elevation data along the railway line in raster form with a resolution of 1 meter. A query for the uncorrected coordinates (E116.352786°, N39.906878°) yields a DEM elevation value of 98.5 meters (actual ground elevation), which deviates from the uncorrected elevation of 101.9 meters by 3.4 meters. The elevation correction parameter Δh = 98.5 - 101.9 = -3.4 meters is used to compensate for the GNSS elevation error and the Z-axis deviation of the UAV's coordinate system.

[0123] Step S5027: Perform vertical position compensation on the uncorrected latitude and longitude coordinates according to the elevation correction parameters to generate preliminarily corrected latitude and longitude coordinates.

[0124] The uncorrected elevation of 101.9 meters was adjusted to 98.5 + (-3.4) = 95.1 meters (the actual elevation should be 98.5 meters according to the DEM, which needs to be recalculated here: the corrected elevation should be 98.5 meters, Δh = 98.5 - 101.9 = -3.4 meters, so the corrected elevation = 101.9 + (-3.4) = 98.5 meters). The horizontal coordinates have a negligible impact on the plane projection due to elevation error, so the initially corrected coordinates are (E116.352786°, N39.906878°, elevation 98.5 meters).

[0125] Step S5028: Load the vector centerline data of the railway track from the geographic information system data, and calculate the shortest projection distance between the preliminarily corrected latitude and longitude coordinates and the vector centerline.

[0126] The vector centerline consists of a series of continuous three-dimensional points, describing the precise geometry of the track centerline. Taking the initial calibration coordinates (E116.352786°, N39.906878°) as an example, we traverse all segments of the centerline and calculate the shortest distance from each point to each segment. Assuming the coordinates of the nearest segment endpoints are (E116.352790°, N39.906875°) and (E116.352800°, N39.906880°), the shortest distance is calculated to be 2.3 meters using the vector projection formula, and this point is located to the right of the centerline (based on the train's direction of travel).

[0127] Step S5029: Based on the shortest projection distance, perform lateral offset correction on the preliminarily corrected latitude and longitude coordinates, adjust the preliminarily corrected latitude and longitude coordinates along the normal direction of the vector center line to a preset orbital proximity threshold range, and generate optimized latitude and longitude coordinates.

[0128] The track proximity threshold is set to 0.5 meters, meaning the abnormal area must be within 0.5 meters to either side of the track centerline. Since the initial correction coordinates deviated from the centerline by 2.3 meters, they need to be moved inwards by 2.3 - 0.5 = 1.8 meters along the centerline normal direction. The normal direction is calculated using the centerline tangent vector; for example, if the tangent vector is northeast, then the normal direction is northwest. The adjusted optimized coordinates are (E116.352786 - 0.000002°, N39.906878 + 0.000020°), corresponding to 0.5 meters to the right of the centerline.

[0129] Step S50210: According to the coordinate system transformation rules in the geographic information system data of the railway line, the optimized latitude and longitude coordinates are transformed from the local engineering coordinate system to the global geodetic coordinate system to generate standardized latitude and longitude coordinates.

[0130] The local engineering coordinate system may use the Gauss-Kruger projection (e.g., EPSG:4547), which needs to be transformed to the WGS-84 coordinate system (EPSG:4326) using a seven-parameter transformation model (translation, rotation, scaling). For example, the optimized coordinates (E116.352786°, N39.906878°) are (500325.12, 4421786.34) in the local coordinate system, and after applying the transformation parameters, we get the WGS-84 coordinates (E116.352791°, N39.906882°).

[0131] Step S50211: Verify the topological consistency between the standardized latitude and longitude coordinates and the vector centerline. If there are anomalies with coordinate jumps or deviations from the track, perform interpolation smoothing based on the coordinates of adjacent spatiotemporal frame sequences. The topological consistency check ensures that the coordinates change continuously along the track direction. For example, if the coordinates of an anomaly region in five consecutive frames are K325+500, K325+505, K325+510, K325+515, and K325+520, and the coordinates in the third frame abnormally jump to K325+600, it is identified as a coordinate jump. In this case, linear interpolation is used to calculate the correction value K325+510 for the third frame based on the coordinates of the second frame (K325+505) and the fourth frame (K325+515).

[0132] Step S50212: Bind the verified and interpolated latitude and longitude coordinates to the collection timestamp of the railway inspection data to generate a set of latitude and longitude coordinates of the abnormal area with spatiotemporal attributes. The spatiotemporal attributes include the collection time (e.g., 2023-10-05 12:00:00.000), the coordinate validity period (e.g., 12:00:00-12:00:10), and the associated sensor data index. For example, the abnormal area coordinate set is recorded as follows: json { "Coordinates": [ {"Longitude": 116.352791, "Latitude": 39.906882, "Time": "2023-10-05T12:00:00Z"}, {"Longitude": 116.352865, "Latitude": 39.906910, "Time": "2023-10-05T12:00:05Z"} ] } This data set will serve as the core data input for the anomaly location report into the subsequent operation and maintenance system, driving the scheduling of maintenance resources and the analysis of historical anomalies.

[0133] Based on the above inventive concept, in another preferred embodiment, step S504, generating an anomaly location report containing visual markers based on the mileage identifier and anomaly type, includes: Step S5041: Calculate the latitude and longitude coordinates of the abnormal area, the mileage marker, and the corresponding abnormal type.

[0134] Latitude and longitude coordinates are output as a standardized geographic coordinate set through the coordinate transformation module. For example, the coordinate set of a certain track gauge over-limit anomaly contains three points: (E116.352791°, N39.906882°), (E116.352810°, N39.906890°), and (E116.352825°, N39.906900°), with mileage markers of K325+500 (left 1.2 meters), K325+505 (left 1.5 meters), and K325+510 (left 1.8 meters), respectively. The anomaly type is coded as "GJ-002 type track gauge over-limit". The statistical process aggregates the spatial distribution of all detection points in the same anomaly event. For example, the geometric center point of the coordinate set (E116.352808°, N39.906890°) is calculated as the main location of the anomaly, and the maximum deviation value (7 mm) and average duration (25 minutes) are recorded.

[0135] Step S5042: Based on the track topology in the GIS data of the railway line, determine the section number and sleeper number of the mileage marker in the railway line.

[0136] The track topology divides the railway line into several management sections. For example, K320+000 to K330+000 is "5th Maintenance Section - Section 3". Within each section, sleepers are numbered consecutively at 0.6-meter intervals. Taking the mileage marker K325+500 as an example, its section number is "5-3-12" (representing the 12th sub-segment of Section 3, 5th Maintenance Section). The nearest sleeper number, "Sleeper #325500-Left 2", is found through spatial index matching. Here, "325500" indicates that the sleeper is installed at K325+500, and "Left 2" indicates that it is the second sleeper from the left of the track centerline.

[0137] Step S5043: Match the corresponding visual marker symbol from the predefined marker symbol library based on the anomaly type; wherein the visual marker symbol includes shape, color and texture attributes.

[0138] The symbol library is a structured database. For example, "track gauge overrun" corresponds to a red circular icon (RGB 255,0,0), 10 pixels in diameter, with jagged edges to indicate urgency; "missing bolts" corresponds to a yellow triangle icon (RGB 255,255,0) with an exclamation mark; and "subgrade settlement" uses a gradient diamond icon (from blue to red, indicating settlement depth). The matching process indexes the symbol library using anomaly type codes (such as GJ-002) to obtain the corresponding SVG graphic files and rendering parameters. For example, the track gauge overrun icon needs to flash twice per second in the report.

[0139] Step S5044: Bind the section number and sleeper number to the visual marker symbol to generate a marker symbol instance with spatial location attributes.

[0140] Attribute binding is achieved through XML tags. For example, the section number "5-3-12" and the sleeper number "sleeper #325500-left 2" can be written into the attribute field of the tag symbol, generating the following example: xml <symbol id="GJ-002-325500" type="circle" color="#FF0000" size="10"> <attribute name="区段编号"> 5-3-12< / attribute> <attribute name="轨枕编号"> Sleeper #325500-Left 2< / attribute> <position lon="116.352791" lat="39.906882" / > < / symbol> This instance can be parsed by the GIS engine as a visual element with interactive attributes, and a detailed information window will pop up when clicked.

[0141] Step S5045: Extract elevation distribution data and surrounding feature outline data from the GIS data of the railway line to generate a three-dimensional geographic background layer corresponding to the section number.

[0142] Elevation distribution data comes from a digital elevation model (DEM) with a resolution of 1 meter. For example, the elevation range corresponding to section "5-3-12" is from 95.2 meters to 103.5 meters above sea level. Surrounding feature contour data includes vector boundaries of bridges, tunnels, and signal towers. For instance, at K325+500, there is a 200-meter-long steel truss bridge; its 3D model is constructed from point cloud data, including geometric details of the piers, sleepers, and guardrails. The 3D geographic background layer is rendered using the Cesium engine, overlaying satellite imagery and terrain undulations to generate a scene with realistic lighting effects.

[0143] Step S5046: Overlay the marker instance onto the corresponding latitude and longitude coordinates of the three-dimensional geographic background layer to generate an anomaly location annotation map containing spatial context information.

[0144] The overlay process uses a georeferencing algorithm to map the planar coordinates (E116.352791°, N39.906882°) of the marker symbol to its corresponding position in the 3D scene. For example, the track gauge overrun symbol is suspended 2 meters above the steel truss bridge and aligned with the solid model of the bridge deck sleepers. Spatial context information includes the distance between the anomaly and surrounding features (e.g., 15.3 meters from the nearest signal tower) and the elevation difference (e.g., 0.5 meters above the bridge deck). This information is displayed next to the marker symbol in the form of a semi-transparent label.

[0145] Step S5047: Extract a standard anomaly description template from the railway equipment knowledge base according to the anomaly type, fill the placeholders of the standard anomaly description template with the mileage identifier, the section number, and the sleeper number, and generate anomaly description text.

[0146] The standard anomaly description template is a structured text frame. For example, the template content for track gauge exceeding the limit is: [Exception Type]: {type} [Position]: {mileage}, {section}, {sleeper} [Description]: The track gauge value detected is {value} millimeters, which exceeds the threshold {threshold} millimeters, and lasts for {duration} minutes.

[0147] The specific text is generated after filling in: [Abnormality Type]: GJ-002 type track gauge exceeds limit [Location]: K325+500, 1.2 meters to the left, section 5-3-12, sleeper #325500-2 meters to the left [Description]: The track gauge value was detected to be 1438 mm, exceeding the threshold of 1435 mm, and lasted for 25 minutes.

[0148] Step S5048: Merge the anomaly location annotation image and the anomaly description text, and generate an initial version of the anomaly location report according to preset layout rules. The layout rules stipulate that images are left-aligned, text is right-aligned, the title font is bold 16pt, and the anomaly description items use a numbered list, for example: 1. Anomaly Location Annotation Map (occupies 60% of the page width): Displays markers and surrounding features in the 3D scene; 2. Exception description text (occupies 40% of the page width): Lists the type, location, parameters, and historical comparison data for each item; 3. Legend and map scale: Placed at the bottom of the page, explaining the meaning of symbols and map scale (1:500).

[0149] Step S5049: Retrieve maintenance records associated with the section number from the historical anomaly location report, and extract priority tags and maintenance timeliness indicators from the maintenance records.

[0150] For example, searching the records of section "5-3-12" over the past year revealed three track gauge over-limit maintenance records, the most recent of which was on September 10, 2023, with a maintenance timeliness index of "Level II response, repair within 48 hours". Priority tags are generated based on the urgency of the maintenance, such as "high priority" (requires processing within 24 hours) or "normal priority" (requires processing within 72 hours).

[0151] Step S50410: Adjust the flashing frequency and alarm level identifier of the visual marker in the anomaly location report according to the priority label and maintenance timeliness index.

[0152] If the priority label is "High Priority," the flashing frequency of the track gauge over-limit symbol will be increased from 2 times / second to 5 times / second, and a red warning light icon will be superimposed on the top of the symbol; the maintenance timeliness indicator "48 hours" will be converted into a countdown label, displayed as "Remaining Repair Time: 47:30:00." Alarm level indicators are classified according to railway safety regulations, such as "Level I" (red background), "Level II" (orange background), and "Level III" (yellow background).

[0153] Step S50411: Update the initial version of the anomaly location report based on the adjusted visual marker symbols and alarm level identifiers to generate the final visual marker anomaly location report.

[0154] The updated report displays a high-frequency flashing red circular symbol in the 3D scene, adds a "High Priority" label and a countdown module to the right text area, and attaches historical repair records in table form below the description text, listing the personnel who handled the repairs and the results.

[0155] Step S50412: Adapt the format of the visual marker anomaly location report to the display protocol of the railway operation and maintenance terminal, and convert it into a vector graphic report that supports interactive operation.

[0156] The railway maintenance terminal uses the WebGL protocol to render vector graphics, and the report format is converted to a hybrid GeoJSON and HTML5 format. It allows users to click on marker symbols to view real-time sensor data (such as the current track gauge value), drag the viewpoint to view the 3D scene, and zoom the map to millimeter-level precision. For example, clicking on the marker symbol for sleeper #325500-left 2 will display a pop-up window showing the sleeper's installation date, material properties, and data curves from the three most recent inspections.

[0157] Step S50413: Embed keyframe screenshots of the spatiotemporal frame sequence of the abnormal region in the vector graphics report to generate a visually marked abnormality location report containing multi-dimensional evidence.

[0158] Keyframe screenshots are selected from the frames with the most significant anomaly features in the spatiotemporal frame sequence. For example, in the image frame at 12:05:30, the track gauge exceeds the limit, showing a significant outward expansion of the track head. The screenshot is annotated with measurement lines (marking the actual track gauge of 1438 mm) and comparison lines (marking the standard track gauge of 1435 mm). The screenshots are embedded as thumbnails below the report text area. Clicking on them expands them into a high-resolution image and links them to the original data files (such as LiDAR point clouds and infrared thermal images). The final report integrates spatial annotations, text descriptions, historical records, real-time data, and multimodal evidence to form a complete anomaly localization decision support document.

[0159] Based on the above-described inventive concept, in another preferred embodiment, step S505, which involves sending the anomaly location report to the railway operation and maintenance terminal and triggering a priority scheduling instruction to initiate the maintenance process, includes: Step S5051: Obtain the latitude and longitude coordinates of the abnormal area and the associated mileage markers in the abnormal location report, and extract the abnormal type and impact level.

[0160] Anomaly location reports are stored in a structured data format. Their content fields include a set of latitude and longitude coordinates (e.g., [E116.352791°, N39.906882°]), mileage markers (e.g., K325+500 left 1.2 meters), anomaly type codes (e.g., GJ-002 type gauge exceedance), and impact levels (e.g., Level II). The data extraction module parses the XML or JSON tags in the report to locate fields such as "coordinates," "mileage_marker," "anomaly_type," and "severity_level." For example, key parameters can be extracted from the following JSON fragment: json { "anomaly_id": "A20231005-001", "coordinates": [{"lon": 116.352791, "lat": 39.906882}], "mileage": "K325+500 left 1.2 meters", "type": "GJ-002", "severity": Level II } After extraction, the anomaly type "GJ-002" is mapped to the standard name "gauge overrun" in the railway equipment knowledge base, and the impact level "Level II" corresponds to a maintenance response time of 48 hours.

[0161] Step S5052: Match the emergency response template in the railway operation and maintenance rule base according to the anomaly type, and determine the maintenance resource type and standard processing time limit corresponding to the anomaly type.

[0162] The railway operation and maintenance rule base is an associated database containing anomaly types and handling rules. For example, the emergency response template associated with "GJ-002 type track gauge over-limit" is defined as follows: { Resource Type: ["Track Gauge", "Hydraulic Adjuster", "Rail Vehicle"], Processing timeframe: 48 hours Staffing: ["2 line workers", "1 technician"], Safety Measures: ["Speed ​​limit 80km / h", "Protective signals"] } The matching process is implemented through SQL queries or graph database traversal. For example, the rule base is retrieved using the exception type code as an index to return the above resource list and time limit requirements. If the exception type is "missing bolts" (code LS-005), the resource type is changed to "torque wrench" or "spare bolt set", and the processing time is shortened to 24 hours.

[0163] Step S5053: Based on the impact level and the railway section topology relationship where the mileage marker is located, calculate the maintenance emergency index of the abnormal area; wherein, the railway section topology relationship includes the train operation density, track load and historical maintenance records of adjacent sections.

[0164] The operational urgency index quantifies the priority of anomaly handling using a weighted formula, which is: Emergency Index = Impact Level Weight × Train Density Coefficient + Track Load Coefficient + Historical Maintenance Coefficient.

[0165] For example, the impact level weighting is as follows: Level II corresponds to a weight of 0.7 (Level I is 1.0, and Level III is 0.5). Train density coefficient: Based on the real-time operation diagram of the line where section K325+500 is located, the number of trains passing through in the next 12 hours is calculated (e.g., 28 trains). The density coefficient = number of trains / maximum throughput capacity (30 trains) = 0.93. Track load coefficient: Based on axle load monitoring data, the current section load rate is calculated (e.g., 85%). The coefficient = load rate / 100 = 0.85. Historical maintenance coefficient: The number of similar anomalies in this section over the past year is retrieved (e.g., 3 times). The coefficient = 1 - (1 / (number of anomalies + 1)) = 0.75. Substituting these values, the emergency index is calculated as 0.7 × 0.93 + 0.85 + 0.75 = 2.25 (range 0-3, higher values ​​indicate higher priority).

[0166] Step S5054: Sort multiple abnormal areas according to the maintenance urgency index and generate maintenance task queues classified by priority level.

[0167] If three abnormal regions exist simultaneously: 1. Area A (gauge exceeds limit, emergency index 2.25) 2. Area B (missing bolts, urgency index 2.80) 3. Area C (subgrade settlement, emergency index 1.90) The queue is generated by sorting the elements in descending order of their exponents: [Region B → Region A → Region C]. The queue data structure is a priority heap. Each time, the top element of the heap (highest priority) is extracted to generate a work order. For example, the work order ID for Region B is "WO-20231005-001", which includes a resource list and time limit requirements.

[0168] Step S5055: Dynamically adapt the maintenance task queue to the maintenance resource type to generate a resource allocation scheme and maintenance time window for each abnormal area.

[0169] Dynamic adaptation is achieved through resource scheduling algorithms, such as matching idle resources with task requirements based on the Hungarian algorithm: For example, in Zone B (bolt shortage): torque wrenches are needed (3 in stock, 2 usable) and spare bolts (200 sets in stock). "Wrench #002" and "Bolt #50-100" are assigned, with a time window of 14:00-16:00 on October 5th (avoiding peak train traffic). In Zone A (gauge overrun): the track vehicle needs to be dispatched from Warehouse D (15 km from K325+500). Based on traffic conditions, the arrival time is calculated to be 15:30 on October 5th, with a maintenance window of 16:00-18:00. In case of resource conflicts (e.g., multiple tasks competing for the same equipment), a preemptive dispatch system is used, prioritizing tasks with high urgency and delaying or splitting time windows for lower priority tasks.

[0170] Step S5056: Generate a scheduling instruction message according to the resource allocation scheme, bind the scheduling instruction message with the abnormal location report, and transmit it to the railway operation and maintenance terminal through railway communication protocol encryption.

[0171] The dispatch instruction message uses the ASN.1 encoding format and includes fields such as work order ID, resource list, time window, and safety measures. For example: WorkOrder ::= SEQUENCE { id INTEGER "WO-20231005-001", resources SEQUENCE OF UTF8String {"torque wrench #002", "bolt #50-100"}, timeWindow SEQUENCE { startTime GeneralizedTime "20231005140000", endTime GeneralizedTime "20231005160000" }, Safety Measures SEQUENCE OF UTF8String {"Speed ​​Limit 80km / h", "Protection Signal"} } The messages are encrypted using railway-specific communication protocols (such as RFC 7864), and the AES-256 algorithm and pre-shared key are used to ensure that the transmission process is tamper-proof and eavesdropping-proof.

[0172] Step S5057: Receive the maintenance resource confirmation signal and maintenance team location data returned by the railway maintenance terminal, and update the task status in the maintenance task queue in real time.

[0173] For example, the confirmation signal returned by the maintenance team's terminal is in JSON format, such as: json { "workorder_id": "WO-20231005-001", "status": "confirmed", "team_location": {"lon": 116.350000, "lat": 39.900000}, "eta": "20231005135000" } The task queue state machine updates accordingly: the status of region B changes from "pending allocation" to "confirmed", the estimated arrival time (ETA) is 15:50, and subsequent path planning is triggered. If the terminal returns a "resource insufficiency" signal, step S5055 is re-executed for dynamic adaptation.

[0174] Step S5058: Based on the location data of the maintenance team and the latitude and longitude coordinates of the abnormal area, perform path planning to generate the optimal inspection route and maintenance personnel scheduling list.

[0175] The route planning uses Dijkstra's algorithm, with input parameters including real-time traffic conditions (such as road closures due to construction), vehicle speed (track vehicle speed limit 40km / h), and safety rules (such as nighttime restrictions). For example, the route from the team's current location (E116.350000°, N39.900000°) to K325+500 is: travel 8 km along the maintenance access road → turn onto the track siding → move to the target point, with a total time of 25 minutes. The scheduling list is detailed down to the personnel assignments; for example, technicians are responsible for operating torque wrenches, and track workers are responsible for bolt replacement and tightening checks.

[0176] Step S5059: Push the optimal inspection route and maintenance personnel scheduling list to the maintenance team terminal, and simultaneously start the maintenance process timer to monitor the execution progress of the standard processing time limit.

[0177] The pushed data includes a path file in GPX format and a schedule list in PDF format. The terminal map application automatically loads the navigation route. The timer starts at the work order confirmation time (15:00), with a standard processing time limit of 24 hours. The remaining time is displayed in real time as "23:59:59" and decreases every second. If the maintenance team uploads a "bolt replacement completed" signal at 18:00, the timer calculates 3 hours of elapsed time, leaving 45 hours remaining.

[0178] Step S50510: When the maintenance process timer detects a maintenance completion signal or a timeout alarm event, it triggers the log update command of the railway operation and maintenance terminal to record the maintenance result and close the work order status of the abnormal location report.

[0179] The maintenance completion signal triggers the following operations: 1. Log recording: Write the maintenance results (such as "track gauge adjusted to 1435mm, 5 sets of bolts replaced") into the database and associate them with the work order ID and the exception ID.

[0180] 2. Status Closed: The work order status changes from "In Progress" to "Closed", the speed limit measures are lifted and the normal train operation schedule is restored.

[0181] 3. Knowledge base update: Add the data from this maintenance to the historical case database to optimize the resource prediction accuracy of subsequent emergency response templates.

[0182] If the timer expires (e.g., not completed within 48 hours), an alarm event is triggered, which is escalated to the superior dispatch center and the backup resource allocation process is initiated. At the same time, a violation record is generated for accountability.

[0183] In summary, this invention achieves a breakthrough improvement in the accuracy and intelligent decision-making of railway inspection anomaly detection through multi-dimensional spatiotemporal data fusion and dynamic parameter adaptation mechanisms. A multi-dimensional data coupling analysis framework is constructed based on spatiotemporal frame sequences continuously collected by UAVs. By extracting spatial structural features and temporal evolution features in parallel, it overcomes the limitations of traditional single-dimensional detection methods in representing complex scenarios. An innovative dynamic matching relationship model is established, nonlinearly mapping monitoring command parameters with spatiotemporal feature vectors, enabling anomaly area screening to possess parameter-adaptive decision-making characteristics, effectively solving the problem of insufficient adaptability of fixed threshold detection to multiple types of anomalies. Through a spatiotemporal joint analysis model, collaborative reasoning of cross-modal features is realized, capturing the anomaly evolution patterns in the temporal dimension while preserving spatial topological relationships, significantly improving the early identification capability of progressive hidden dangers such as crack propagation and component loosening. Finally, combined with the inverse mapping mechanism of the physical coordinate system, a bidirectional interpretable link from data features to entity location is constructed, providing technical support for railway infrastructure maintenance that combines spatial positioning accuracy and anomaly evolution trend analysis.

[0184] The embodiments of the present invention form an intelligent diagnostic closed loop with environmental perception capabilities through hierarchical feature interaction and dynamic matching mechanisms, which greatly improves the reliability and predictability of railway anomaly detection in complex operating environments.

[0185] This invention also provides a railway anomaly monitoring system based on unmanned aerial vehicle (UAV) inspection, such as... Figure 2 As shown, the railway anomaly monitoring system 100 based on UAV inspection includes a processor 101 and a memory 103. The processor 101 and the memory 103 are connected, for example, via a bus 102. Optionally, the railway anomaly monitoring system 100 based on UAV inspection may also include a transceiver 104. It should be noted that in practical applications, the transceiver 104 is not limited to one type, and the structure of this railway anomaly monitoring system 100 based on UAV inspection does not constitute a limitation on the embodiments of the present invention.

[0186] Processor 101 may be a CPU, a general-purpose processor, a GPU, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 101 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0187] Bus 102 may include a pathway for transmitting information between the aforementioned components. Bus 102 may be a PCI bus or an EISA bus, etc. Bus 102 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0188] The memory 103 may be a ROM or other type of static storage device capable of storing static information and instructions, RAM or other type of dynamic storage device capable of storing information and instructions, or it may be an EEPROM, CD-ROM or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0189] The memory 103 is used to store application code that executes the present invention, and its execution is controlled by the processor 101. The processor 101 is used to execute the application code stored in the memory 103 to implement the content shown in any of the foregoing method embodiments.

[0190] This invention provides a railway anomaly monitoring system based on unmanned aerial vehicle (UAV) inspection. The railway anomaly monitoring system based on UAV inspection in this invention includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors. When the one or more programs are executed by the processors, they implement the method provided in this invention.

[0191] This invention provides a computer-readable storage medium storing a computer program that, when run on a processor, enables the processor to execute the corresponding content described in the aforementioned method embodiments.

[0192] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0193] The above description is only a partial embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A railway anomaly monitoring method based on unmanned aerial vehicle (UAV) inspection, characterized in that, include: The system acquires railway inspection data collected by drones and extracts monitoring instructions from the railway inspection data. The railway inspection data includes multiple spatiotemporal frame sequences collected continuously, and each spatiotemporal frame sequence contains data in spatial and temporal dimensions. The monitoring command is parsed to obtain the set of abnormal monitoring parameters corresponding to the monitoring command; Spatiotemporal feature extraction processing is performed on the multiple spatiotemporal frame sequences to generate spatial feature vectors and temporal feature vectors for each spatiotemporal frame sequence; Based on the dynamic matching relationship between the anomaly monitoring parameter set and the spatial feature vector and the temporal feature vector, at least one anomaly region is selected from the multiple spatiotemporal frame sequences; The anomaly identification model is invoked to perform joint analysis of the spatiotemporal characteristics of the anomaly area, generate railway anomaly monitoring results, and map the railway anomaly monitoring results to the physical coordinate system of the railway line to generate an anomaly location report.

2. The method as described in claim 1, characterized in that, The step of parsing the monitoring command to obtain the set of abnormal monitoring parameters corresponding to the monitoring command includes: The monitoring command is subjected to multimodal data preprocessing to generate preprocessed command text data; wherein, the multimodal data preprocessing includes noise filtering, time sequence alignment and format standardization; The parameter encoder is invoked to perform hierarchical feature extraction on the preprocessed instruction text data, generating global and local parameter features of the monitoring instruction; The global parameter features are associated and mapped with a predefined railway anomaly type library to determine the target anomaly type of the monitoring instruction; Based on the local parameter features and the spatiotemporal constraints of the target anomaly type, the anomaly monitoring parameter set is generated. The anomaly monitoring parameter set includes a spatially sensitive area threshold, a temporal continuity weight, and an anomaly intensity determination threshold.

3. The method as described in claim 2, characterized in that, The call parameter encoder performs hierarchical feature extraction on the preprocessed instruction text data to generate global and local parameter features of the monitoring instruction, including: The instruction text data is semantically segmented to generate multiple semantic units and contextual dependencies between them. Perform time encoding on each semantic unit to generate a timestamp-associated vector for each semantic unit; Each semantic unit is transformed and encoded to generate a domain adaptation vector for each semantic unit; The global parameter features are generated by fusing the timestamp association vector and the domain adaptation vector. Attention weights are assigned to the multiple semantic units based on the context dependencies to generate the local parameter features.

4. The method as described in claim 1, characterized in that, The step of performing spatiotemporal feature extraction processing on the multiple spatiotemporal frame sequences to generate spatial feature vectors and temporal feature vectors for each spatiotemporal frame sequence includes: Each spatiotemporal frame sequence is divided into spatial grid cells and temporal window segments; The spatial grid cells are spatially encoded to generate a spatial feature matrix for each spatial grid cell. The time window segments are time-encoded to generate a time feature sequence for each time window segment; The spatiotemporal encoder is invoked to perform cross-dimensional fusion of the spatial feature matrix and the temporal feature sequence to generate a joint spatiotemporal feature vector of the spatiotemporal frame sequence; wherein, the spatiotemporal encoder includes a three-dimensional convolutional kernel and a bidirectional recurrent network layer, which are used to simultaneously capture spatial structure changes and temporal evolution patterns.

5. The method as described in claim 4, characterized in that, The step of invoking the spatiotemporal encoder to perform cross-dimensional fusion of the spatial feature matrix and the temporal feature sequence to generate a joint spatiotemporal feature vector of the spatiotemporal frame sequence includes: The spatial feature matrix is ​​input into the three-dimensional convolution kernel to perform multi-scale spatial feature extraction, generating spatial pyramid features; The time feature sequence is input into the bidirectional recurrent network layer for time-series dependency modeling to generate time context features; The spatial pyramid features and the temporal context features are concatenated to generate an initial fused feature; The initial fused features are dynamically weighted by a gating attention mechanism to generate the joint spatiotemporal feature vector; wherein the weights of the gating attention mechanism are jointly controlled by the spatially sensitive region threshold and the temporal continuity weight in the anomaly monitoring parameter set.

6. The method as described in claim 1, characterized in that, The step of selecting at least one abnormal region from the multiple spatiotemporal frame sequences based on the dynamic matching relationship between the anomaly monitoring parameter set and the spatial feature vector and the temporal feature vector includes: Calculate the regional matching degree between the spatial feature vector and the spatially sensitive region threshold in the anomaly monitoring parameter set; Calculate the time matching degree between the time feature vector and the time continuity weights in the anomaly monitoring parameter set; Based on the weighted sum of the regional matching degree and the temporal matching degree, a comprehensive anomaly score is generated for each spatiotemporal frame sequence. The comprehensive anomaly score is compared with the anomaly intensity determination threshold, and spatiotemporal frame sequences with scores exceeding the threshold are selected as candidate anomaly regions. Cluster analysis is performed based on the spatiotemporal continuity of the candidate anomalous regions, and spatiotemporal frame sequences of adjacent regions are merged to generate the at least one anomalous region.

7. The method as described in claim 6, characterized in that, The clustering analysis based on the spatiotemporal continuity of the candidate anomaly regions includes: Extract the spatial coordinates and timestamp information of the candidate anomaly regions to construct a spatiotemporal distance matrix; Calculate the region similarity based on the spatial Euclidean distance and time interval distance in the spatiotemporal distance matrix; The regions are grouped based on their similarity using a density clustering algorithm to generate initial clusters. Boundary optimization is performed on each initial cluster to remove discrete noise points and expand the continuous coverage area; The optimized clusters are mapped to the topology of the railway line to generate anomaly region boundaries with physical properties.

8. The method as described in claim 1, characterized in that, The method of calling the anomaly identification model to jointly analyze the spatiotemporal characteristics of the anomaly region and generate railway anomaly monitoring results includes: The joint spatiotemporal feature vector of the abnormal region is input into a pre-trained spatiotemporal graph neural network; The spatial features in the joint spatiotemporal feature vector are iteratively optimized by the node update layer of the spatiotemporal graph neural network to obtain spatially optimized features; The time transition features in the joint spatiotemporal feature vector are modeled by the time propagation layer of the spatiotemporal graph neural network to obtain the time transition features. By fusing the spatial optimization features and the temporal transition features, a semantic description vector for the abnormal region is generated. The semantic description vector is matched with the railway equipment knowledge base to determine the anomaly type and impact level; The step of matching the semantic description vector with the railway equipment knowledge base to determine the anomaly type and impact level includes: The embedded representation of standard anomaly features is extracted from the railway equipment knowledge base; Calculate the cosine similarity between the semantic description vector and the embedded representation of each standard anomaly feature; Select the standard anomaly features with the highest similarity as candidate matching results; The candidate matching results are verified based on the target anomaly type in the anomaly monitoring parameter set; If the verification is successful, the anomaly type and impact level corresponding to the candidate matching result will be written into the railway anomaly monitoring result.

9. The method as described in claim 1, characterized in that, The process of mapping the railway anomaly monitoring results to the physical coordinate system of the railway line includes: Obtain the original position data of the abnormal area in the UAV acquisition coordinate system; Based on the drone's positioning parameters and the railway line's GIS data, coordinate transformation is performed to generate the latitude and longitude coordinates of the abnormal area; The latitude and longitude coordinates are associated with the mileage markers of the railway tracks to generate mileage markers for abnormal locations; An anomaly location report containing visual markers is generated based on the mileage identifier and anomaly type; The abnormal location report is sent to the railway operation and maintenance terminal, and a priority scheduling instruction is triggered to start the maintenance process.

10. A railway anomaly monitoring system based on unmanned aerial vehicle (UAV) inspection, characterized in that, include: One or more processors; Memory; One or more computer programs; The one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and when the one or more computer programs are executed by the processors, they implement the method as described in any one of claims 1 to 9.

Images