Railway catenary operation and maintenance detection false alarm suppression method based on multi-modal large model

Abstract

The application discloses a railway catenary operation and maintenance detection false alarm suppression method based on a multi-modal large model, and belongs to the technical field of intelligent monitoring of railway traffic; visual and language large models are used to perform cross-modal alignment on synchronously collected visual images, three-dimensional point clouds and text information, to form comprehensive features by fusing knowledge graph environment constraints, to perform geometric, timing and multi-modal mutual verification through a decoder, and to calculate false alarm probability to suppress alarms. The method breaks through pixel limitations, expands to multi-dimensional semantic space, and filters out false alarms such as light and shadow stains; in combination with text priori and context, the method solves the problem of identifying similar and heterogeneous diseases; dynamic graph integrates expert experience, improves small sample long tail robustness; self-explaining output and online learning continuously optimize, and relieve alarm fatigue; hierarchical filtering and quantization accelerate to realize high-performance large-scale inspection. The method reconstructs decision logic from pattern recognition to semantic cognition, reduces operation and maintenance cost, and improves intelligent level.

Description

Technical Field

[0001] This invention belongs to the field of intelligent monitoring technology for railway traffic, specifically a method for suppressing false alarms in railway catenary operation and maintenance based on a multimodal large model. Background Technology

[0002] With the continuous expansion of the global railway catenary network and the significant increase in operational density, ensuring the safety and operational stability of the railway catenary system, including contact wires, catenary wires, insulators, positioning clamps, and surrounding power supply facilities, has become a core issue for maintaining the smooth operation of the national economic artery and urban public transportation systems.

[0003] Traditional overhead contact line maintenance has long relied heavily on periodic manual inspections. However, faced with the ever-expanding scale of the railway network assets and the increasingly compressed maintenance window, manual inspections have revealed significant limitations in terms of operational efficiency, the completeness of inspection coverage, and the subjective consistency of evaluation standards. Against this backdrop, automated inspection systems based on machine vision and sensor technologies have emerged and are gradually being widely deployed in the railway overhead contact line maintenance field. Existing intelligent inspection solutions generally employ optical imaging components mounted on high-precision inspection trains or overhead contact line maintenance vehicles. By acquiring high-resolution image or video stream data and combining it with mainstream deep learning models such as deep convolutional neural networks (CNNs), they automatically identify and classify various typical overhead contact line component defects and foreign object intrusions.

[0004] This data-driven technological paradigm has indeed demonstrated significant technical advantages in improving detection throughput and initial screening speed under specific controlled environments and standard operating conditions, effectively alleviating the intensity of basic maintenance work. Specifically, early technical solutions typically built dedicated feature extractors for specific types of overhead contact line defects, trained them on labeled datasets through supervised learning, and thus achieved digital representation of problems such as broken contact wire strands, damaged insulators, and loose critical connectors.

[0005] However, as the engineering applications of related technologies deepen and application scenarios place increasingly stringent demands on detection accuracy and system robustness, some inherent characteristics of the aforementioned technical solutions at the principle level have gradually revealed insurmountable limitations when dealing with complex and ever-changing actual operating environments. The most significant technical contradiction lies in the highly dynamic and heterogeneous nature of the overhead contact line operating environment. Complex light and shadow interplay within tunnels, drastic changes in outdoor lighting, adverse weather conditions such as rain, snow, and fog, as well as stray light reflected from the contact wire and metal component surfaces, all severely interfere with imaging quality.

[0006] Existing detection models are essentially pattern matching based on local pixel features, lacking a deep understanding of the global semantics of the scene and the ability to recognize the laws of physical evolution. This high dependence on low-level visual features makes the system prone to generating a large number of false alarms when dealing with interference that is highly similar to the characteristics of the defect, such as dirt on the surface of insulators, oil stains on contact wires, rust marks on the surface of metal components, water stains, or the superposition effect of hanging foreign objects and shadows.

[0007] Further analysis reveals that the root cause of this high false alarm rate lies in the singularity and isolation of information dimensions in the existing detection architecture. In actual operation, determining whether a detected target is a real defect requires not only local visual texture features, but also often the combination of historical maintenance records of the area, geospatial location attributes (such as pole number and anchor segment), multidimensional redundant information from sensors, and cross-modal contextual relationships.

[0008] Existing technologies often reduce detection tasks to simple binary or multi-class classification problems, ignoring the rich prior knowledge inherent in overhead contact network scenarios. High false alarm rates have become a bottleneck restricting the efficiency of intelligent operation and maintenance: on the one hand, massive amounts of false alarm data greatly increase the workload of manual review at the back end, even exceeding the limits of human capacity, leading to serious misallocation and waste of operation and maintenance resources; on the other hand, long-term false positive alarms can cause significant alarm fatigue among frontline operators, subconsciously reducing their trust in the system's warnings. Once a real serious safety hazard occurs, it is highly likely that human negligence will lead to irreversible safety accidents.

[0009] Furthermore, fault samples in overhead contact line scenarios typically exhibit small sample sizes and long-tailed distribution characteristics. Traditional deep learning models often demonstrate insufficient generalization ability when faced with rare abnormal operating conditions or novel equipment components, which logically exacerbates the uncertainty of detection results. Even with the introduction of deeper residual networks or attention mechanisms, without the support of semantic reasoning and cross-modal association, the model still struggles to distinguish between visually similar but differently attributed complex noise.

[0010] In summary, ensuring a high detection rate while introducing higher-dimensional semantic cognition and logical reasoning capabilities at the fundamental level—by integrating multimodal data such as images, text descriptions, spatial geography, and time series—to construct an intelligent mechanism capable of deeply analyzing complex correlations in overhead contact line scenarios and accurately suppressing false defect alarms, has become a key challenge and urgent technical problem for those skilled in the art. Existing technologies urgently need a new generation of solutions that evolve from simple feature matching to semantic decision-making, enabling a paradigm shift in overhead contact line operation and maintenance monitoring from data-driven to a dual-driven model of knowledge and data. Summary of the Invention

[0011] The purpose of this invention is to provide a method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model, so as to solve the problem of high false alarm rate in existing automated detection systems for railway catenary maintenance and detection, as mentioned in the background art, due to environmental interference, single information dimension, and lack of semantic reasoning ability.

[0012] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model includes the following steps: Step 1, suspected defect target extraction: During the operation of the inspection vehicle, the physical information of the contact network scene is acquired in real time through the onboard sensor array. The physical information includes high-dimensional visual feature images and spatial geometric cross-sectional point cloud data. The physical information is preprocessed with timestamp hard synchronization, and preliminary target detection is performed using a convolutional neural network to output structured suspected defect targets containing image blocks, local point cloud regions, and location information. Step 2, cross-modal feature space mapping: Image patches from suspected disease targets are input into the visual encoder of a pre-trained visual and language large model to capture the continuity and breakage features of disease texture details in the spatial dimension, and the feature channels are explicitly weighted to suppress stray light interference and generate high-dimensional feature vectors; at the same time, historical maintenance records and inspection procedure descriptions are input into the text encoder and converted into technical terminology text embedding vectors of the same dimension. Step 3: Constructing environmental constraint semantics: Parse the historical operation and maintenance database to construct a dynamic knowledge graph covering asset topology, disease evolution trends over the years, and geographical environmental attributes; During online reasoning, use the location information of suspected disease targets as query anchors to retrieve local subgraphs, and use graph convolutional neural networks to perform representation learning and attention pooling on the local subgraphs to generate semantic constraint vectors that encode prior information of the current detection location environment. Then, concatenate the semantic constraint vectors and high-dimensional feature vectors at the feature fusion layer along the channel dimension to output fused features. Step 4, Multi-dimensional Physical Constraint Decision: The fused features are input into the multimodal large model decoder for logical reasoning. By deploying a multi-dimensional physical constraint joint decision verification operator at the decoder's output layer, a geometric continuity verification branch reflecting whether the component surface has foreign object adhesion or a real geometric fracture is introduced, a temporal evolution logic verification branch reflecting the credibility of historical maintenance status, and a spatial shadow shading mutual verification branch reflecting environmental optical condition interference are introduced. The original logarithmic probability vector output by the decoder is dynamically scaled and corrected, and the final decision probability is calculated. When the decision probability of being judged as a false alarm exceeds the preset hard filtering threshold, the edge system automatically suspends the suspected defect target.

[0013] According to the above technical solution, the multi-dimensional physical constraint joint decision verification operator deployed in the decoder output layer in step four... Its mathematical analytical expression is defined as:

[0014]

[0015] In the formula, This represents the input multimodal feature matrix containing suspected diseased image patches and local point cloud subsets; This represents the contextual semantic constraint vector extracted and propagated from the knowledge graph by the environment constraint operator; This represents the raw, unconstrained output of the multimodal large model decoder. An unnormalized log-probability vector; Indicates the target decision category variable to be evaluated; This represents the traversal index of all determinate output categories in the system; This represents the total number of pre-defined decision categories that can be determined by the system. and These represent the first and second halves of the logarithmic probability vector, respectively. The and the first The original score components of each category; This represents a surface continuity gate function constructed based on differential geometric curvature, where the calculated Gaussian curvature... The absolute value is less than the preset lower threshold and the average curvature The function outputs 1 when the absolute value is less than the preset upper threshold, and 0 otherwise. The indicator function is defined if and only if the input category points to the system's default false alarm category. The output is 1 if the condition is met, otherwise the output is 0. This indicates that the preset false alarm category dynamically adjusts and amplifies the weighting coefficient. This represents the absolute value of the time difference between the current inspection time and the time of the most recent valid maintenance action retrieved from the knowledge graph; Indicates the effective protection period time constant; This represents a ray-based spatial shading indicator function, which uses the calculated solar azimuth angle. Solar altitude angle And the three-dimensional boundary box geometric parameter model of the structure extracted in real time from the B-BIM database. As input, if the shadow intersection point is found to fall within the visual bounding box through ray tracing, the output is 1; otherwise, the output is 0.

[0016] According to the above technical solution, the time stamp hard synchronization preprocessing in step one is as follows: using the second pulse signal output by the global navigation satellite system receiver as the global synchronization reference, the hardware synchronization trigger controller divides the second pulse to drive each sensor to perform synchronous data acquisition; each sensor latches its local timestamp and corresponding pulse count value at the instant it receives the trigger signal, forming a triplet time identifier composed of the second pulse count, the trigger pulse sequence number, and the sensor's local timestamp; in the post-processing stage, the correspondence between the calibration delay parameters of each sensor and the trigger pulse sequence number is used to perform offset compensation and interpolation alignment on the original timestamp, so as to control the residual time axis deviation of multi-source data measured at the same physical sampling position within the preset nanosecond tolerance.

[0017] According to the above technical solution, the large-scale vision and language model in step two includes a dynamic adapter fine-tuning structure. During the training phase, a difficult negative sample mining strategy is established to address high-similarity false alarm interference, and an adaptive loss function for difficult samples to spatial topological boundaries is adopted. Dynamic collaborative gradient backpropagation of all network parameters is performed, and the overall mathematical analytical expression of the loss function is defined as follows:

[0018]

[0019] In the formula: This represents the total number of positive sample pairs in the current training batch; This represents the index for looping through positive sample pairs within the current batch; This represents the traversal index of all text features within the current batch; Indicates the first The high-dimensional feature vector of the i-th image and its corresponding i-th image Cosine similarity score between each correct prior text embedding vector; Indicates the first The high-dimensional feature vector of the i-th image and the i-th Cosine similarity score between unpaired text embedding vectors; A preset temperature scalar coefficient representing the degree of smoothness in controlling the similarity distribution; This represents the adaptive weighting coefficient used to balance the global contrastive loss and the marginal loss term for hard negative samples; This refers to the set of difficult negative samples selected in the current batch using the difficult negative sample mining strategy, whose visual cosine similarity to the current anchor sample is greater than a set hard threshold. Represents the set of difficult negative samples The total number of negative samples within; Represents the set of difficult negative samples The current negative sample index variable in the text; This represents the central spatial feature vector output by the visual encoder for the anchor point sample; This represents the projection vector of the interfering negative sample in the feature space; This represents the cosine similarity function used to calculate the similarity between two vectors. This represents the preset topological geometric boundary distance threshold parameter, used to forcibly generate inter-class isolation zones on the hypersphere.

[0020] According to the above technical solution, in step three, the knowledge graph extracts key fields of time, space, component type, status and maintenance action from historical data text through a named entity recognition algorithm based on conditional random fields, assembles them into a five-tuple structured record, and extracts and stores the state transition sequence of multiple records of the same entity by sorting them according to the time axis. During online inference, based on the graph convolutional neural network update formula, the features of neighboring nodes are normalized and aggregated by degree and transformed by learnable matrices to realize the multi-hop information flow propagation of the spatiotemporal context environment prior on the topological subgraph.

[0021] According to the above technical solution, the geometric continuity verification branch in step four operates as follows: Nearest neighbor point cloud data within a preset control window is extracted around the spatial coordinates of the current suspected defect. An explicit quadratic polynomial surface equation is fitted using least squares regression. The first and second partial derivatives of the fitted equation are substituted into the first and second basic forms of the surface for analytical solution to obtain the surface coefficients at the core point to be evaluated, and then the Gaussian curvature is calculated. The mean curvature C is used to quantify the discontinuous variation and undulation of the surface of abnormal defects.

[0022] According to the above technical solution, the spatial shadow occlusion mutual verification branch in step four is calculated as follows: the solar declination angle is derived from the Julian day based on the current high-precision timestamp, and the high-precision solar altitude angle at the current moment is calculated by combining the geographical latitude and longitude and the true solar hour angle. With azimuth The angle is used as the ray direction vector. During online inference, real-time odometer information is used as the retrieval key value. The geometric parameters of the boundary box of the structure in the railway standard spatial line B-BIM database that matches the current kilometer mark are dynamically and synchronously extracted. The ray tracing projection intersection is solved under known spatial position and size. If the intersection of the ray and the surface of the inspection piece falls inside the visual boundary box, the indicator function is activated.

[0023] Based on the above technical solution, a multi-dimensional physical constraint joint decision verification operator... A dual-threaded asynchronous decoupled architecture is deployed in the embedded computing module: The first thread only executes the decision verification operator that does not include token serial autoregressive decoding. The forward tensor matrix calculation directly drives and intercepts alarm suspension responses in real-time inspection with extremely low latency of less than a preset millisecond. The second thread, acting as a self-explanatory module, only asynchronously calls the large model autoregressive text generation engine in the background after the first thread outputs a false alarm and the system hangs. It then appends the generated structured causal explanation text to the historical maintenance log to ensure that the long text generation behavior does not block the main real-time inspection pipeline.

[0024] According to the above technical solution, the method also includes an online closed-loop learning mechanism: for low-confidence samples with judgment probabilities in the fuzzy interval, the samples are automatically pushed to the manual review end for labeling, and the model parameters are incrementally updated online using the composite distillation loss function. By introducing a representative old sample playback set selected from historical data according to the class balance sampling strategy, the cross-entropy loss is used to supervise the fitting of new labels, while the KL divergence constraint is used to update the output probability distribution of the model before and after on the old sample playback set, so as to force the model to learn new knowledge while constraining the output of old knowledge to not change significantly.

[0025] According to the above technical solution, the input end of the visual encoder is equipped with adaptive contrast enhancement processing: real-time analysis of the grayscale histogram of the image, for local low contrast areas caused by low illumination conditions in the tunnel or strong light overexposure conditions outdoors, a limited contrast adaptive histogram equalization algorithm is used to stretch the local contrast of overlapping sub-blocks, and the local contrast amplification factor is limited by a preset cropping threshold to ensure that the bottom pixel features entering the encoder have recognizability in order to reduce false alarms caused by image quality degradation.

[0026] Compared with the prior art, the present invention has the following beneficial effects: By introducing a multimodal large model, this invention overcomes the limitations of traditional visual detection models that rely solely on local pixel features, expanding the detection dimension from a single visual image to a multidimensional semantic space that includes spatial geometry, historical records, and environmental constraints. Utilizing the logical reasoning capabilities of the large model, it can effectively identify and filter out false alarms caused by light and shadow interplay, water stains, oil stains, and environmental debris.

[0027] By constructing a dynamic knowledge graph and representing it as environmental constraint operators, this invention transforms industry regulations and historical patterns in the field of railway catenary operation and maintenance into computational features that the model can understand. This knowledge-enhanced detection mechanism significantly improves the model's robustness and generalization ability when facing small-sample defects and complex long-tail conditions.

[0028] This invention outputs decision-making criteria through a self-explanatory module and combines it with an online closed-loop learning mechanism to achieve continuous iteration of model performance. High-quality feedback after manual review directly impacts model fine-tuning, enabling the system to continuously incorporate the judgment logic of frontline operations personnel, thereby continuously reducing the false alarm rate and alleviating alarm fatigue.

[0029] By employing a hierarchical filtering architecture and model quantization acceleration technology, this invention achieves high-dimensional logical reasoning while maintaining extremely high processing performance. It can adapt to high-frequency, large-scale automated inspection tasks, providing a solid technical guarantee for the inherent safety of railway catenary maintenance networks. In summary, this invention, through the organic combination of multimodal large models, environmental constraint operators, and cross-modal alignment mechanisms, fundamentally reconstructs the decision-making logic of railway catenary maintenance and inspection. It achieves a leap from simple pattern recognition to complex semantic cognition, possessing significant theoretical and engineering application value for reducing maintenance costs and improving the intelligent management level of railway and urban railway catenary maintenance. Attached Figure Description

[0030] Figure 1 This is a flowchart of the operation and maintenance detection false alarm suppression method of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Example 1

[0033] like Figure 1 As shown, a method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model is presented. The method includes the following specific steps: The first step is to construct a multi-dimensional data synchronous acquisition and preprocessing system. The acquisition system uses a sensor array mounted on the overhead contact line inspection vehicle to acquire real-time physical information of the overhead contact line scene during the vehicle's operation. The sensor array includes a high-resolution linear industrial camera, a 3D LiDAR, an inertial measurement unit, a global navigation satellite system receiver, and an onboard edge computing unit. The linear industrial camera is used to acquire high-dimensional visual feature images of key components of the overhead contact line (such as contact wire, catenary wire, insulators, and positioning clamps). The sampling resolution is set to 0.1 mm / pixel, and the exposure time is dynamically compensated according to the vehicle speed to eliminate motion blur. Specifically, the speed-adaptive exposure compensation module establishes a dynamic mapping relationship between the vehicle speed and the line-scan camera's trigger cycle by acquiring pulse signals from the wheel axle encoder in real time. This ensures that the longitudinal displacement of the inspection vehicle during a single exposure is always less than a preset blur tolerance threshold, thereby guaranteeing the consistency of spatial resolution of the acquired images at different vehicle speeds. The 3D LiDAR synchronously acquires point cloud data of the spatial geometric cross-sections of the overhead contact line equipment. The inertial measurement unit (IMU) and the global navigation satellite system (GNSS) receiver perform integrated navigation calculations using extended Kalman filtering, outputting high-precision spatial pose and odometer information of the inspection vehicle in the WGS-84 coordinate system. The edge computing unit preprocesses the acquired raw data, including image flat-field correction to eliminate sensor pixel response inconsistencies and optical vignetting effects, target region extraction from the point cloud (segmenting the contact line and support device point sets based on a random sampling consensus algorithm), and hard synchronization of timestamps from multiple sensors.

[0034] Specifically, the implementation scheme for hard synchronization of timestamps is as follows: using the second pulse signal output by the global navigation satellite system receiver as the global synchronization reference, a multi-channel synchronization trigger controller built by field-programmable logic devices divides the second pulse into a trigger pulse sequence that matches the acquisition frequency of each sensor, thereby driving the line trigger signal of the linear array camera and the scanning start trigger signal of the lidar respectively; each sensor latches its local timestamp and corresponding pulse count value at the instant it receives the trigger signal, forming a triplet time identifier of PPS count + pulse sequence number + sensor local timestamp; in the data post-processing stage, the vehicle edge computing unit uses the correspondence between the calibration delay parameters of each sensor and the trigger pulse sequence number to perform offset compensation and interpolation alignment on the original timestamp, controlling the residual time axis deviation of multi-source data measured at the same physical sampling position within a preset 500 nanoseconds.

[0035] Furthermore, after preprocessing, the method also includes a preliminary step for generating candidate regions for potential defects. Specifically, a lightweight convolutional neural network is used to perform preliminary target detection on the flat-field calibrated linear scan camera image. This lightweight convolutional neural network adopts a feature pyramid network structure and outputs bounding boxes of several potential defects along with their corresponding confidence scores. Simultaneously, based on the timestamps of multi-sensor hard synchronization, local point cloud regions that overlap with the bounding boxes are extracted from the point cloud at the corresponding locations. The bounding boxes and their associated image patches, local point cloud regions, and positional information provided by the integrated navigation system together constitute the potential defect targets, serving as input for subsequent steps.

[0036] The second step involves constructing a cross-modal semantic feature space mapping mechanism. This mechanism uses a pre-trained large-scale visual and language model as its foundation to extract deep features from image patches of suspected disease targets output from the initial detection step. The large-scale model consists of a visual encoder, a text encoder, and a cross-modal alignment layer. The visual encoder employs a Transformer-based visual backbone network, specifically a hierarchical Vision Transformer architecture, containing four stages with different resolutions. Each stage integrates spatial attention and channel attention mechanisms. The visual encoder divides image patches into fixed-size pixel blocks and encodes them into high-dimensional vector sequences. The spatial attention mechanism globally models the pairwise correlations between all pixel blocks within the sequence, capturing the continuity and discontinuity features of subtle texture details in the spatial dimension of contact network defects. The channel attention mechanism explicitly weights the dependencies between channels in the feature map, adaptively selecting feature channels that contribute significantly to defect classification and effectively suppressing stray light features generated by rainwater reflection and metal surface reflection. The text encoder processes prior knowledge text, historical maintenance records, and inspection procedure descriptions related to catenary defects, transforming them into text embedding representations of the same dimension as image features. During the initialization phase, the text encoder employs a pre-trained Chinese language model fine-tuned based on a catenary-specific corpus. Its vocabulary covers all technical terms in the "Classification and Codes of Catenary Equipment" and the "Catenary Operation and Maintenance Procedures," enabling accurate parsing of the semantic connotations of specialized terms such as broken strands, wear, insulator damage, and loose electrical connections.

[0037] Furthermore, a dynamic Adapter fine-tuning structure is introduced into the cross-modal alignment layer. This structure, while freezing the backbone parameters of the pre-trained large model, embeds a lightweight, learnable bottleneck layer in the residual connection path of the Transformer, specifically for learning the mapping relationship between contact wire-specific terminology and defect images. For difficult sample pairs such as contact wire wear and surface stains, which have highly similar visual features but drastically different semantic attributes, this invention designs an adaptive loss function with adaptive topological boundary constraints for the spatial topological boundary of difficult sample pairs to maintain general common-sense understanding while forcibly separating highly similar false alarm features in the latent space. The overall loss function is defined as:

[0038]

[0039] The above formula constructs the core optimization boundary of this scheme through deep decoupling and superposition of the global cross-modal generalization comparison term and the local hyperspherical constraint term. The meanings of each character and variable parameter are explicitly defined as follows: This represents the total number of positive sample pairs in the current training batch. This represents the index for looping through positive sample pairs within the current batch; This represents the traversal index of all text features within the current batch; Indicates the first The high-dimensional feature vector of the i-th image and its corresponding i-th image Cosine similarity score between each correct prior text embedding vector; Indicates the first The high-dimensional feature vector of the i-th image and the i-th Cosine similarity score between unpaired text embedding vectors; A preset temperature scalar coefficient representing the degree of smoothness in controlling the similarity distribution; This represents the adaptive weighting coefficient used to balance the global contrastive loss and the marginal loss term for hard negative samples; This refers to the set of difficult negative samples selected in the current batch using the difficult negative sample mining strategy, whose visual cosine similarity to the current anchor sample is greater than a set hard threshold. Represents the set of difficult negative samples The total number of negative samples within; Represents the set of difficult negative samples The current negative sample index variable in the text; This represents the central spatial feature vector output by the visual encoder for the contact wire wear anchor point sample; This represents the projection vector of negative samples with high false alarm interference, such as stains on the contact wire surface, in the feature space. This represents the cosine similarity function used to calculate the similarity between two vectors. This represents the preset topological geometric boundary distance threshold parameter, used to forcibly generate inter-class isolation zones on the hypersphere.

[0040] The third step involves constructing a knowledge graph-based overhead contact system environmental constraint operator. This operator parses a historical maintenance database to build a dynamic knowledge graph encompassing the topological relationships of overhead contact system assets, historical trends in damage evolution, and geographical environmental attributes. Specifically, the knowledge graph construction includes entity extraction from unstructured text and relation evolution modeling. Entity extraction utilizes a named entity recognition algorithm based on conditional random fields to extract key fields from historical work orders, inspection reports, and maintenance logs, including timestamps, pole or anchor number, component type, status description, and maintenance action type. These fields are then assembled into structured records. Relationship evolution modeling sorts multiple structured records of the same location and component along a timeline, extracts state transition sequences, and stores these sequences as temporal evolution attributes of component entities in the knowledge graph. Entities in the graph include overhead contact system section numbers, pole numbers, contact wire models, insulator models, environmental corrosion levels (e.g., coastal salt spray areas), and seasonal lighting characteristics; relation types include connection, support, previously occurred, and repaired.

[0041] Furthermore, in the online inference stage, this invention uses a graph convolutional neural network to learn the representation of the knowledge graph. The system uses the location information (pole number or anchor segment number) of suspected disease targets output from the preliminary detection step as the query anchor point. It retrieves all related entities and relationships within a preset radius from the graph database to form a local subgraph. Multi-hop graph convolutional propagation and attention pooling are applied to the local subgraph to aggregate and transform long-term external deterministic environmental priors, such as the asset status attributes of spatial neighbor nodes, the geographical salt spray corrosion rating of the area, and the frequency of historical diseases, into an environmental constraint vector. The environmental constraint vector and the visual feature vector extracted in the second step are concatenated along the channel dimension at the feature fusion layer, enabling the model to make comprehensive judgments based on spatiotemporal context.

[0042] The fourth step involves executing false alarm suppression decisions based on large-scale model logical reasoning. This step inputs the comprehensive features of suspected defects, incorporating environmental constraint information, into a multimodal large-scale model decoder with reasoning capabilities. The decoder employs an autoregressive architecture based on instruction fine-tuning, performing attribute verification on alarm targets using preset logical decision templates. These templates include: geometric consistency verification, temporal evolution logic verification, and multimodal cross-verification.

[0043] Specifically, geometric consistency verification assesses surface anomalies by analyzing the local differential geometric features of discrete point cloud data. First, it retrieves the 3D LiDAR point cloud at the corresponding location and extracts data around the spatial coordinates of the suspected defect. Using nearest neighbor point cloud data, the explicit quadratic polynomial surface equation is fitted using least squares regression: .in, These are the spatial three-dimensional axial coordinate elements in the local measurement coordinate system; This represents the total number of nearest neighbor point cloud targets within the preset local feature extraction window; Let represent the coefficients of the quadratic polynomial surface obtained by least squares regression fitting. Based on the first and second partial derivatives of the surface equation, the first and second fundamental form coefficients of the surface are analytically solved at the core points to be evaluated, and then the Gaussian curvature reflecting the continuity of the component surface is calculated. With mean curvature .

[0044] Furthermore, the temporal evolution logic verification utilizes a time-exponential decay model to quantitatively assess the effectiveness of historical maintenance records. The system queries the knowledge graph to obtain the time of the most recent maintenance action to eliminate the current suspected defect at this pole location, and combines this with the current inspection time to calculate the current protection reliability.

[0045] Furthermore, multimodal cross-verification is used to identify false alarm interference caused by environmental optical conditions on imaging. Based on the current high-precision timestamp and latitude / longitude coordinates provided by the integrated navigation system, the system calculates the current high-precision solar declination angle, true solar hour angle, and corresponding solar altitude angle using a standard solar position model. With azimuth Simultaneously, the system pre-imports a railway standard spatial alignment B-BIM (Bridge-Building Information Modeling) database. During online inference, it uses real-time odometer information as the retrieval key to dynamically and synchronously extract three-dimensional bounding box geometric parameter models of structures along the line, such as supports, beams, and additional conductors, that match the current kilometer marker. Given the known spatial location and dimensions, ray tracing projection calculations are performed using the sun's direction vector as the ray direction. The coordinates of the intersection point between the ray and the surface of the inspected component are accurately calculated, thereby determining whether the current visual inspection frame falls completely within the shadow shading area generated by the structure.

[0046] To perform high-dimensional parallel decision-making on the multidimensional physical logic of geometric continuity, historical time series reliability, and optical shading factor, this invention derives and constructs a highly integrated spatiotemporal-optical multidimensional physical constraint joint decision verification operator. This operator is deployed directly at the output layer of the large model decoder, and its mathematical analytical expression is defined as:

[0047]

[0048] The aforementioned composite operator achieves a deep and strong coupling between the data-driven large model and the physical prior rules by dynamically embedding the deterministic boundary laws of the physical world into the last layer of the network in the form of a nonlinear decay matrix. Among these, This represents the input multimodal feature matrix containing suspected diseased image patches and local point cloud subsets; This represents the contextual semantic constraint vector extracted and propagated from the knowledge graph by the environment constraint operator; This represents the raw, unconstrained output of the multimodal large model decoder. An unnormalized log-probability vector; Indicates the target decision category variable to be evaluated; This represents the traversal index of all determinate output categories in the system; This represents the total number of pre-defined decision categories that can be determined by the system. and These represent the first and second halves of the logarithmic probability vector, respectively. The and the first The original score components of each category; This represents a surface continuity gate function constructed based on differential geometric curvature, where the calculated Gaussian curvature... The absolute value is less than the preset lower threshold and the average curvature The function outputs 1 when the absolute value is less than the preset upper threshold, and 0 otherwise. The indicator function is defined if and only if the input category points to the system's default false alarm category. The output is 1 if the condition is met, otherwise the output is 0. This indicates that the preset false alarm category dynamically adjusts and amplifies the weighting coefficient. This represents the absolute value of the time difference between the current inspection time and the time of the most recent valid maintenance action retrieved from the knowledge graph; Indicates the effective protection period time constant; This represents a ray-based spatial shading indicator function, which uses the calculated solar azimuth angle. Solar altitude angle And the three-dimensional boundary box geometric parameter model of the structure extracted in real time from the B-BIM database. As input, if the shadow intersection point is found to fall within the visual bounding box through ray tracing, the output is 1; otherwise, the output is 0.

[0049] Furthermore, when the operator finally outputs the target probability Greater than the preset hard filter threshold ( When the value is between 0.85 and 0.95 (preferably 0.92), the system automatically suspends the alarm. The false alarm suppression decision also includes a self-explanatory module based on comparative reasoning. When the system determines that a suspected alarm is a false alarm, the large model decoder synchronously calls the background autoregressive engine to generate a structured technical text explaining the physical logic for suppressing the alarm.

[0050] Furthermore, the method of this invention also includes an online closed-loop learning mechanism. For marginal samples that the large model determines to be uncertain or with low confidence (i.e., samples whose maximum probability value of the operator output is between 0.4 and 0.6), the system automatically pushes them to the manual review end. The true labels after manual review, along with the corresponding multimodal raw data, are used to update the model online through an incremental learning algorithm and a composite distillation loss function. The system introduces a representative old sample replay set selected from historical data according to a class balance sampling strategy to constrain the output distribution of the model on the old samples before and after the update to prevent explicit drift, thus achieving continuous evolution of the model.

[0051] In a preferred embodiment of the present invention, the multimodal large model employs FP16 quantization and operator fusion technology during deployment. This integrates the model inference operator into an embedded computing module with layer fusion and automatic kernel tuning, achieving a real-time processing speed of at least 25 frames per second, meeting the real-time online detection requirements of inspection equipment operating at 80 km / h. Specifically, to meet the high frame rate real-time processing requirements, the large model decoder adopts a dual-thread asynchronous decoupled architecture: the first thread executes only the multi-dimensional physical constraint joint decision verification operator. The forward tensor matrix calculation, since it does not involve the serial autoregressive decoding of tokens, has a forward single inference latency of less than 15 milliseconds, directly determining and driving the real-time suspension or reporting response of alarm signals; the second thread is a self-explanatory module, which only asynchronously calls the autoregressive text generation engine in the background after the first thread outputs a false alarm and the system determines that it is suspended, and appends the generated structured causal explanation text to the historical maintenance log, thereby ensuring that the long text generation behavior of the large model will never block the main waterline of real-time inspection.

[0052] In a preferred embodiment of the present invention, the input end of the visual encoder is equipped with an adaptive contrast enhancement algorithm. This algorithm analyzes the grayscale histogram of the image in real time, and for low-contrast areas caused by insufficient lighting in the tunnel or strong outdoor light, it uses Limiting Contrast Adaptive Histogram Equalization (CLAHE) to perform local contrast stretching.

[0053] In a preferred embodiment of the present invention, the text encoder employs a Chinese pre-trained model fine-tuned based on a catenary corpus during the initialization phase. Its vocabulary covers all technical terms in the "Classification and Code of Catenary Equipment" and the "Catenary Operation and Maintenance Procedures." This enables the model to accurately parse complex technical instructions.

[0054] In a preferred embodiment of the present invention, the system has multi-level alarm filtering logic. In the initial detection stage, a convolutional neural network performs high-speed target extraction; in the secondary fine verification stage, the multimodal large model of the present invention performs semantic re-evaluation on the initially extracted suspected targets. This hierarchical filtering architecture reduces the overall false alarm rate by more than 85% while ensuring system throughput.

[0055] Example 2

[0056] This description aims to enable those skilled in the art to fully reproduce the functional modules and processes covered by this invention through detailed technical descriptions. In the field of automated operation and maintenance of railway catenary systems, the false alarm suppression method for railway catenary operation and maintenance based on a multimodal large model proposed in this invention solves the engineering problem of traditional algorithms' inability to distinguish between real catenary defects and environmental noise under complex weather conditions, light and shadow interference, and single visual features through the deep integration of multi-source sensor arrays and a deep semantic reasoning engine. The method of this invention starts with the synchronous acquisition of underlying physical data, constructs a cross-modal semantic feature space mapping mechanism, introduces a catenary environmental constraint operator based on a knowledge graph, and finally relies on a multimodal large model decoder with logical reasoning capabilities to achieve accurate suppression of false alarms.

[0057] Specifically, the first step of this invention is to construct a multi-dimensional data synchronous acquisition and preprocessing system. The hardware foundation of this system is a sensor array mounted on a catenary inspection vehicle (such as a comprehensive inspection train, catenary maintenance vehicle, or mobile robot). This array includes a high-resolution linear industrial camera, a 3D LiDAR, an inertial measurement unit (IMU), a Global Navigation Satellite System (GNSS) receiver, and an onboard edge computing unit. The high-resolution linear industrial camera uses a CMOS sensor with a high dynamic range and an effective pixel count of no less than 8192 points to acquire high-dimensional visual feature images of key components of the catenary. In actual engineering deployment, the sampling resolution is set to 0.1 mm / pixel to ensure the capture of micro-cracks in the contact wire and the condition of fasteners. To eliminate motion blur, the system integrates a speed-adaptive exposure compensation module. This module establishes a dynamic mapping relationship between the vehicle speed and the line-scan camera's trigger cycle by acquiring the pulse signals fed back from the wheel axle encoder in real time. The 3D LiDAR synchronously acquires high-density point cloud data of the spatial geometric cross-section of the catenary equipment, with a scanning frequency of no less than 200 Hz and a point cloud accuracy controlled within ±1 mm. The inertial measurement unit (IMU) and the global navigation satellite system (GNSS) receiver work together to output high-precision spatial pose and odometer information through a combined navigation algorithm. The onboard edge computing unit is responsible for real-time preprocessing of the acquired raw data, including image flat-field correction, target component point set segmentation of point cloud data, and hard synchronization of timestamps from multiple sensor data. Afterwards, offset compensation and interpolation alignment are performed using calibration delay parameters to control the residual deviation of measurements of the same physical sampling location from multiple sources within 500 nanoseconds on the time axis, ensuring pixel-level alignment of multi-source data on a spatial scale even at high speeds.

[0058] Furthermore, after preprocessing, the method performs a preliminary suspected defect candidate region generation step. Specifically, a lightweight convolutional neural network is used to perform forward inference on the entire linear camera image after flat-field correction, generating a series of suspected defect candidate regions containing bounding box coordinates and preliminary confidence scores. Simultaneously, based on the timestamps of multi-sensor hard synchronization, a subset of local point clouds that spatially overlaps with each bounding box is cropped from the point cloud of the corresponding frame. These bounding boxes, image patches, local point clouds, and pole numbers or anchor segment numbers provided by the integrated navigation system together constitute structured suspected defect targets.

[0059] In the second step of this invention, constructing a cross-modal semantic feature spatial mapping mechanism is crucial for converting visual information into semantic information. This mapping mechanism uses a pre-trained visual and language large model as its basic framework to perform deep feature extraction on each image patch of suspected defects output from the preliminary detection step. The large model consists of a visual encoder, a text encoder, and a cross-modal alignment layer. Specifically, the visual encoder adopts a hierarchical VisionTransformer architecture, with each stage integrating spatial attention and channel attention mechanisms. The spatial attention mechanism captures the texture details of defects, while the channel attention mechanism adaptively selects feature channels that contribute highly to defect classification, thereby effectively suppressing stray light features generated by rainwater reflection and metal reflection. The text encoder is responsible for processing prior knowledge texts related to catenary defects, historical maintenance records, and regulations such as the "Catenary Operation and Maintenance Procedures." In the initialization phase, the text encoder uses a pre-trained Chinese language model fine-tuned based on a catenary professional corpus, whose vocabulary covers all technical terms in the "Catenary Equipment Classification and Codes" and the "Catenary Operation and Maintenance Procedures." The cross-modal alignment layer calculates the cosine similarity between local image features and text semantic features using a contrastive learning algorithm. A dynamic adapter fine-tuning structure is introduced in this process. This structure embeds a lightweight, learnable bottleneck layer into the residual connection path of the Transformer while freezing the backbone parameters of the pre-trained large model. During training, a hard negative sample mining strategy is employed: using worn samples as anchors, stain samples with a visual feature cosine similarity exceeding a threshold are marked as hard negative samples. During training, the aforementioned hard samples are used to adaptively adjust the spatial topological boundary loss function. Dynamically coordinate gradient backpropagation of all network parameters is performed. By using a penalty term, the relative distance between two types of ground objects that are visually extremely similar but have completely different physical properties is forcibly widened in the hyperspherical feature space, thus solving the long-tail misclassification problem that is prone to occur in conventional detection algorithms.

[0060] Furthermore, the third step of this invention provides external knowledge support for false alarm suppression by constructing a knowledge graph-based overhead contact system environmental constraint operator. The operator constructs a dynamic knowledge graph encompassing the topological relationships of overhead contact system assets, the evolution trends of defects over the years, and geographical environmental attributes by parsing historical maintenance databases. The construction of the knowledge graph specifically includes entity extraction from unstructured text and relation evolution modeling. The system utilizes a named entity recognition algorithm based on conditional random fields to extract key fields from historical work orders, overhead contact system inspection reports, and maintenance logs, assembling them into structured records, and further constructing a state transition sequence according to time sequence, storing it in a distributed graph database. Entities in the graph include, but are not limited to, overhead contact system section numbers, contact wire models, insulator models, environmental conditions (such as coastal salt spray areas), and seasonal lighting characteristics; relation types include connection, support, previously occurred, and repaired. This invention uses graph convolutional neural networks to perform representation learning on knowledge graphs. Using the pole number or anchor segment number provided in the preliminary detection step as the query anchor point, it retrieves local subgraphs. Through multi-hop graph convolutional propagation and attention pooling, it aggregates the inherent anti-corrosion rating information of the geographical environment along the route, and then generates a fixed-dimensional external environment prior semantic constraint vector, which is then fed into the fusion layer.

[0061] In the fourth step, this invention performs false alarm suppression decision-making based on large-model logical reasoning. This step inputs suspected disease features, incorporating environmental constraint information, into a multimodal large-model decoder with reasoning capabilities. The decoder employs an autoregressive architecture based on instruction fine-tuning, performing multidimensional verification of alarm targets using preset logical judgment templates. Specifically, the system utilizes a spatiotemporal-optical multidimensional physical constraint joint decision verification operator deployed in the last layer of the decoder. It replaces the conventional pure probability output.

[0062] In the operator's operational logic, the geometric consistency branch first retrieves the discrete laser point cloud under the corresponding kilometer marker, and extracts the area around the spatial coordinates of the current suspected defect. Using nearest neighbor point cloud data, the explicit quadratic polynomial surface equation is fitted using least squares regression: The system identifies the independent variable, dependent variable, and regression polynomial coefficients, and uses these to quickly establish the equation of a locally quadratic continuous characteristic surface. It then analytically derives the differential geometric partial derivative coefficients of this surface and calculates the Gaussian curvature and mean curvature, which characterize the surface undulations of the abnormal defect. If the absolute value of the calculated Gaussian curvature is less than a preset lower threshold and the absolute value of the mean curvature is less than a preset upper threshold, it indicates that the geometric structure of the suspected area is continuous and has not fractured or collapsed. In this case, the output of the geometric continuity gating function is 1.

[0063] Meanwhile, the temporal evolution branch queries the time span between the component and the last major overhaul or replacement operation in the current graph, and calculates the exponential decay of the protective performance using the time constant.

[0064] Simultaneously, the optical cross-verification branch calculates the high-precision solar altitude angle and azimuth angle at the current moment using the current timestamp provided by the Global Navigation Satellite System receiver. Using this angle as the ray direction vector, it retrieves the railway standard spatial alignment B-BIM database stored in the onboard edge computing unit and dynamically synchronized with the real-time odometer position. Ray intersection point tracking is then performed on the structure boundary box model. If the calculation indicates that the ray path is intercepted by a support or beam, and the intersection of the ray and the surface of the inspected object falls exactly within the current visual recognition frame, the solar shading indicator function is activated and outputs 1.

[0065] Ultimately, the operator synthesizes these three types of physical indicators and dynamically rescales the log-probability vector output by the deep semantic feature layer of the original large model, thereby directly blocking the illusions caused by light and shadow in the large model. When the overall correction probability of a false alarm exceeds a preset hard threshold (set to 0.92 in this embodiment), the edge system initiates a dual-thread asynchronous decoupling architecture. The first thread instantly suspends and intercepts the false alarm signal; simultaneously, the second thread is released to asynchronously and serially generate a structured causal technical explanation text in the background and append it to the system maintenance log. This ensures that the main inspection line maintains a real-time performance of no less than 25 frames per second while giving the automated machine vision system highly traceable self-explanatory logic.

[0066] Example 3

[0067] This embodiment selects a railway trunk line as the experimental section, with a total length of 35 kilometers. This section includes elevated, tunnel, and ground-level sections, with complex environmental lighting conditions and numerous sources of visual false alarms caused by rain, dirt, and light and shadow.

[0068] Data acquisition: Using the sensor array of the present invention, the resolution of the linear array camera is set to 8192 points, and the point cloud density of the lidar is set to no less than 5000 points per meter of travel.

[0069] Model configuration: A hierarchical ViT is used as the visual encoder with approximately 300M parameters. It integrates a knowledge graph of overhead contact line assets extracted based on conditional random fields, and the false alarm suppression threshold is set to 0.92. In the model optimization and decision layers, the adaptive loss function for hard sample pairs with spatial topological boundaries and the joint decision verification operator for spatiotemporal-optical multidimensional physical constraints derived in this invention are fully utilized, and a dual-threaded asynchronous decoupled architecture is used for edge hardware deployment.

[0070] Comparative Example 1 uses a traditional contact network component defect detection algorithm based on deep convolutional neural networks (CNN, such as ResNet-101). This algorithm only relies on visual images for feature extraction and classification and does not have the ability to fuse cross-modal information or perform semantic reasoning.

[0071] Comparative Example 2 uses a visual detection + simple rule filtering method, that is, based on the CNN detection results, false positives are eliminated by manually preset geometric parameter hard thresholds (such as area and aspect ratio), which does not have the logical reasoning and knowledge graph constraint functions of large models.

[0072] The data compiled from a 30-day continuous inspection test on the aforementioned experimental road section is as follows.

[0073] Regarding the total number of alarm outputs, Comparative Example 1 had the highest total number of alarms, reaching 15,420; Comparative Example 2, after being filtered using hard rules, saw its total number of alarms decrease to 12,105; and Example 1 of this invention had a total number of alarms of 9,850. In all three tests, the number of actual defects confirmed by manual verification was 1,250. Regarding false alarms, Comparative Example 1 had a high number of false alarms, reaching 14,170, with a false alarm rate of 91.89% and an overall accuracy rate of only 8.11%; Comparative Example 2 saw a decrease in false alarms to 10,855, with a false alarm rate of 89.67% and an overall accuracy rate of 10.33%, but the number of missed detections increased from 42 to 85, and the recall rate decreased from 96.64% to 93.20%.

[0074] Embodiment 1 of this invention employs a hierarchical filtering architecture. In the first stage, a lightweight network generates 9850 preliminary alarms, including 1250 actual diseases and 8600 preliminary false alarms. It should be noted that in the first stage, the lightweight network completely covers and detects 1232 actual diseases during forward target extraction (i.e., the inherent missed detection in the first stage is 18, which is due to individual differences caused by the limitations of the low-level feature expression of the lightweight coarse-screening network). In the second stage, 8160 suspected targets are input into a multimodal large model. Through forward matrix operations based on a multidimensional physical constraint joint decision verification operator, 7478 false alarms are successfully suppressed, resulting in 682 false alarms, a false alarm rate of 6.92%, an accuracy of 64.68%, and only 18 missed detections, achieving a recall rate of 98.56%. This demonstrates that the large model can extremely accurately intercept false alarms in independent environments during the secondary fine-checking stage, without mistakenly eliminating any real defects successfully detected in the first stage. This ensures that the effective detection results from the preceding stages are perfectly preserved after passing through a high-intensity false alarm suppression operator. Because the verification operator operates directly at the decoding output layer using tensor scaling without token generation, the latency of a single inference operation on the first thread is controlled at 14 milliseconds, and the overall system processing speed reaches 26 FPS, fully meeting the requirements for completely real-time online suspension response at 80 km / h.

[0075] In specific engineering scenarios, when the inspection vehicle passes through an elevated section, strong sunlight casts alternating shadows on the contact wire surface, and traditional algorithms misjudge the edges of these shadows as cracks. This invention combines GPS data, elevation angles derived from the Julian day, and Gaussian curvature data from point clouds obtained through least-squares quadratic surface fitting. A geometric gating function confirms the surface continuity without abrupt changes, and ray tracing combined with boundary boxes extracted from the B-BIM database proves the shadows originate from the supports. At this point, the joint decision verification operator outputs a false alarm probability of 0.96, and the first thread suspends it, triggering a second thread in the background to generate the corresponding self-explanatory technical text. Inside the tunnel, contamination on the insulator surface is misjudged as damage by traditional CNNs. This invention, through historical maintenance records, discovers that the section has recently been cleaned. The environmental constraint operator reduces the weight of defects, and the hard sample boundary loss function, during training, has already widened the spatial distribution of such high-similarity samples, thus accurately suppressing false alarms.

[0076] In addition, the online closed-loop learning mechanism processes the visual features of novel composite material repair through a specially maintained old sample replay set (traversed by dedicated index variables), and after incremental learning, similar false alarms are reduced by more than 95%.

[0077] In summary, this invention reduces the false alarm rate from over 90% to 6.92%, improves the suppression efficiency by over 85%, while maintaining high recall and consistency, effectively solving the problem of false alarms in overhead contact line visual inspection.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0079] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model, characterized by: Includes the following steps: Step 1, suspected defect target extraction: During the operation of the inspection vehicle, the physical information of the contact network scene is acquired in real time through the onboard sensor array. The physical information includes high-dimensional visual feature images and spatial geometric cross-sectional point cloud data. The physical information is preprocessed with timestamp hard synchronization, and preliminary target detection is performed using a convolutional neural network to output structured suspected defect targets containing image blocks, local point cloud regions, and location information. Step 2, cross-modal feature space mapping: Image patches from suspected disease targets are input into the visual encoder of a pre-trained visual and language large model to capture the continuity and breakage features of disease texture details in the spatial dimension, and the feature channels are explicitly weighted to suppress stray light interference and generate high-dimensional feature vectors; at the same time, historical maintenance records and inspection procedure descriptions are input into the text encoder and converted into technical terminology text embedding vectors of the same dimension. Step 3: Constructing environmental constraint semantics: Parse the historical operation and maintenance database to construct a dynamic knowledge graph covering asset topology, disease evolution trends over the years, and geographical environmental attributes; During online reasoning, use the location information of suspected disease targets as query anchors to retrieve local subgraphs, and use graph convolutional neural networks to perform representation learning and attention pooling on the local subgraphs to generate semantic constraint vectors that encode prior information of the current detection location environment. Then, concatenate the semantic constraint vectors and high-dimensional feature vectors at the feature fusion layer along the channel dimension to output fused features. Step 4, Multi-dimensional Physical Constraint Decision: The fused features are input into the multimodal large model decoder for logical reasoning. By deploying a multi-dimensional physical constraint joint decision verification operator at the decoder's output layer, a geometric continuity verification branch reflecting whether the component surface has foreign object adhesion or a real geometric fracture is introduced, a temporal evolution logic verification branch reflecting the credibility of historical maintenance status, and a spatial shadow shading mutual verification branch reflecting environmental optical condition interference are introduced. The original logarithmic probability vector output by the decoder is dynamically scaled and corrected, and the final decision probability is calculated. When the decision probability of being judged as a false alarm exceeds the preset hard filtering threshold, the edge system automatically suspends the suspected defect target.

2. The method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model as described in claim 1, characterized in that: Step four involves deploying the multidimensional physical constraint joint decision verification operator at the decoder output layer. Its mathematical analytical expression is defined as: In the formula, This represents the input multimodal feature matrix containing suspected diseased image patches and local point cloud subsets; This represents the contextual semantic constraint vector extracted and propagated from the knowledge graph by the environment constraint operator; This represents the raw, unconstrained output of the multimodal large model decoder. An unnormalized log-probability vector; Indicates the target decision category variable to be evaluated; This represents the traversal index of all determinate output categories in the system; This represents the total number of pre-defined decision categories that can be determined by the system. and These represent the first and second halves of the logarithmic probability vector, respectively. The and the first The original score components of each category; This represents a surface continuity gate function constructed based on differential geometric curvature, where the calculated Gaussian curvature... The absolute value is less than the preset lower threshold and the average curvature The function outputs 1 when the absolute value is less than the preset upper threshold, and 0 otherwise. The indicator function is defined if and only if the input category points to the system's default false alarm category. The output is 1 if the condition is met, otherwise the output is 0. This indicates that the preset false alarm category dynamically adjusts and amplifies the weighting coefficient. This represents the absolute value of the time difference between the current inspection time and the time of the most recent valid maintenance action retrieved from the knowledge graph; Indicates the effective protection period time constant; This represents a ray-based spatial shading indicator function, which uses the calculated solar azimuth angle. Solar altitude angle And the three-dimensional boundary box geometric parameter model of the structure extracted in real time from the B-BIM database. As input, if the shadow intersection point is found to fall within the visual bounding box through ray tracing, the output is 1; otherwise, the output is 0.

3. The method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model as described in claim 1, characterized in that: The time stamp hard synchronization preprocessing in step one is as follows: using the second pulse signal output by the global navigation satellite system receiver as the global synchronization reference, the hardware synchronization trigger controller divides the second pulse to drive each sensor to perform synchronous data acquisition; each sensor latches its local timestamp and corresponding pulse count value at the instant it receives the trigger signal, forming a triplet time identifier composed of the second pulse count, the trigger pulse sequence number, and the sensor's local timestamp; in the post-processing stage, the correspondence between the calibration delay parameters of each sensor and the trigger pulse sequence number is used to perform offset compensation and interpolation alignment on the original timestamp, controlling the residual time axis deviation of multi-source data measured at the same physical sampling position within a preset nanosecond tolerance.

4. The method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model as described in claim 1, characterized in that: The large-scale vision and language model in step two includes a dynamic adapter fine-tuning structure. During the training phase, a hard negative sample mining strategy is established to address high-similarity false alarm interference, and an adaptive loss function for hard samples to spatial topological boundaries is adopted. Dynamic collaborative gradient backpropagation of all network parameters is performed, and the overall mathematical analytical expression of the loss function is defined as follows: In the formula: This represents the total number of positive sample pairs in the current training batch; This represents the index for looping through positive sample pairs within the current batch; This represents the traversal index of all text features within the current batch; Indicates the first The high-dimensional feature vector of the i-th image and its corresponding i-th image Cosine similarity score between each correct prior text embedding vector; Indicates the first The high-dimensional feature vector of the i-th image and the i-th Cosine similarity score between unpaired text embedding vectors; A preset temperature scalar coefficient representing the degree of smoothness in controlling the similarity distribution; This represents the adaptive weighting coefficient used to balance the global contrastive loss and the marginal loss term for hard negative samples; This refers to the set of difficult negative samples selected in the current batch using the difficult negative sample mining strategy, whose visual cosine similarity to the current anchor sample is greater than a set hard threshold. Represents the set of difficult negative samples The total number of negative samples within; Represents the set of difficult negative samples The current negative sample index variable in the text; This represents the central spatial feature vector output by the visual encoder for the anchor point sample; This represents the projection vector of the interfering negative sample into the feature space; This represents the cosine similarity function used to calculate the similarity between two vectors. This represents the preset topological geometric boundary distance threshold parameter, used to forcibly generate inter-class isolation zones on the hypersphere.

5. The method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model according to claim 1, characterized in that: In step three, the knowledge graph extracts key fields of time, space, component type, status, and maintenance action from historical data text using a named entity recognition algorithm based on conditional random fields. These fields are then assembled into quintuple structured records. Multiple records of the same entity are sorted along the time axis to extract and store the state transition sequence. During online inference, a graph convolutional neural network update formula is used to perform degree normalization aggregation and learnable matrix transformation on the features of neighboring nodes, enabling multi-hop information flow propagation of the spatiotemporal context environment prior on the topological subgraph.

6. The method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model according to claim 1, characterized in that: The geometric continuity verification branch in step four operates as follows: It extracts nearest-neighbor point cloud data within a preset control window around the spatial coordinates of the suspected defect; it then uses least squares regression to fit an explicit quadratic polynomial surface equation; finally, it substitutes the first and second partial derivatives of the fitted equation into the first and second fundamental forms of the surface for analytical solving to obtain the surface coefficients at the core point to be evaluated, and then calculates the Gaussian curvature. The mean curvature C is used to quantify the discontinuous variation and undulation of the surface of abnormal defects.

7. The method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model as described in claim 6, characterized in that: The spatial occlusion cross-validation branch in step four is calculated as follows: The solar declination angle is derived from the Julian day based on the current high-precision timestamp; then, the high-precision solar altitude angle at the current moment is calculated by combining the geographical latitude and longitude with the true solar hour angle. With azimuth and by solar altitude angle With azimuth As the ray direction vector; during online inference, real-time odometer information is used as the retrieval key value, and the geometric parameters of the structure boundary box in the railway standard spatial line B-BIM database that matches the current kilometer mark are dynamically and synchronously extracted. Under the known spatial location and size, the ray tracing projection intersection point is solved. If the intersection point of the ray and the surface of the inspected piece falls inside the visual boundary box, the indicator function is activated.

8. The method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model according to claim 1, characterized in that: Multidimensional physical constraint joint decision verification operator A dual-threaded asynchronous decoupled architecture is deployed in the embedded computing module: The first thread only executes the decision verification operator that does not include token serial autoregressive decoding. The forward tensor matrix calculation directly drives and intercepts alarm suspension responses in real-time inspection with extremely low latency of less than a preset millisecond. The second thread, acting as a self-explanatory module, only asynchronously calls the large model autoregressive text generation engine in the background after the first thread outputs a false alarm and the system hangs. It then appends the generated structured causal explanation text to the historical maintenance log to ensure that the long text generation behavior does not block the main real-time inspection pipeline.

9. The method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model according to claim 1, characterized in that: The method also includes an online closed-loop learning mechanism: for low-confidence samples whose decision probabilities are in the fuzzy interval, the samples are automatically pushed to the manual review end for labeling, and the model parameters are incrementally updated online using the composite distillation loss function. By introducing a representative old sample playback set selected from historical data according to the class balance sampling strategy, the cross-entropy loss is used to supervise the fitting of new labels, while the KL divergence constraint is used to update the output probability distribution of the model before and after on the old sample playback set, so as to force the model to learn new knowledge while constraining the output of old knowledge to not change significantly.

10. The method for suppressing false alarms in railway catenary maintenance and detection based on a multimodal large model according to claim 1, characterized in that: The input end of the visual encoder is equipped with adaptive contrast enhancement processing: real-time analysis of the grayscale histogram of the image; for local low contrast areas caused by low illumination conditions in tunnels or strong light overexposure conditions outdoors, a limited contrast adaptive histogram equalization algorithm is used to stretch the local contrast of overlapping sub-blocks, and the local contrast amplification factor is limited by a preset cropping threshold to ensure that the underlying pixel features entering the encoder have discernibility in order to reduce false alarms caused by image quality degradation.

Images