Rail transit multi-mode inspection dynamic adaptation AR (Augmented Reality) collaboration method

Abstract

The invention belongs to the technical field of intelligent traffic control, and particularly relates to a rail transit multi-modal inspection dynamic adaptation AR collaboration method, which comprises the following steps: S10, synchronously acquiring multi-modal data of rail transit facilities through a heterogeneous sensor network deployed in inspection equipment, and performing preprocessing and feature extraction by using a multi-modal feature extraction network to obtain a multi-modal feature extraction result; generating a unified feature representation of the facility state; s20, confidence evaluation, dynamic weight distribution and feature fusion are carried out on the unified feature representation, and a facility health state vector is generated; s30, at an edge computing node, performing anomaly recognition and risk assessment through a lightweight deep learning model and an adaptive Kalman filtering algorithm, and generating an AR instruction and a control instruction; s40, the AR terminal equipment receives the AR instruction and the control instruction, dynamically adapts to the content of an AR interface, and displays and interacts; and S50, establishing a closed-loop optimization mechanism, and updating model parameters in the step S20 and the step S30. The method has the effect of improving the operation safety and the maintenance efficiency.

Description

Technical Field

[0001] This application belongs to the field of intelligent transportation control technology, specifically relating to a dynamic adaptation AR collaborative method for multimodal inspection of rail transit. Background Technology

[0002] Rail transit inspection is a crucial link in ensuring operational safety. Traditional inspection methods mainly rely on manual labor or automated equipment with a single sensor (such as lidar or infrared thermal imaging), which has the following limitations:

[0003] Limited perception dimension: As described in patent CN112249091B, the system relies solely on lidar point cloud data, which cannot effectively identify thermal anomalies, vibration anomalies, or surface defects under visible light, leading to missed detections.

[0004] The model is rigid and has poor adaptability: For example, the infrared detection method with a fixed threshold used in patent CN113524032B is prone to false alarms when the ambient temperature changes or the equipment is momentarily overloaded, and it cannot be linked with other sensor data for verification.

[0005] Insufficient system synergy: Existing AR inspection systems (such as CN112249091B) lack dynamic and intelligent adaptation capabilities between the AR interface and the real environment and sensor data stream. For example, the brightness of the AR interface does not adjust with ambient light, and the information density cannot dynamically change according to the inspection task, resulting in poor readability and information overload in strong light or complex scenes.

[0006] High response latency: Data processing relies on the cloud, and the path from perception to AR presentation or control command issuance is lengthy. For example, the Jinan Metro inspection system has a response latency of more than 1 second, which cannot meet the needs of real-time early warning and interaction.

[0007] Lack of closed-loop optimization: The system cannot autonomously optimize the perception algorithm, fusion strategy and AR interaction logic based on historical inspection data and environmental changes, making it difficult to continuously improve performance.

[0008] Therefore, improvements are needed. Summary of the Invention

[0009] To address the above technical issues, this application provides a dynamic adaptation AR collaborative method for multimodal inspection of rail transit.

[0010] The first objective of this invention is achieved through the following technical solution:

[0011] A dynamic adaptation AR collaborative method for multimodal inspection of rail transit includes the following steps:

[0012] S10: Through a heterogeneous sensor network deployed on inspection equipment, multimodal data of rail transit facilities are collected synchronously, and preprocessed and feature extracted using a multimodal feature extraction network to generate a unified feature representation of facility status;

[0013] S20: Based on the spatiotemporal attention mechanism, the unified feature representation is subjected to confidence evaluation, dynamic weight allocation and feature fusion to generate a facility health status vector;

[0014] S30: At the edge computing node, based on the facility health status vector, anomaly identification and risk assessment are performed using a lightweight deep learning model and an adaptive Kalman filter algorithm, and AR commands and control commands are generated.

[0015] S40: The AR terminal device receives the AR instructions and control instructions, and dynamically adapts the content, display and interaction of the AR interface in combination with environmental parameters;

[0016] S50: Establish a closed-loop optimization mechanism and use the data uploaded by edge nodes through a federated learning framework to update the model parameters in S20 and S30.

[0017] In a preferred embodiment, step S10 involves: synchronously collecting multimodal data of rail transit facilities through a heterogeneous sensor network deployed on the inspection equipment, and preprocessing and extracting features using a multimodal feature extraction network to generate a unified feature representation of the facility status, specifically including:

[0018] S101: The multimodal data includes: lidar point cloud data, visible light video stream, infrared thermal imaging video stream, vibration sensor data, ambient light intensity data, and ambient humidity data;

[0019] S102: The unified feature representation is a structured data set obtained after standardizing and reducing the feature dimensionality of the multimodal data.

[0020] In a preferred embodiment, step S20, based on a spatiotemporal attention mechanism, involves confidence evaluation, dynamic weight allocation, and feature fusion of the unified feature representation to generate a facility health status vector, specifically including:

[0021] S201: Real-time assessment of the confidence level of each sensor's data in the current environment;

[0022] S202: Based on the confidence level, dynamically allocate the weights of each modality data in the fusion process;

[0023] S203: The weighted multimodal features are fused in the feature space to output a facility health status vector.

[0024] In a preferred embodiment, step S30: at the edge computing node, based on the facility health status vector, anomaly identification and risk assessment are performed using a lightweight deep learning model and an adaptive Kalman filter algorithm, and AR commands and control commands are generated, specifically including:

[0025] S301: The lightweight deep learning model includes a YOLOv8 model for object detection and an LSTM model for behavior prediction;

[0026] S302: The generated control commands include triggering an audible and visual alarm, extending the door closing time, or sending a warning signal to the dispatch center.

[0027] In a preferred embodiment, step S40: The AR terminal device receives the AR instructions and control instructions, and dynamically adapts the content, display, and interaction of the AR interface in conjunction with environmental parameters. The dynamic adaptation specifically includes:

[0028] S401: Content adaptation, dynamically adjusts the content and display density of virtual information in the AR interface according to the anomaly type and risk level;

[0029] S402: Display adaptation, dynamically adjusts the overall brightness, contrast and transparency of virtual information of the AR interface based on ambient light sensor data;

[0030] S403: Interactive collaboration, triggering audible and visual alarms and pushing early warning information when a high-risk situation is detected.

[0031] In a preferred embodiment, step S50: establishing a closed-loop optimization mechanism, using a federated learning framework to update the model parameters in S20 and S30 using data uploaded from edge nodes, specifically includes:

[0032] S501: Edge nodes encrypt and upload the desensitized processing results, model performance data, and environmental samples to the cloud;

[0033] S502: The cloud utilizes data from multiple edge nodes to jointly train and update the multimodal fusion model and the anomaly detection model through federated learning;

[0034] S503: Distribute the optimized model parameters to each edge node to achieve continuous autonomous evolution of the system model.

[0035] The second objective of this invention is achieved through the following technical solution:

[0036] Module 1: S10: Through a heterogeneous sensor network deployed on the inspection equipment, multimodal data of rail transit facilities are collected synchronously, and preprocessed and feature extracted using a multimodal feature extraction network to generate a unified feature representation of the facility status;

[0037] Second module: S20: Based on the spatiotemporal attention mechanism, perform confidence evaluation, dynamic weight allocation and feature fusion on the unified feature representation to generate a facility health status vector;

[0038] The third module: S30: At the edge computing node, based on the facility health status vector, anomaly identification and risk assessment are performed using a lightweight deep learning model and an adaptive Kalman filter algorithm, and AR commands and control commands are generated.

[0039] Fourth module: S40: The AR terminal device receives the AR instructions and control instructions, and dynamically adapts the content, display and interaction of the AR interface in combination with environmental parameters;

[0040] Module 5: S50: Establish a closed-loop optimization mechanism, using a federated learning framework to update the model parameters in S20 and S30 with the data uploaded by edge nodes.

[0041] The third objective of this invention is achieved through the following technical solution:

[0042] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the aforementioned dynamic adaptation AR collaborative method for multimodal inspection of rail transit.

[0043] The fourth objective of this invention is achieved through the following technical solution:

[0044] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described dynamic adaptation AR collaborative method for multimodal inspection of rail transit.

[0045] In summary, this application has at least one of the following beneficial effects:

[0046] It has achieved intelligent, real-time and adaptive rail transit inspection, significantly improving the comprehensiveness and accuracy of facility status monitoring, the timeliness of anomaly response, and the precision of AR guidance. At the same time, the continuous learning mechanism ensures the long-term evolution and reliability of system performance, ultimately achieving the goal of improving operational safety and maintenance efficiency. Attached Figure Description

[0047] Figure 1 This is a flowchart of an implementation of a dynamic adaptation AR collaborative method for multimodal inspection of rail transit, as proposed in this application.

[0048] Figure 2 This is an implementation flowchart of step S20 of the multimodal inspection dynamic adaptation AR collaborative method for rail transit in this application;

[0049] Figure 3 This is a schematic block diagram of a computer device according to this application. Detailed Implementation

[0050] The following is in conjunction with the appendix Figure 1-3 This application will be described in further detail.

[0051] In one embodiment, such as Figure 1 As shown, this application discloses a dynamic adaptation AR collaborative method for multimodal inspection of rail transit, which specifically includes the following steps:

[0052] S10: Through a heterogeneous sensor network deployed on inspection equipment, multimodal data of rail transit facilities are collected synchronously, and preprocessed and feature extracted using a multimodal feature extraction network to generate a unified feature representation of facility status;

[0053] S20: Based on the spatiotemporal attention mechanism, the unified feature representation is subjected to confidence evaluation, dynamic weight allocation and feature fusion to generate a facility health status vector;

[0054] S30: At the edge computing node, based on the facility health status vector, anomaly identification and risk assessment are performed using a lightweight deep learning model and an adaptive Kalman filter algorithm, and AR commands and control commands are generated.

[0055] S40: The AR terminal device receives the AR instructions and control instructions, and dynamically adapts the content, display and interaction of the AR interface in combination with environmental parameters;

[0056] S50: Establish a closed-loop optimization mechanism and use the data uploaded by edge nodes through a federated learning framework to update the model parameters in S20 and S30.

[0057] In this embodiment, the method constructs a complete technology chain from multimodal perception, intelligent fusion, edge decision-making to AR interaction, and finally closed-loop optimization. Its core principle lies in the organic integration of heterogeneous sensor data, spatiotemporal attention mechanisms, edge computing, AR human-machine collaboration, and federated learning through a closed-loop system architecture of "perception-fusion-decision-interaction-optimization," realizing the intelligent flow of information from the physical world to the digital space, and then back to physical operations. In terms of effectiveness, this method achieves intelligent, real-time, and adaptive rail transit inspection, significantly improving the comprehensiveness and accuracy of facility status monitoring, the timeliness of anomaly response, and the precision of AR guidance. Simultaneously, the continuous learning mechanism ensures the long-term evolution and reliability of system performance, ultimately achieving the goal of improving operational safety and maintenance efficiency.

[0058] S10 specifically includes:

[0059] S101: The multimodal data includes: lidar point cloud data, visible light video stream, infrared thermal imaging video stream, vibration sensor data, ambient light intensity data, and ambient humidity data;

[0060] S102: The unified feature representation is a structured data set obtained after standardizing and reducing the feature dimensionality of the multimodal data.

[0061] In this embodiment, S10 synchronously collects multimodal data through a heterogeneous sensor network and uses a multimodal feature extraction network for preprocessing and feature extraction to generate a unified feature representation. Its principle is based on multimodal learning theory, overcoming the limitations of single-modality data by integrating complementary information from different sensors (such as LiDAR, visible light video, etc.) and using deep learning models (such as CNN) for feature dimensionality reduction and standardization to form a structured data set. In terms of effectiveness, this method improves the comprehensiveness and robustness of data representation, reduces data redundancy and noise, and provides efficient and consistent feature input for subsequent fusion analysis, thereby improving the basic data quality of the entire inspection system.

[0062] like Figure 2 S20 specifically includes:

[0063] S201: Real-time assessment of the confidence level of each sensor's data in the current environment;

[0064] S202: Based on the confidence level, dynamically allocate the weights of each modality data in the fusion process;

[0065] S203: The weighted multimodal features are fused in the feature space to output a facility health status vector.

[0066] In this embodiment, S20 performs confidence assessment, dynamic weight allocation, and feature fusion on the unified feature representation based on a spatiotemporal attention mechanism to generate a facility health status vector. The principle behind this is that the attention mechanism can simulate human visual attention, prioritizing high-confidence data in the spatiotemporal dimension and adjusting the contribution of each modality through dynamic weights to achieve adaptive feature fusion. In terms of effectiveness, this method enhances the system's adaptability and reliability in complex environments, reduces the impact of low-quality data through weight allocation, improves the accuracy of health status assessment, provides more robust input for anomaly identification, and thus reduces the risk of false positives and false negatives.

[0067] S30, specifically includes:

[0068] S301: The lightweight deep learning model includes a YOLOv8 model for object detection and an LSTM model for behavior prediction;

[0069] S302: The generated control commands include triggering an audible and visual alarm, extending the door closing time, or sending a warning signal to the dispatch center.

[0070] In this embodiment, S30, at the edge computing node, uses a lightweight deep learning model (such as YOLOv8 and LSTM) and an adaptive Kalman filter algorithm based on the facility health status vector to perform anomaly identification and risk assessment, and generates AR commands and control commands. Its principle combines the high precision of deep learning with the dynamic estimation capability of Kalman filtering. The lightweight model ensures efficient operation of edge devices, and the adaptive algorithm handles sensor noise and environmental changes. In terms of results, it achieves real-time, low-latency anomaly detection and risk response. The generated control commands (such as triggering audible and visual alarms) improve emergency response speed, while edge computing reduces the burden on the cloud, improving system reliability and autonomy.

[0071] S40, specifically includes:

[0072] The dynamic adaptation specifically includes:

[0073] S401: Content adaptation, dynamically adjusts the content and display density of virtual information in the AR interface according to the anomaly type and risk level;

[0074] S402: Display adaptation, dynamically adjusts the overall brightness, contrast and transparency of virtual information of the AR interface based on ambient light sensor data;

[0075] S403: Interactive collaboration, triggering audible and visual alarms and pushing early warning information when a high-risk situation is detected.

[0076] In this embodiment, in step S40, the AR terminal device receives AR commands and control commands, and dynamically adapts the content, display, and interaction of the AR interface based on environmental parameters. Its principle is based on human-computer interaction and environmental adaptation technology. Content adaptation adjusts information density according to the type of anomaly, display adaptation adjusts visual parameters according to ambient lighting, and interactive collaboration triggers an early warning mechanism. In terms of effectiveness, this method optimizes the readability and user experience of the AR interface, ensures clear information presentation under different environmental conditions, and improves the operator's response efficiency and safety, especially in high-risk situations where it can quickly guide response actions.

[0077] S50, specifically includes:

[0078] S501: Edge nodes encrypt and upload the desensitized processing results, model performance data, and environmental samples to the cloud;

[0079] S502: The cloud utilizes data from multiple edge nodes to jointly train and update the multimodal fusion model and the anomaly detection model through federated learning;

[0080] S503: Distribute the optimized model parameters to each edge node to achieve continuous autonomous evolution of the system model.

[0081] In this embodiment, S50 establishes a closed-loop optimization mechanism, using data uploaded by edge nodes to update model parameters through a federated learning framework. Its principle is based on distributed machine learning, where edge nodes upload anonymized data and performance metrics, and joint training is performed in the cloud without sharing the original data, protecting privacy while enabling model iteration. In terms of effectiveness, this achieves continuous autonomous evolution of the system, with model parameters constantly optimized to adapt to new scenarios and data distributions, improving the accuracy of multimodal fusion and anomaly detection. Simultaneously, federated learning reduces the risk of centralized data storage, enhancing the system's scalability and long-term stability.

[0082] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0083] In one embodiment, a dynamic adaptation AR collaborative system for multimodal inspection of rail transit is provided. This system corresponds one-to-one with the dynamic adaptation AR collaborative method for multimodal inspection of rail transit described in the previous embodiment. The dynamic adaptation AR collaborative system for multimodal inspection of rail transit includes:

[0084] Module 1: S10: Through a heterogeneous sensor network deployed on the inspection equipment, multimodal data of rail transit facilities are collected synchronously, and preprocessed and feature extracted using a multimodal feature extraction network to generate a unified feature representation of the facility status;

[0085] Second module: S20: Based on the spatiotemporal attention mechanism, perform confidence evaluation, dynamic weight allocation and feature fusion on the unified feature representation to generate a facility health status vector;

[0086] The third module: S30: At the edge computing node, based on the facility health status vector, anomaly identification and risk assessment are performed using a lightweight deep learning model and an adaptive Kalman filter algorithm, and AR commands and control commands are generated.

[0087] Fourth module: S40: The AR terminal device receives the AR instructions and control instructions, and dynamically adapts the content, display and interaction of the AR interface in combination with environmental parameters;

[0088] Module 5: S50: Establish a closed-loop optimization mechanism, using a federated learning framework to update the model parameters in S20 and S30 with the data uploaded by edge nodes.

[0089] Optional, also includes:

[0090] Module 6: S101: The multimodal data includes: lidar point cloud data, visible light video stream, infrared thermal imaging video stream, vibration sensor data, ambient light intensity data, and ambient humidity data;

[0091] Module 7: S102: The unified feature representation is a structured data set obtained after standardizing and reducing the feature dimensionality of the multimodal data.

[0092] Optional, also includes:

[0093] Module 8: S201: Real-time assessment of the confidence level of each sensor's data in the current environment;

[0094] Module 9: S202: Based on the confidence level, dynamically allocate the weights of each modality data in the fusion process;

[0095] Module 10: S203: Fuse the weighted multimodal features in the feature space and output the facility health status vector.

[0096] Optional, also includes:

[0097] Module 11: S301: The lightweight deep learning model includes a YOLOv8 model for object detection and an LSTM model for behavior prediction;

[0098] Module 12: S302: The generated control commands include triggering an audible and visual alarm, extending the door closing time, or sending a warning signal to the dispatch center.

[0099] Optional, also includes:

[0100] Module 13: S401: Content Adaptation, dynamically adjusts the content and display density of virtual information in the AR interface according to the anomaly type and risk level;

[0101] Module 14: S402: Display adaptation, dynamically adjusts the overall brightness, contrast and transparency of virtual information of the AR interface based on ambient light sensor data;

[0102] Module 15: S403: Interactive Collaboration. When a high-risk situation is detected, it triggers an audible and visual alarm and pushes a warning message.

[0103] Optional, also includes:

[0104] Module 16: S501: Edge nodes encrypt and upload the desensitized processing results, model performance data, and environmental samples to the cloud;

[0105] Module 17: S502: The cloud utilizes data from multiple edge nodes to jointly train and update the multimodal fusion model and the anomaly detection model through federated learning;

[0106] Module 18: S503: Distributes the optimized model parameters to each edge node to achieve continuous autonomous evolution of the system model.

[0107] Specific limitations regarding the dynamic adaptation AR collaborative system for multimodal inspection of rail transit can be found in the limitations of the dynamic adaptation AR collaborative method for multimodal inspection of rail transit described above, and will not be repeated here. Each module in the aforementioned dynamic adaptation AR collaborative system for multimodal inspection of rail transit can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0108] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a dynamic adaptation AR collaborative method for multimodal inspection in rail transit.

[0109] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0110] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0111] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0112] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0113] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A dynamic adaptation AR collaborative method for multimodal inspection of rail transit, characterized in that, Includes the following steps: S10: Through a heterogeneous sensor network deployed on inspection equipment, multimodal data of rail transit facilities are collected synchronously, and preprocessed and feature extracted using a multimodal feature extraction network to generate a unified feature representation of facility status; S20: Based on the spatiotemporal attention mechanism, the unified feature representation is subjected to confidence evaluation, dynamic weight allocation and feature fusion to generate a facility health status vector; S30: At the edge computing node, based on the facility health status vector, anomaly identification and risk assessment are performed using a lightweight deep learning model and an adaptive Kalman filter algorithm, and AR commands and control commands are generated. S40: The AR terminal device receives the AR instructions and control instructions, and dynamically adapts the content, display and interaction of the AR interface in combination with environmental parameters; S50: Establish a closed-loop optimization mechanism and use the data uploaded by edge nodes through a federated learning framework to update the model parameters in S20 and S30.

2. The AR collaborative method for multimodal inspection of rail transit according to claim 1, characterized in that, Step S10: Multimodal data of rail transit facilities are synchronously collected via a heterogeneous sensor network deployed on the inspection equipment. Preprocessing and feature extraction are then performed using a multimodal feature extraction network to generate a unified feature representation of the facility's status. Specifically, this includes: S101: The multimodal data includes: lidar point cloud data, visible light video stream, infrared thermal imaging video stream, vibration sensor data, ambient light intensity data, and ambient humidity data; S102: The unified feature representation is a structured data set obtained after standardizing and reducing the feature dimensionality of the multimodal data.

3. The AR collaborative method for multimodal inspection of rail transit according to claim 1, characterized in that, Step S20: Based on the spatiotemporal attention mechanism, confidence evaluation, dynamic weight allocation, and feature fusion are performed on the unified feature representation to generate a facility health status vector, specifically including: S201: Real-time assessment of the confidence level of each sensor's data in the current environment; S202: Based on the confidence level, dynamically allocate the weights of each modality data in the fusion process; S203: The weighted multimodal features are fused in the feature space to output a facility health status vector.

4. The dynamic adaptation AR collaborative method for multimodal inspection of rail transit according to claim 1, characterized in that, Step S30: At the edge computing node, based on the facility health status vector, anomaly identification and risk assessment are performed using a lightweight deep learning model and an adaptive Kalman filter algorithm, and AR commands and control commands are generated, specifically including: S301: The lightweight deep learning model includes a YOLOv8 model for object detection and an LSTM model for behavior prediction; S302: The generated control commands include triggering an audible and visual alarm, extending the door closing time, or sending a warning signal to the dispatch center.

5. The AR collaborative method for multimodal inspection of rail transit according to claim 1, characterized in that, Step S40: The AR terminal device receives the AR instructions and control instructions, and dynamically adapts the content, display, and interaction of the AR interface in conjunction with environmental parameters. This dynamic adaptation specifically includes: S401: Content adaptation, dynamically adjusts the content and display density of virtual information in the AR interface according to the anomaly type and risk level; S402: Display adaptation, dynamically adjusts the overall brightness, contrast and transparency of virtual information of the AR interface based on ambient light sensor data; S403: Interactive collaboration, triggering audible and visual alarms and pushing early warning information when a high-risk situation is detected.

6. The dynamic adaptation AR collaborative method for multimodal inspection of rail transit according to claim 1, characterized in that, Step S50: Establish a closed-loop optimization mechanism, using a federated learning framework to update the model parameters in S20 and S30 with data uploaded from edge nodes, specifically including: S501: Edge nodes encrypt and upload the desensitized processing results, model performance data, and environmental samples to the cloud; S502: The cloud utilizes data from multiple edge nodes to jointly train and update the multimodal fusion model and the anomaly detection model through federated learning; S503: Distribute the optimized model parameters to each edge node to achieve continuous autonomous evolution of the system model.

7. A dynamic adaptation AR collaborative system for multimodal inspection of rail transit, characterized in that, include: Module 1: S10: Through a heterogeneous sensor network deployed on the inspection equipment, multimodal data of rail transit facilities are collected synchronously, and preprocessed and feature extracted using a multimodal feature extraction network to generate a unified feature representation of the facility status; Second module: S20: Based on the spatiotemporal attention mechanism, perform confidence evaluation, dynamic weight allocation and feature fusion on the unified feature representation to generate a facility health status vector; The third module: S30: At the edge computing node, based on the facility health status vector, anomaly identification and risk assessment are performed using a lightweight deep learning model and an adaptive Kalman filter algorithm, and AR commands and control commands are generated. Fourth module: S40: The AR terminal device receives the AR instructions and control instructions, and dynamically adapts the content, display and interaction of the AR interface in combination with environmental parameters; Module 5: S50: Establish a closed-loop optimization mechanism, using a federated learning framework to update the model parameters in S20 and S30 with the data uploaded by edge nodes.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the dynamic adaptation AR collaborative method for multimodal inspection of rail transit as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the dynamic adaptation AR collaborative method for multimodal inspection of rail transit as described in any one of claims 1 to 6.

Images