An intelligent railway operation and maintenance system, method, equipment, medium and product

Abstract

The present application discloses an intelligent railway operation and maintenance system, method, equipment, medium and product, which relate to the field of railway operation and maintenance. The system includes an air-ground-vehicle integrated transmission module for collecting and transmitting sensor data of key train components; a drone cluster module for collecting and transmitting environmental image data within a set range of the railway line; a digital twin big model training module for constructing a digital twin big model based on the sensor data and environmental image data, determining an integrated big model, and using the integrated big model to predict train fault types and natural disasters, and generate emergency guidance plans; the integrated big model is a neural network model. The present application improves the real-time detection and is more suitable for future ultra-high-speed scenarios.

Description

Technical Field

[0001] The present application relates to the field of railway operation and maintenance, and in particular to an intelligent railway operation and maintenance system, method, equipment, medium and product. Background Art

[0002] With the advancement of technology and the growth of economic demand, further increasing the speed of trains has become an important trend in railway development. For example, the new high-speed railway trains currently being developed in China have a speed of nearly 400 kilometers per hour. With the maturity of magnetic levitation technology, the speed of future trains may even exceed 600 kilometers per hour. However, the existing railway operation and maintenance methods are relatively primitive, relying on regular manual inspections of trains, bridges, and areas prone to natural disasters to ensure the safety of train operations. For example, after the high-speed trains finish running during the day, they return to the maintenance base, and workers use ultrasonic detectors to check for cracks in the running gear and bearings at night. Every few hours, workers need to patrol along the railway line to check whether there is rust on the bridges or whether there are floating objects such as advertising banners on the overhead catenary system (OCS).

[0003] Currently, there is a lack of effective early warning measures for natural disasters such as landslides. The only way to reduce disaster losses is to install protective nets on the hillsides. As train speeds continue to increase, these outdated operation and maintenance methods pose a significant risk to high-speed railway safety. Summary of the Invention

[0004] The purpose of this application is to provide an intelligent railway operation and maintenance system, method, equipment, medium and product to solve the problem that manual inspection methods have poor real-time performance and cannot be applied to future ultra-high-speed scenarios.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides an intelligent railway operation and maintenance system, comprising:

[0007] An integrated air-ground-train transmission module, used to collect and transmit sensor data from key train components;

[0008] The drone cluster module is used to collect and transmit environmental image data within the set range of the railway line;

[0009] The digital twin big model training module is used to build a digital twin big model based on the sensor data and environmental image data, determine the integrated big model, and use the integrated big model to predict train failure types and natural disasters and generate emergency guidance plans; the integrated big model is a neural network model.

[0010] In a second aspect, the present application provides an intelligent railway operation and maintenance method, comprising:

[0011] Utilize the air-ground-train integrated transmission module to collect and transmit sensor data from key train components;

[0012] Use the drone cluster module to collect and transmit environmental image data within the set range of the railway line;

[0013] Using the digital twin big model training module, a digital twin big model is constructed based on the sensor data and environmental image data, an integrated big model is determined, and the integrated big model is used to predict train failure types and natural disasters and generate emergency guidance plans; the integrated big model is a neural network model.

[0014] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any one of the above-described intelligent railway operation and maintenance systems.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any one of the above-mentioned intelligent railway operation and maintenance systems.

[0016] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned intelligent railway operation and maintenance systems.

[0017] According to the specific embodiments provided in this application, this application discloses the following technical effects: This application uses an air-ground-vehicle integrated transmission module and a drone cluster module to collect data on the environment around the train and the railway line, namely sensor data and environmental image data, to gain a more comprehensive understanding of the current status of the railway. Subsequently, a digital twin large model is constructed based on the above-mentioned collected data, and an integrated large model is determined to predict the type of train failure and natural disasters, generate emergency guidance plans, and replace manual detection methods with automatic detection methods, thereby improving the real-time detection and making it more suitable for future ultra-high-speed scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1The overall architecture diagram of the intelligent railway operation and maintenance system provided for this application;

[0020] Figure 2 Schematic diagram of TSN switch deployment provided for this application;

[0021] Figure 3 Schematic diagram of the self-organizing mechanism of the drone swarm provided for this application;

[0022] Figure 4 Flowchart of the intelligent railway operation and maintenance method provided in this application;

[0023] Figure 5 This is the transmission process interaction diagram of the air-ground-vehicle integrated transmission module provided by this application;

[0024] Figure 6 Schematic diagram of the large-scale model training and reasoning framework based on federated learning provided for this application;

[0025] Figure 7 Schematic diagram of the computer equipment provided for this application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0028] like Figure 1 As shown, the embodiment of the present application provides an intelligent railway operation and maintenance system, including:

[0029] The air-ground-train integrated transmission module is used to collect and transmit sensor data from key train components.

[0030] The drone cluster module is used to collect and transmit environmental image data within the set range of the railway line.

[0031] The digital twin big model training module is used to build a digital twin big model based on the sensor data and environmental image data, determine the integrated big model, and use the integrated big model to predict train failure types and natural disasters and generate emergency guidance plans; the integrated big model is a neural network model.

[0032] Furthermore, the air-ground-vehicle integrated transmission module and the drone swarm module are considered two independent components, which in turn work in series with the digital twin large model training module. The digital twin large model training module is a generative service based on the large model and digital twin.

[0033] Specifically, the integrated air-ground-train transmission section is responsible for collecting and transmitting sensor data from key train components, including bogies and contact lines.

[0034] The drone swarm module is responsible for collecting and transmitting image data about the railway line and its surrounding environment, except for the case where the relay drone forwards data for the train.

[0035] The digital twin large model training module needs to obtain data from the above two parts to train the large model and provide equipment diagnosis, natural disaster warnings, and generative emergency response plans for train crew members.

[0036] Each module works together to provide high-quality intelligent operation and maintenance services.

[0037] In an exemplary embodiment, the air-ground-vehicle integrated transmission module specifically includes: a wireless terminal deployed on the train, an on-board TSN switch, a 6G base station, and a maintenance base.

[0038] The vehicle-mounted TSN switch is connected to the wireless terminal via an optical fiber and is used to periodically send the collected sensor data to the wireless terminal.

[0039] The wireless terminal is used to send a detection data packet to the maintenance base through the 6G base station based on different transmission modes; the transmission mode includes a normal transmission mode and an emergency transmission mode.

[0040] The maintenance base is used to exchange information with the wireless terminal according to the detection data packet.

[0041] Furthermore, the integrated air-ground-train transmission module consists of wireless terminals deployed on the train, onboard Time Sensitive Network (TSN) switches, relay drones, and various 6G ground networks. To cope with the transmission of massive amounts of data, a transmission mechanism integrating the public 6G network and the railway-specific 6G network (i.e., the 6G-R network, corresponding to 6G-R base stations) is adopted.

[0042] Trains switch between the public 6G network and the 6G-R network based on the urgency of data transmission. Trains in normal conditions have the lowest communication urgency and can tolerate higher latency and packet loss rates. Wireless terminals on these trains send data through the public 6G networks of multiple telecom operators to conserve bandwidth on the 6G-R network.

[0043] For trains that have already experienced a malfunction or are identified as potentially malfunctioning by the large-scale model, the 6G-R base station will take over train-to-ground communications, providing deterministic transmission of sensor data. If the train detects severe channel fading at the currently connected 6G-R base station—that is, when both the 6G-R and 6G network signals are poor—the wireless terminal will transmit the data to a nearby relay drone. The drone will then forward the data via line-of-sight transmission to the currently connected base station or to another base station with better channel conditions.

[0044] The role and connection relationship of the onboard TSN switch are as follows: the 6G-R base station will act as a logical TSN bridge and collaborate with the TSN switches on the maintenance base and train. Figure 2 As shown in the figure, the train is equipped with an onboard TSN switch in each car. These switches are connected to the train wireless terminal via optical fiber and are responsible for regularly collecting sensor data and forwarding it to the wireless terminal. The wireless terminal then sends the data back to the maintenance base through the 6G-R base station. At the intersection of wired and wireless networks, two types of TSN converters are deployed, namely the device-side TSN converter (DS-TT) and the network-side TSN converter (NW-TT). Figure 2 As shown in the upper right corner of the figure, these two converters work together to complete the communication protocol conversion between the IP network and the air interface of the data packet.

[0045] In an exemplary embodiment, the drone cluster module specifically includes: a leader drone, an imaging drone, and a relay drone.

[0046] The leader drone is used to manage and control the entire drone cluster.

[0047] The imaging drone is used to collect point cloud images; the point cloud images are the environmental image data.

[0048] The relay drone is used to forward the sensor data to the 6G base station when the wireless terminal detects that the channel quality of the 6G base station is lower than a channel quality threshold.

[0049] Table 1 is the parameter table of each drone in the drone cluster module, as shown in Table 1.

[0050] Table 1

[0051] Battery Level Cost of Use Computing power Energy consumption Team Leader Drone high expensive outstanding medium Relay drone high medium medium high Imaging drones limited High cost performance limited limited

[0052] Furthermore, imaging drones are equipped with high-resolution cameras or lidars, which are small, low-cost, and numerous to obtain detailed video data.

[0053] Relay drones have higher endurance and are fewer in number. They are mainly responsible for forwarding data from wireless sensors on trains or in mountainous areas to base stations, thereby ensuring the reliability of long-distance communications.

[0054] Each swarm has only one leader drone. It manages the entire swarm and exchanges control signals with the maintenance base via the telecom operator's public 6G network. Equipped with high-performance servers and long flight time, the leader drone oversees operations and ensures effective collaboration within the swarm.

[0055] The self-organizing mechanism of drone swarms and the attributes of various drones such as Figure 3 As shown in the figure, the base station (BS) communicates with the drone cluster in proxy mode. That is, the base station only exchanges control signaling with the cluster's leader drone and exchanges sensor data about the environment or train with relay drones. Other drones do not communicate directly with the base station. The control signaling includes the results of computing tasks completed by the server on the leader drone, a list of subordinate drones in the cluster, and instructions issued by the maintenance base to specific subordinate drones in the cluster. Relay drones regularly collect data from wireless sensors in the mountainous area and send the data back to the maintenance base via the public network or 6G-R base station. In addition, if the maintenance base requests raw data from the imaging drone, the relay drone will also retrieve the data from the imaging drone and forward it.

[0056] In an exemplary embodiment, the digital twin big model training module adopts a big model training and reasoning framework based on federated learning. The big model training and reasoning framework based on federated learning specifically includes: a central supercomputing center and multiple local server rooms located in each maintenance base.

[0057] The local server room is used to build a digital twin model based on the sensor data and environmental image data, and train a local neural network model.

[0058] The central supercomputing center is connected to the local server room and is used to train and aggregate multiple local neural network models to determine an integrated large model, and use the integrated large model to predict train failure types and natural disasters and generate emergency guidance plans.

[0059] Furthermore, these modules provide the maintenance base with a large amount of text, audio, and video data. This data is used to build a digital twin model of the train and mountainous area, enabling early prediction of faults and natural disasters, and providing customized training for train crews on how to handle specific emergency situations.

[0060] Because this data is distributed across multiple geographically dispersed bases, centralizing model training at a single supercomputing center can result in significant communication overhead and privacy risks. To this end, this module employs a large-model training and inference framework based on federated learning. This framework comprises a high-performance central supercomputing center and multiple local server rooms located at each maintenance base. The local server rooms are equipped with standard GPUs for training local neural network models, while the central center is equipped with high-performance, dedicated large-model GPUs for model aggregation. During training, the raw data is used only locally and not sent over the network. Only model parameters, particularly backpropagation gradients, are transmitted over the network between the central and local centers.

[0061] The embodiment of the present application provides an intelligent railway operation and maintenance method, which is executed by a computer device, specifically a computer device such as a terminal or a server, or a terminal and a server. In the embodiment of the present application, Figure 4 As shown, the method includes the following steps.

[0062] S1: Use the air-ground-train integrated transmission module to collect and transmit sensor data of key train components.

[0063] S2: Use the drone cluster module to collect and transmit environmental image data within the set range of the railway line.

[0064] S3: Using the digital twin big model training module, a digital twin big model is constructed based on the sensor data and environmental image data, an integrated big model is determined, and the integrated big model is used to predict train failure types and natural disasters, and generate an emergency guidance plan; the integrated big model is a neural network model.

[0065] In an exemplary embodiment, S1 may be replaced by the following steps.

[0066] S11: Use the on-board TSN switch to periodically send the collected sensor data to the wireless terminal.

[0067] S12: Using the wireless terminal, based on different transmission modes, through the 6G base station, control the train to send a path delay detection message to a maintenance base within a set distance range; the path delay detection message is a detection data packet; the transmission mode includes a normal transmission mode and an emergency transmission mode.

[0068] S13: Filtering a detection base closest to the train based on the round-trip time of information exchange between the detection base and the wireless terminal.

[0069] S14: Control the train to send a train identifier to a detection base closest to the train, and control the detection base to feed back a station identifier to the train.

[0070] S15: Based on the site identifier and the channel quality of the 6G base station, the sensor data of the key components of the train are transmitted to the detection base; if the channel quality of the 6G base station is lower than the channel quality threshold, the sensor data is forwarded to the 6G base station by using a relay drone, and then the 6G base station transmits the sensor data of the key components of the train to the detection base.

[0071] In practical applications, S1 can also be replaced by the following steps.

[0072] Step 1. The TSN switch on the train regularly sends the collected sensor data to the wireless terminal.

[0073] Step 2. The train sends a path delay detection message to multiple nearby maintenance bases.

[0074] Furthermore, the wireless terminal on the train sends a probe data packet without specific content to a nearby maintenance base through the public 6G network, which together with step 3 completes the total delay measurement of the data packet from the train to the maintenance base and then back to the train, namely the round-trip time (RTT).

[0075] The train will select the maintenance base with the shortest round-trip delay as the receiving unit for the collected sensor data, because this will minimize the data transmission delay. The sensor data will be periodically sent to the maintenance base regardless of whether there is a problem with the train. The difference is that when there is no problem with the train, the sensor data is sent to the maintenance base. Figure 5 The normal transmission mode is used when the problem occurs. Figure 5 Emergency transfer mode.

[0076] Step 3. After receiving the detection message, the maintenance base immediately replies to the train. The train then obtains the round-trip time (RTT) of different maintenance stations.

[0077] Step 4. The train selects an appropriate maintenance station based on the RTT to send the sensor data back.

[0078] Step 5. Before the transmission begins, the train sends a train identifier to the maintenance base to identify the data source and provide a reference for subsequent large model training.

[0079] Furthermore, the computer equipment at the front of the train will store the train identifier of the train. When sending sensor data, the train identifier will be added to the front of the sensor data. The maintenance base uses the train identifier to determine which train the data it receives belongs to, laying a foundation for subsequent training of the neural network model.

[0080] Step 6. The maintenance base sends its station identifier back to the train, and the train maintains a list to record the stations that sent data at different times.

[0081] Furthermore, the train's computer records the station identifiers on its hard drive, storing them in a data table. Because the minimum RTT required for communication with the train varies at different times, sensor data is sent to different maintenance bases at different times, documenting the data transmission path over time.

[0082] Step 7. The wireless terminal detects the channel quality of the 6G networks of different telecom operators.

[0083] Furthermore, the wireless terminal sends channel detection requests to 6G base stations of different operators, and then different 6G base stations will send wireless broadcasts without actual content to the train. The wireless terminal determines the quality of the 6G network channel by detecting the signal strength of different base stations at the receiving antenna.

[0084] Step 8. The train decides the amount of 6G network to use based on the type of sensor data and channel quality, and determines the corresponding methods for generating and eliminating information redundancy.

[0085] Furthermore, the sensor data type is first checked. If it's vehicle surveillance video, it has a lower priority and is sent through a single 6G base station. If it's tracking axle temperature, it has a higher priority and, depending on the current channel quality, two or three 6G base stations are used to send the data. If the channel quality is good, information redundancy generation technology is not used. If it's poor, network coding technology is used to increase the number of data packets to improve data redundancy. This way, even if not all packets are received, the decoder can still recover the complete information from partial packets, significantly reducing the probability of retransmissions. The maintenance base can eliminate redundancy by simply implementing the appropriate decoding technology.

[0086] Step 9. The train sends data to the maintenance station via multiple paths.

[0087] Step 10. If the train does not encounter an emergency such as a breakdown or natural disaster, repeat Step 7 to Step 10.

[0088] Furthermore, the digital twin large model training module determines whether a fault or natural disaster has occurred based on the sensor data sent back by the train wireless terminal.

[0089] Step 11: The train sends a message to the maintenance station to report the emergency situation. The maintenance station can also proactively notify the train of potential equipment failures based on the predictions from the large model. After this step, the train enters emergency communication mode.

[0090] Step 12. The wireless terminal detects the channel quality of the 6G-R base station and decides whether to obtain communication support from the relay drone. For trains located in mountainous areas or with other obstacles, data can be first sent to the drone, which then forwards it to a nearby 6G-R base station.

[0091] Step 13. The 6GR base station allocates sufficient spectrum resources to the train based on transmission requirements. Within the 6G-R base station, slicing technology is used to isolate train control services from diagnostic services to prevent mutual interference. Within each slice, appropriate spectrum resources are allocated to the train based on data volume and latency requirements.

[0092] At this stage, this application introduces an algorithm based on deep reinforcement learning to improve decision-making efficiency and accuracy. Figure 2 As shown in the lower left corner of the figure, the proposed algorithm jointly allocates TSN time slots and spectrum for 6G-R base stations. Micro-time slots are used as scheduling units for 6G-R base stations to match the granularity of TSN time slots. Wireless terminals and 6G-R base stations can forward packets within a single time slot or utilize the remaining time within two time slots. Based on the selected time plan, appropriate spectrum is allocated to the train.

[0093] Step 14. The train sends the data back to the maintenance station via the 6G-R base station.

[0094] Step 15. After the emergency situation is resolved, the train returns to normal transmission mode and repeats Step 7-Step 10.

[0095] In an exemplary embodiment, S2 may be replaced by the following steps.

[0096] S21: Under the control of the team leader drone, multiple imaging drones are used to scan the area within the set range of the railway line to collect environmental image data; the area includes bridges, railway lines and areas prone to natural disasters.

[0097] S22: Control the imaging drone to perform a recognition task based on the environmental image data, generate a recognition result, and send the recognition result to the team leader drone.

[0098] S23: Control the team leader drone to notify the maintenance base or staff to handle the abnormal situation based on the recognition result, and generate a task offloading decision based on the environmental image data and complex tasks; the complex tasks include splicing point cloud image tasks, path planning tasks, and dynamic routing tasks.

[0099] S24: Using a multi-agent reinforcement learning algorithm, plan the flight paths of each category of drones based on the task offloading decision.

[0100] S25: Based on the flight path, a distributed reinforcement learning routing algorithm is used to determine the optimal transmission path according to multiple system factors, and the imaging drone is controlled to transmit environmental image data according to the optimal transmission path; the multiple system factors include channel status, transmission delay, and link throughput.

[0101] In practical applications, S2 can also be replaced by the following steps.

[0102] The drone cluster module for environmental modeling and automatic inspection transmits the monitoring information of infrastructure along the railway and the sensor monitoring information of the surrounding mountainous areas to the digital twin large model training module, such as Figure 3 shown.

[0103] Sub-step 1: Under the control of a team leader drone, multiple imaging drones work together to scan bridges, railway lines, and areas prone to natural disasters using high-resolution cameras or lidar.

[0104] Sub-step 2: For identification tasks, such as identifying personnel intruding on railway lines or objects hanging from the overhead contact system (OCS), the imaging drone completes the task locally and transmits the results to the leader drone. The leader drone then notifies the maintenance base or personnel to address the anomaly. For complex tasks such as laser point cloud image stitching, the leader drone determines the appropriate calculation location.

[0105] Sub-step 3: The leader drone offloads complex tasks, such as laser point cloud image stitching, path planning, and dynamic routing, to appropriate computational locations. The leader drone uses deep reinforcement learning to make task offloading decisions based on constraints such as task latency requirements, computational complexity, and the drone's remaining energy.

[0106] Sub-step 4: Based on the task offloading results from Sub-step 3, the leader drone begins planning the flight paths for each drone. Specifically, the leader drone and other imaging and relay drones jointly determine the optimal path using a multi-agent reinforcement learning algorithm, such as the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. If the trajectory planning task is too complex and beyond the capabilities of a small drone (such as an imaging drone), the leader drone can also use a single-agent algorithm, such as the Deep Deterministic Policy Gradient (DDPG) algorithm, to centrally plan the flight paths for the entire swarm.

[0107] Sub-step 5: The UAV cluster adopts a routing algorithm based on distributed reinforcement learning to adaptively obtain the optimized transmission path based on factors such as channel status, transmission delay, and link throughput.

[0108] Sub-step 6: Due to cost constraints, the scanning range of a single imaging drone is limited, and the point cloud density of its laser scans is insufficient, making it difficult to create a satisfactory digital twin model of the mountain area. To address this limitation, multiple imaging drones are used to perform collaborative imaging by overlapping and scanning key areas. The imaging drones then transmit the point cloud images to the leader drone or maintenance base via the transmission path determined in Sub-step 5. The leader drone or maintenance base uses a pre-trained generative adversarial network (GAN) to seamlessly stitch the multiple point cloud images from the multiple imaging drones and intelligently repair blurred areas.

[0109] In an exemplary embodiment, S3 may be replaced by the following steps.

[0110] In order to provide various generative services based on digital twins, it is necessary to train multiple large models, such as the running gear temperature prediction model and the flood prediction model. These large models differ in the number of parameters, data volume, and computational complexity. Therefore, in the large model training module based on digital twins, this application adopts a training and reasoning framework based on federated learning (FL), such as Figure 6 shown.

[0111] Sub-step 1: First, select an instance for implementation based on the requirements of a specific large model, such as a train axle failure warning large model and a fire emergency guidance large model.

[0112] This example contains the basic options for large-scale model training, namely {purpose, algorithm, training mode, interaction mode...}.

[0113] For example, in running gear temperature prediction, the required computational complexity is relatively low, so an implementation example includes {prediction, FedProx, centralized, synchronous interaction}. Here, {prediction} refers to the use of the model for diagnostic services, {FedProx} represents the algorithm for model aggregation, and {synchronous interaction} means that all local models are updated only once per iteration. In this application, all FL-based model training follows the centralized {training mode}.

[0114] Sub-step 2: Once appropriate instances are selected, training of the large model begins accordingly. The central supercomputing center receives model gradients from the local server room and uses a related algorithm to calculate the aggregated gradient. Commonly used algorithms include the Federated Proximal Algorithm (FedProx) and the Federated Averaging (FedAvg).

[0115] Sub-step 3: These aggregated gradients are then sent back to the local computer room for model parameter update. Sub-steps 2 and 3 are repeated until the model reaches the desired accuracy.

[0116] Sub-step 4: After training is complete, the large model is deployed for inference at the dispatch center. To reduce network load and avoid disrupting the train control system, parameter updates during the inference phase are typically performed at night. Only critical model vulnerabilities are updated during the day.

[0117] The following three items are the working methods used in sub-steps 2 and 3 to reduce communication overhead, improve training speed, and improve computing resource utilization. They are not separate steps.

[0118] Traffic compression: This technology is used to reduce the overhead of a single communication. In the model aggregation based on the Federated Learning (FL) framework, both the local server room and the central supercomputing center use quantization encoding to transmit gradients. The sender approximates the back-propagated gradient as a 16-bit or even 8-bit floating point number. At the receiving end, an error compensation mechanism is applied to mitigate the impact of gradient quantization on model convergence. In addition, the central supercomputing center evaluates the importance of different elements in the gradient to model training and identifies key elements. The indices of these important elements are then sent to the local computer room. In the subsequent training process, the local computer room and the central center only exchange the gradients of these key elements and ignore other elements. Figure 6 As shown in Figure 3, this traffic compression does not reduce the total throughput, but rather smoothes the traffic curve by reducing the peak rate, thus playing the role of traffic shaping.

[0119] Communication and computation overlap: Although traffic compression reduces the overhead of a single communication, it also leads to an increase in the number of training iterations. To this end, this application adopts a method of communication and computation overlap to speed up training. This overlap allows the transmission of partial calculation results before the model parameter calculation is completed. Specifically, the back propagation of gradients is performed in layers. When the gradient calculation of the last layer is completed, the local server room can initiate communication with the central supercomputing center. While sending the gradient of the last layer to the central supercomputing center, the local server room continues to perform the back propagation gradient calculation of the previous layer. By performing calculations and communications in parallel, the overall training time is significantly reduced.

[0120] Parameter update: The overlap of communication and computation leads to another problem, that is, local computer rooms with higher computing power can send gradients to the central supercomputing center faster, but they must wait for the slowest local center to send its gradients before they can proceed to the next round of model updates, resulting in a waste of computing resources. To solve this problem, this application adopts different interaction modes according to the parameter size of the model. For models that require high training accuracy or have fewer parameters, a synchronous interaction mode is used to ensure that all local computer rooms update their parameters at the same time after each iteration. However, for large models with more parameters, a quasi-synchronous interaction mode is used to allow faster local computer rooms to perform multiple parameter updates in a single training iteration. In addition, if Figure 6 As shown in Figure 2, to prevent these fast local data centers from updating too frequently and negatively affecting the model accuracy, the maximum number of updates allowed in each iteration is set to 3.

[0121] Publication number CN118840846A, titled "A Method and Storage Medium for Real-Time Warning of Abnormal Long-Distance Highway Damage," implements a method for providing real-time warning of abnormal long-distance highway damage by embedding optical fibers in the road section to be tested and installing a receiving end every n kilometers, connected to a distributed acoustic sensing system (DAS). During data collection by the DAS equipment, each monitoring unit automatically emits optical pulses at regular intervals, collecting raw data on road vibration. After preprocessing the raw vibration data, the data from multiple monitoring units is spliced ​​based on distance to generate a matrix image of the vibration signals of the entire highway over a period of time. Using an algorithm, the resulting vibration signal matrix image is processed and classified to determine the different vibration types. These vibrations include those generated by normal vehicle driving, construction, landslides, and abnormal traffic accidents. The DAS cloud platform compares newly collected vibration data with previous data to provide real-time warnings of road vibrations, ensuring highway driving safety.

[0122] Although this method has been applied to highways to a certain extent, the use of fiber optic early warning methods in the railway field has many problems: (1) Railway lines are long and usually far away from urban areas, and the cost of daily maintenance is high. Using fiber optics to lay the entire line is not economically feasible. (2) It does not contain a wireless communication module and lacks flexibility. For railway lines in extreme environments, wireless monitoring technologies such as drones are needed to improve monitoring efficiency. (3) Fiber optic and vibration sensing technologies can only monitor the environment along the line and cannot detect and warn vehicles traveling on the road.

[0123] In response to the above issues:

[0124] 1) This application uses wireless means instead of optical fiber, such as drones, public 5 / 6G networks, etc.

[0125] 2) This application has a wireless communication module and a flexible working mode.

[0126] 3) The air-ground-train integrated transmission part of the present application can effectively detect faults in the train itself.

[0127] Publication number CN105844995A, titled "A Railway Line Operation and Maintenance Measurement Method Based on Vehicle-Mounted LiDAR Technology," utilizes a LiDAR system installed on trains, replacing traditional surveyors to automatically inspect the stability of railway lines and their associated structures. Based on laser point cloud technology, this technology establishes a basic control network and a target control network on the ground to complete control network measurement and adjustment. The resulting point cloud is then segmented and constrained to further improve data accuracy. Finally, the constrained data is classified to obtain line element information such as rail horizontal and vertical section parameters.

[0128] However, this method has the following shortcomings in use: (1) The laser point cloud technology used in this technology realizes the automated detection of railway lines, but it still lacks the ability to automatically monitor the environment along the railway, especially the mountainous environment. There are still blind spots in monitoring safety risk factors. (2) This technology has proposed the deployment of additional detection devices on trains, but it cannot provide any data collection function for the operation and maintenance of the train itself. It is entirely possible to consider using the detection devices to monitor key components such as pantographs and axles of trains at the same time.

[0129] In response to the above issues:

[0130] 1) This application uses laser and high-resolution photography of the railway surroundings, especially in mountainous areas, to solve the problem of blind spots in monitoring.

[0131] 2) Many sensors are deployed on the train to detect the working conditions of the train's pantograph and axles.

[0132] This application achieves timely early warning of potential hazards by establishing a digital twin model of key equipment and the environment along the line.

[0133] This application has the following unique features:

[0134] 1) Existing deterministic transmission research only focuses on wireless access, but TSN switches are applied to trains and the integration of TSN switches and 6G base stations is studied to achieve end-to-end deterministic transmission in heterogeneous networks.

[0135] 2) In order to reduce costs while completing multiple functions, drone swarms include multiple types of drones with different capacities and costs, and a learning-based self-organizing mechanism is proposed to promote collaboration among them.

[0136] 3) According to a search on the Patent Office website, this application is one of the earliest to propose the application of large models and generative AI to high-speed railway equipment diagnosis and natural disaster prediction. Currently, there are few published patents related to railway digital twins and large models. From this perspective, it is difficult to find other alternative solutions to achieve this invention's purpose, but other solutions may be proposed during the implementation process.

[0137] This application states that in the future ultra-high-speed operation scenario, railway operation and maintenance will be more intelligent and real-time, providing further safety guarantees for trains. First, by establishing an integrated air-ground-vehicle transmission network, trains will be able to achieve real-time communication of big data with maintenance bases. Compared with the current GSM-R network that can only transmit voice and text, the amount of data that can be transmitted per unit time on the new network has been significantly increased. Secondly, by introducing automated inspection technologies such as drones, the level of detection of railway lines and surrounding environments can be greatly improved. Compared with traditional manual inspections, automated equipment has better performance in terms of inspection frequency, coverage and monitoring accuracy. Third, large model technology can make full use of the large amount of data currently accumulated by railway operation and maintenance to achieve early warning of potential faults, and provide customized disposal solutions through generative technology, which will effectively help train staff improve their emergency response capabilities.

[0138] In an exemplary embodiment, a computer device is provided, such as Figure 7 As shown, the computer device can be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store intelligent railway operation and maintenance data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an intelligent railway operation and maintenance method is implemented.

[0139] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above method when executing the computer program.

[0140] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above method when executed by a processor.

[0141] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the above method when executed by a processor.

[0142] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0143] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0144] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0145] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] This application uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An intelligent railway operation and maintenance system, characterized in that: The intelligent railway operation and maintenance system includes: An integrated air-ground-train transmission module, used to collect and transmit sensor data from key train components; the module specifically includes a wireless terminal deployed on the train, an onboard TSN switch, a 6G base station, and a maintenance base; The vehicle-mounted TSN switch is connected to the wireless terminal via an optical fiber and is used to periodically send the collected sensor data to the wireless terminal; The wireless terminal is configured to send a detection data packet to the maintenance base via the 6G base station based on different transmission modes; the transmission modes include a normal transmission mode and an emergency transmission mode; The maintenance base is configured to exchange information with the wireless terminal according to the detection data packet; A drone cluster module is used to collect and transmit environmental image data within a set range of the railway line; the drone cluster module specifically includes: a leader drone, an imaging drone, and a relay drone; The leader drone is used to manage and control the entire drone cluster; The imaging drone is used to collect point cloud images; the point cloud images are the environmental image data; The relay drone is configured to forward the sensor data to the 6G base station when the wireless terminal detects that the channel quality of the 6G base station is lower than a channel quality threshold; The digital twin big model training module is used to build a digital twin big model based on the sensor data and environmental image data, determine the integrated big model, and use the integrated big model to predict train failure types and natural disasters and generate emergency guidance plans; the integrated big model is a neural network model; the digital twin big model training module adopts a big model training and inference framework based on federated learning.

2. The intelligent railway operation and maintenance system according to claim 1, characterized in that: The digital twin large model training module adopts a large model training and reasoning framework based on federated learning. The large model training and reasoning framework based on federated learning specifically includes: a central supercomputing center and multiple local server rooms located in each maintenance base; The local server room is used to build a digital twin model based on the sensor data and environmental image data, and train a local neural network model; The central supercomputing center is connected to the local server room and is used to train and aggregate multiple local neural network models to determine an integrated large model, and use the integrated large model to predict train failure types and natural disasters and generate emergency guidance plans.

3. An intelligent railway operation and maintenance method, characterized in that: The intelligent railway operation and maintenance method is applied to the intelligent railway operation and maintenance system according to any one of claims 1 to 2, and the intelligent railway operation and maintenance method includes: Utilize the air-ground-train integrated transmission module to collect and transmit sensor data from key train components; Use the drone cluster module to collect and transmit environmental image data within the set range of the railway line; Using the digital twin big model training module, a digital twin big model is constructed based on the sensor data and environmental image data, an integrated big model is determined, and the integrated big model is used to predict train failure types and natural disasters and generate emergency guidance plans; the integrated big model is a neural network model.

4. The intelligent railway operation and maintenance method according to claim 3, characterized in that: The air-ground-train integrated transmission module collects and transmits sensor data from key train components, including: Using the on-board TSN switch, the collected sensor data is periodically sent to the wireless terminal; Using the wireless terminal, based on different transmission modes, the train is controlled to send a path delay detection message to a maintenance base within a set distance through a 6G base station; the path delay detection message is a detection data packet; the transmission mode includes a normal transmission mode and an emergency transmission mode; Selecting a maintenance base closest to the train based on the round-trip time of information exchange between the maintenance base and the wireless terminal; Controlling the train to send a train identifier to a maintenance base closest to the train, and controlling the maintenance base to feed back a station identifier to the train; Based on the site identifier and according to the channel quality of the 6G base station, the sensor data of the key components of the train are transmitted to the maintenance base; if the channel quality of the 6G base station is lower than the channel quality threshold, the sensor data is forwarded to the 6G base station by using a relay drone, and then the 6G base station transmits the sensor data of the key components of the train to the maintenance base.

5. The intelligent railway operation and maintenance method according to claim 3, characterized in that: The drone cluster module is used to collect and transmit environmental image data within the set range of the railway line, including: Under the control of the team leader drone, multiple imaging drones are used to scan areas within a set range of the railway line to collect environmental image data; the areas include bridges, railway lines, and areas prone to natural disasters; Controlling the imaging drone to perform a recognition task based on the environmental image data, generating a recognition result, and sending the recognition result to the team leader drone; Controlling the team leader drone to notify the maintenance base or staff to handle abnormal situations based on the recognition results, and generating task offloading decisions based on complex tasks, including point cloud image stitching tasks, path planning tasks, and dynamic routing tasks, based on the environmental image data; A multi-agent reinforcement learning algorithm is used to plan the flight paths of each category of drones based on the task offloading decisions. Based on the flight path, a distributed reinforcement learning routing algorithm is used to determine the optimal transmission path according to multiple system factors, and the imaging drone is controlled to transmit environmental image data along the optimal transmission path; the multiple system factors include channel status, transmission delay, and link throughput.

6. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intelligent railway operation and maintenance system according to any one of claims 3 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the intelligent railway operation and maintenance system according to any one of claims 3 to 5 is implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the intelligent railway operation and maintenance system according to any one of claims 3 to 5 is implemented.

Images