Intelligent railway operation and maintenance system, method, equipment, medium and product

Abstract

The invention discloses an intelligent railway operation and maintenance system, method, equipment, medium and product, and relates to the field of railway operation and maintenance, and the system comprises an air-ground-train integrated transmission module which is used for collecting and transmitting sensor data of train key components; the unmanned aerial vehicle cluster module is used for collecting and transmitting environment image data in a set range of a railway line; the digital twinning large model training module is used for constructing a digital twinning large model according to the sensor data and the environment image data, determining an integrated large model, predicting train fault types and natural disasters by using the integrated large model, and generating an emergency guidance scheme; the integrated large model is a neural network model, the detection real-time performance is improved, and the method is more suitable for the ultra-high-speed scene in the future.

Description

Technical Field

[0001] The present application relates to the field of railway operation and maintenance, and in particular to an intelligent railway operation and maintenance system, method, equipment, medium and product. Background Art

[0002] With the advancement of technology and the growth of economic demand, further improving the speed of trains has become an important trend in railway development. For example, the new high-speed railway trains currently being developed in China have a speed of nearly 400 km / h. With the maturity of magnetic levitation technology, the speed of future trains may even exceed 600 km / h. However, the existing railway operation and maintenance methods are relatively primitive, relying on regular manual inspections of trains, bridges, and areas prone to natural disasters to ensure the safety of train operations. For example, after the high-speed trains finish running during the day, they return to the maintenance base, and workers use ultrasonic detectors to check for cracks in the running gear and bearings at night. Every few hours, workers need to patrol along the railway line to check whether the bridges are rusted or whether there are floating objects such as advertising banners on the overhead catenary system (OCS) that have invaded.

[0003] Currently, there is a lack of effective early warning measures for natural disasters such as landslides, and the only way to reduce disaster losses is to set up protective nets on the hillsides. As train speeds continue to increase, these outdated operation and maintenance methods pose a major risk to the safety of high-speed railways. Summary of the invention

[0004] The purpose of this application is to provide an intelligent railway operation and maintenance system, method, equipment, medium and product to solve the problem that the manual inspection method has poor real-time performance and cannot be applied to future ultra-high-speed scenarios.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides an intelligent railway operation and maintenance system, comprising:

[0007] Air-ground-train integrated transmission module, used to collect and transmit sensor data of key train components;

[0008] The drone cluster module is used to collect and transmit environmental image data within the set range of the railway line;

[0009] The digital twin big model training module is used to build a digital twin big model based on the sensor data and environmental image data, determine the integrated big model, and use the integrated big model to predict train failure types and natural disasters, and generate emergency guidance plans; the integrated big model is a neural network model.

[0010] In a second aspect, the present application provides an intelligent railway operation and maintenance method, comprising:

[0011] Use the air-ground-train integrated transmission module to collect and transmit sensor data of key train components;

[0012] Use the drone cluster module to collect and transmit environmental image data within the set range of the railway line;

[0013] Using the digital twin big model training module, a digital twin big model is constructed according to the sensor data and environmental image data, an integrated big model is determined, and the integrated big model is used to predict train failure types and natural disasters, and generate emergency guidance plans; the integrated big model is a neural network model.

[0014] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any one of the above-described intelligent railway operation and maintenance systems.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described intelligent railway operation and maintenance systems.

[0016] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned intelligent railway operation and maintenance systems.

[0017] According to the specific embodiments provided in the present application, the present application discloses the following technical effects: the present application adopts an air-ground-vehicle integrated transmission module and a drone cluster module to collect data on the environment around the train and the railway line, namely sensor data and environmental image data, to have a more comprehensive understanding of the current status of the railway. Then, a digital twin large model is constructed based on the above-mentioned collected data, and an integrated large model is determined to predict the type of train failures and natural disasters, and generate emergency guidance plans. By replacing manual detection methods with automatic detection methods, the real-time detection is improved and it is more suitable for future ultra-high-speed scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1The overall architecture diagram of the intelligent railway operation and maintenance system provided for this application;

[0020] Figure 2 A schematic diagram of the TSN switch deployment provided for this application;

[0021] Figure 3 Schematic diagram of the self-organizing mechanism of the drone swarm provided for this application;

[0022] Figure 4 Flow chart of the intelligent railway operation and maintenance method provided in this application;

[0023] Figure 5 An interactive diagram of the transmission process of the air-ground-vehicle integrated transmission module provided in this application;

[0024] Figure 6 Schematic diagram of the large model training and reasoning framework based on federated learning provided for this application;

[0025] Figure 7 Schematic diagram of the computer equipment provided for this application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0028] like Figure 1 As shown, the embodiment of the present application provides an intelligent railway operation and maintenance system, including:

[0029] The air-ground-train integrated transmission module is used to collect and transmit sensor data from key train components.

[0030] The drone cluster module is used to collect and transmit environmental image data within the set range of the railway line.

[0031] The digital twin big model training module is used to build a digital twin big model based on the sensor data and environmental image data, determine the integrated big model, and use the integrated big model to predict train failure types and natural disasters, and generate emergency guidance plans; the integrated big model is a neural network model.

[0032] Furthermore, the air-ground-vehicle integrated transmission module and the drone cluster module are considered as two independent parts, and these two modules cooperate with the digital twin big model training module in a serial mode. Among them, the digital twin big model training module is a generative business based on the big model and digital twin.

[0033] Specifically, the air-ground-train integrated transmission part is responsible for collecting and transmitting sensor data from key components of the train, including bogies and contact network systems.

[0034] The drone swarm module is responsible for collecting and transmitting image data about the railway line and its surroundings, except for the case where the relay drone forwards data for the train.

[0035] The digital twin big model training module needs to obtain data from the above two parts to train the big model and provide equipment diagnosis, early warning of natural disasters, and generative emergency command plans for train crew members.

[0036] Each module works together to provide high-quality intelligent operation and maintenance services.

[0037] In an exemplary embodiment, the air-ground-vehicle integrated transmission module specifically includes: a wireless terminal deployed on a train, an on-board TSN switch, a 6G base station, and a maintenance base.

[0038] The vehicle-mounted TSN switch is connected to the wireless terminal via an optical fiber, and is used to periodically send the collected sensor data to the wireless terminal.

[0039] The wireless terminal is used to send a detection data packet to the maintenance base through the 6G base station based on different transmission modes; the transmission mode includes a normal transmission mode and an emergency transmission mode.

[0040] The maintenance base is used to exchange information with the wireless terminal according to the detection data packet.

[0041] Furthermore, the air-ground-train integrated transmission module consists of wireless terminals deployed on the train, on-board time-sensitive network (TSN) switches, relay drones, and various 6G ground networks. In order to cope with the transmission of massive data, a mechanism of integrated transmission of public 6G network and railway-specific 6G network (i.e. 6G-R network, corresponding to 6G-R base station) is adopted.

[0042] The train switches between the public 6G network and the 6G-R network according to the urgency of data transmission. Trains in a normal state have the lowest communication urgency and can tolerate higher delays and packet loss rates. The wireless terminals on these trains send data through the public 6G networks of multiple telecom operators to save bandwidth on the 6G-R network.

[0043] For trains that have already failed or are judged by the large model to be likely to fail, the 6G-R base station will take over the train-to-ground communication and provide deterministic transmission for sensor data. When the train detects that the channel of the currently connected 6G-R base station is severely attenuated, that is, when the signals of the 6G-R network and the 6G network are both poor, the wireless terminal will send the data to a nearby relay drone, which will forward the data to the currently connected base station through line-of-sight transmission, or to other base stations with good channel conditions.

[0044] The function and connection relationship of the on-board TSN switch are as follows: the 6G-R base station will act as a logical TSN bridge and collaborate with the TSN switches on the maintenance base and train. Figure 2 As shown, the train is equipped with on-board TSN switches in each car. These switches are connected to the train wireless terminal via optical fiber and are responsible for periodically collecting sensor data and forwarding it to the wireless terminal. The wireless terminal then sends the data back to the maintenance base through the 6G-R base station. At the intersection of wired and wireless networks, two types of TSN converters are deployed, namely the device-side TSN converter (DS-TT) and the network-side TSN converter (NW-TT). Figure 2 As shown in the upper right corner of , these two converters work together to complete the communication protocol conversion of data packets between the IP network and the air interface.

[0045] In an exemplary embodiment, the drone cluster module specifically includes: a leader drone, an imaging drone, and a relay drone.

[0046] The leader drone is used to manage and control the entire drone cluster.

[0047] The imaging drone is used to collect point cloud images; the point cloud images are the environmental image data.

[0048] The relay drone is used to forward the sensor data to the 6G base station when the wireless terminal detects that the channel quality of the 6G base station is lower than a channel quality threshold.

[0049] Table 1 is a table of parameters of each drone in the drone cluster module, as shown in Table 1.

[0050] Table 1

[0051] Battery Level Cost Computing power Energy consumption Team leader drone high expensive outstanding medium Relay drone high medium medium high Imaging drones limited High cost performance limited limited

[0052] Furthermore, imaging drones are equipped with high-resolution cameras or lidars, which are small, low-cost, and numerous to obtain detailed video data.

[0053] Relay drones have higher endurance and are fewer in number. They are mainly responsible for forwarding data from wireless sensors on trains or in mountainous areas to base stations, thereby ensuring the reliability of long-distance communications.

[0054] In each swarm, there is only one leader drone. It is responsible for managing the entire drone swarm and interacting with the maintenance base through the public 6G network of the telecom operator for control signaling. The leader drone is equipped with a high-performance server and has a long flight time, which can supervise operations and ensure effective collaboration within the drone swarm.

[0055] The self-organizing mechanism of drone swarms and the attributes of various drones such as Figure 3 As shown. The base station (BS) communicates with the UAV cluster in proxy mode, that is, the base station only exchanges control signaling with the leader UAV of the cluster, and exchanges sensor data of the environment or train with the relay UAV. Other UAVs do not communicate directly with the base station. The content of the control signaling includes the results obtained by the server on the leader UAV to complete the computing task, the list of subordinate UAVs in the cluster, and the instructions issued by the maintenance base to specific subordinate UAVs in the cluster. The relay UAV regularly collects data from wireless sensors in the mountainous area and sends the data back to the maintenance base through the public network or 6G-R base station. In addition, if the maintenance base requests to obtain the raw data of the imaging UAV, the relay UAV will also retrieve the data from the imaging UAV and forward it.

[0056] In an exemplary embodiment, the digital twin big model training module adopts a big model training and reasoning framework based on federated learning. The big model training and reasoning framework based on federated learning specifically includes: a central supercomputing center and multiple local server rooms located in each maintenance base.

[0057] The local server room is used to build a digital twin model based on the sensor data and environmental image data, and train a local neural network model.

[0058] The central supercomputing center is connected to the local server room, and is used to train and aggregate multiple local neural network models to determine an integrated large model, and use the integrated large model to predict train failure types and natural disasters, and generate emergency guidance plans.

[0059] Furthermore, the above modules provide a large amount of text, audio and video data to the maintenance base. These data are used to build digital twin models for trains and mountainous areas, thereby achieving early prediction of faults and natural disasters, or providing customized tutorials for train crews to handle specific emergency situations.

[0060] Since this data is distributed in multiple geographically dispersed bases, concentrating it on a single supercomputing center for model training may result in significant communication overhead and privacy risks. To this end, this module adopts a large model training and inference framework based on federated learning. The framework includes a high-performance central supercomputing center and multiple local server rooms located in each maintenance base. The local server rooms are equipped with ordinary GPUs for training local neural network models, while the central center is equipped with high-performance dedicated large-model GPUs for model aggregation. During the training process, the original data is only used locally and not sent on the network. Only the model parameters, especially the back-propagation gradients, are transmitted between the central and local centers through the network.

[0061] The embodiment of the present application provides an intelligent railway operation and maintenance method, which is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, Figure 4 As shown, the method includes the following steps.

[0062] S1: Use the air-ground-train integrated transmission module to collect and transmit sensor data of key train components.

[0063] S2: Use the drone cluster module to collect and transmit environmental image data within the set range of the railway line.

[0064] S3: Using the digital twin big model training module, construct a digital twin big model according to the sensor data and environmental image data, determine the integrated big model, and use the integrated big model to predict train failure types and natural disasters, and generate emergency guidance plans; the integrated big model is a neural network model.

[0065] In an exemplary embodiment, S1 may be replaced by the following steps.

[0066] S11: Use the vehicle-mounted TSN switch to periodically send the collected sensor data to the wireless terminal.

[0067] S12: Using the wireless terminal, based on different transmission modes, through the 6G base station, control the train to send a path delay detection message to a maintenance base within a set distance range; the path delay detection message is a detection data packet; the transmission mode includes a normal transmission mode and an emergency transmission mode.

[0068] S13: Filtering a detection base closest to the train according to the round-trip time of information exchange between the detection base and the wireless terminal.

[0069] S14: Control the train to send a train identifier to a detection base closest to the train, and control the detection base to feed back a station identifier to the train.

[0070] S15: Based on the site identifier, according to the channel quality of the 6G base station, the sensor data of the key components of the train are transmitted to the detection base; if the channel quality of the 6G base station is lower than the channel quality threshold, the sensor data is forwarded to the 6G base station by a relay drone, and then the 6G base station transmits the sensor data of the key components of the train to the detection base.

[0071] In practical applications, S1 can also be replaced by the following steps.

[0072] Step 1. The TSN switch on the train periodically sends the collected sensor data to the wireless terminal.

[0073] Step 2. The train sends a path delay detection message to multiple nearby maintenance bases.

[0074] Furthermore, the wireless terminal on the train sends a detection data packet without specific content to a nearby maintenance base through the public 6G network, and together with step 3, completes the total delay measurement of the data packet from the train to the maintenance base and then back to the train, that is, the round-trip time (RTT).

[0075] The train will select the maintenance base with the shortest round-trip delay as the receiving unit for the collected sensor data, because this will minimize the data transmission delay. The sensor data will be sent to the maintenance base periodically regardless of whether there is a problem with the train. The difference is that when there is no problem with the train, the sensor data is sent to the maintenance base periodically. Figure 5 The normal transmission mode is used when the problem occurs. Figure 5 Emergency transfer mode.

[0076] Step 3. After receiving the detection message, the maintenance base immediately replies to the train. The train then obtains the round trip time (RTT) of different maintenance stations.

[0077] Step 4. The train selects an appropriate maintenance station based on the RTT to send the sensor data back.

[0078] Step 5. Before the transmission begins, the train sends a train identifier to the maintenance base to identify the data source and provide a reference for subsequent large model training.

[0079] Furthermore, the computer equipment at the front of the train will store the train identifier of the train. When sending sensor data, the train identifier will be added to the front of the sensor data. The maintenance base uses the train identifier to determine which train the data it receives belongs to, laying a foundation for the subsequent training of the neural network model.

[0080] Step 6. The maintenance base sends its station identifier back to the train, and the train maintains a list to record the stations that sent data at different times.

[0081] Furthermore, the computer equipment at the front of the train records the site identifier into the computer hard disk and saves it into a data table. Because the minimum RTT maintenance base for communicating with the train is different in different time periods, the sensor data is sent to different maintenance bases at different times to record the data transmission direction at different times.

[0082] Step 7. The wireless terminal detects the channel quality of 6G networks of different telecom operators.

[0083] Furthermore, the wireless terminal sends channel detection requests to 6G base stations of different operators, and then different 6G base stations will send wireless broadcasts without actual content to the train. The wireless terminal determines the quality of the 6G network channel by detecting the signal strength of different base stations at the receiving antenna.

[0084] Step 8. The train decides the amount of 6G network to use based on the type of sensor data and channel quality, and determines the corresponding methods for generating and eliminating information redundancy.

[0085] Furthermore, first, check the type of sensor data. If it is a surveillance video in the car, the priority is low, and one 6G base station is selected to send it. If it is the temperature of the running gear shaft, the priority is high. According to the current channel quality, 2-3 6G base stations are selected to send data. If the channel quality is good, the information redundancy generation technology is not used. If the quality is poor, the network coding technology is selected to increase the number of data packets to improve data redundancy. In this way, even if not all data packets are received, the decoder can still restore the complete information through some data packets, thereby significantly reducing the probability of retransmission. The maintenance base can eliminate redundancy by adopting the corresponding decoding technology.

[0086] Step 9. The train sends data to the maintenance station via multiple paths.

[0087] Step 10. If the train does not encounter an emergency such as a breakdown or natural disaster, repeat Step 7 to Step 10.

[0088] Furthermore, the digital twin large model training module determines whether a fault or natural disaster has occurred based on the sensor data sent back by the train wireless terminal.

[0089] Step 11. The train sends information to the maintenance station to report the emergency situation, and the maintenance station can also proactively notify the train of potential equipment failures based on the predictions of the large model. After this step, the train enters the emergency communication mode.

[0090] Step 12. The wireless terminal detects the channel quality of the 6G-R base station and decides whether it needs to obtain communication support from the relay drone. For trains located in mountainous areas or with other obstacles, data can be first sent to the drone, which is then forwarded by the drone to a nearby 6G-R base station.

[0091] Step 13. The 6GR base station allocates sufficient spectrum resources to the train according to the transmission requirements. In the 6G-R base station, slicing technology is used to isolate train control services and diagnostic services to avoid mutual interference. Within the slice, appropriate spectrum resources are allocated to the train based on the data volume and delay requirements.

[0092] At this stage, this application introduces an algorithm based on deep reinforcement learning to improve decision-making efficiency and accuracy. Figure 2 As shown in the lower left corner of the figure, the proposed algorithm jointly allocates the time slots of TSN and the spectrum of 6G-R base stations. Micro time slots are used as the scheduling unit of 6G-R base stations to match the granularity of TSN time slots. Wireless terminals and 6G-R base stations can forward packets within one time slot or use all remaining time to complete forwarding within two time slots. According to the selected time plan, appropriate spectrum is allocated to the train.

[0093] Step 14. The train sends data back to the maintenance station via the 6G-R base station.

[0094] Step 15. After the emergency situation is resolved, the train returns to normal transmission mode and repeats Step 7-Step 10.

[0095] In an exemplary embodiment, S2 may be replaced by the following steps.

[0096] S21: Under the control of the team leader drone, multiple imaging drones are used to scan the area within the set range of the railway line to collect environmental image data; the area includes bridges, railway lines and areas prone to natural disasters.

[0097] S22: Control the imaging drone to perform a recognition task based on the environmental image data, generate a recognition result, and send the recognition result to the team leader drone.

[0098] S23: Control the team leader drone to notify the maintenance base or staff to handle abnormal situations according to the recognition results, and generate task offloading decisions based on complex tasks based on the environmental image data; the complex tasks include splicing point cloud image tasks, path planning tasks, and dynamic routing tasks.

[0099] S24: Using a multi-agent reinforcement learning algorithm, plan the flight paths of each category of drones according to the task offloading decision.

[0100] S25: Based on the flight path, a distributed reinforcement learning routing algorithm is used to determine the optimal transmission path according to multiple system factors, and the imaging drone is controlled to transmit environmental image data according to the optimal transmission path; the multiple system factors include channel status, transmission delay and link throughput.

[0101] In practical applications, S2 can also be replaced by the following steps.

[0102] The drone cluster module for environmental modeling and automatic inspection transmits the monitoring information of infrastructure along the railway and the sensor monitoring information of the surrounding mountainous areas to the digital twin large model training module, such as Figure 3 shown.

[0103] Sub-step 1: Under the control of a team leader drone, multiple imaging drones work together to scan bridges, railway lines, and areas prone to natural disasters using high-resolution cameras or lidar.

[0104] Sub-step 2: For identification tasks, such as identifying whether there are people intruding on the railway line or whether there are floating objects hanging on the overhead contact system (OCS-System), the imaging drone completes the task locally and sends the results to the leader drone. The leader drone will notify the maintenance base or staff to deal with the abnormal situation. For complex tasks such as laser point cloud image stitching, the leader drone determines the appropriate calculation location.

[0105] Sub-step 3: The leader drone offloads complex tasks such as laser point cloud image stitching, path planning, and dynamic routing to appropriate computing locations. The leader drone uses deep reinforcement learning technology to make task offloading decisions based on constraints such as task delay requirements, computational complexity, and drone remaining energy.

[0106] Sub-step 4: According to the task offloading results of sub-step 3, the leader drone begins to plan the flight paths of various drones. Specifically, by using a multi-agent reinforcement learning algorithm, such as the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, the leader drone and other imaging and relay drones jointly determine their optimal path. If the trajectory planning task is too complex and exceeds the capabilities of small drones (such as imaging drones), the leader drone can also use a single-agent algorithm to centrally plan the flight trajectory of the entire cluster, where the single-agent algorithm can be a deep deterministic policy gradient algorithm (DDPG).

[0107] Sub-step 5: The drone cluster adopts a routing algorithm based on distributed reinforcement learning to adaptively obtain the optimized transmission path according to factors such as channel status, transmission delay, and link throughput.

[0108] Sub-step 6: Due to cost constraints, the scanning range of a single imaging drone is limited, and the point cloud density of its laser scanning is insufficient, making it difficult to create a satisfactory digital twin model of the mountain area. To address this limitation, multiple imaging drones are used to perform collaborative imaging by overlapping scanning of key areas. The imaging drone then sends the point cloud image to the leader drone or maintenance base through the transmission path determined in sub-step 5. The leader drone or maintenance base uses a pre-trained Generative Adversarial Networks (GAN) to seamlessly stitch multiple point cloud images from multiple imaging drones and intelligently repair blurred parts.

[0109] In an exemplary embodiment, S3 may be replaced by the following steps.

[0110] In order to provide various generative services based on digital twins, it is necessary to train multiple large models, such as the running gear temperature prediction model and the flood prediction model. These large models differ in the number of parameters, data volume, and computational complexity. Therefore, in the large model training module based on digital twins, this application adopts a training and reasoning framework based on federated learning (FL), such as Figure 6 shown.

[0111] Sub-step 1: First, select an instance for implementation according to the requirements of a specific large model, which includes a train axle fault warning large model and a fire emergency guidance large model.

[0112] This example contains the basic options for large-scale model training, namely {purpose, algorithm, training mode, interaction mode...}.

[0113] For example, in the prediction of the running gear temperature, the required computational complexity is relatively low, so an implementation example includes {prediction, FedProx, centralized, synchronous interaction}. Among them, {prediction} refers to the use of the model for diagnostic services, {FedProx} represents the algorithm for model aggregation, and {synchronous interaction} means that all local models are updated only once per iteration. In this application, all FL-based model training follows the centralized {training mode}.

[0114] Sub-step 2: Once the appropriate instance is selected, the large model is trained accordingly. The central supercomputing center receives the model gradient from the local server room and uses related algorithms to calculate the aggregate gradient. Commonly used related algorithms include: Federalized Proximal Algorithm (FedProx), Federated Averaging (FedAvg), etc.

[0115] Sub-step 3: These aggregated gradients are then sent back to the local machine room for model parameter update. Sub-steps 2 and 3 are repeated until the model reaches the desired accuracy.

[0116] Sub-step 4: After training is completed, the large model will be deployed for inference at the dispatch center. In order to reduce network load and avoid interfering with the train control system, parameter updates in the inference phase are usually performed at night. Only major model vulnerabilities will be updated during the day.

[0117] The following three items are working methods used in sub-steps 2 and 3 to reduce communication overhead, improve training speed and computing resource utilization, and are not separate steps.

[0118] Traffic compression: This technology is used to reduce the overhead of a single communication. In model aggregation based on the Federated Learning (FL) framework, both the local server room and the central supercomputing center use quantization encoding to transmit gradients. The sender approximates the back-propagated gradient as a 16-bit or even 8-bit floating point number. At the receiving end, an error compensation mechanism is applied to mitigate the impact of gradient quantization on model convergence. In addition, the central supercomputing center evaluates the importance of different elements in the gradient to model training and identifies the key elements. Then, the indices of these important elements are sent to the local computer room. In the subsequent training process, the local computer room and the central center only exchange the gradients of these key elements and ignore other elements. Figure 6 As shown in Figure 1, this traffic compression does not reduce the total throughput, but smoothes the traffic curve by reducing the peak rate, thus playing the role of traffic shaping.

[0119] Communication and computation overlap: Although traffic compression reduces the overhead of a single communication, it also leads to an increase in the number of training iterations. To this end, the present application adopts a method of overlapping communication and computation to speed up training. This overlap allows the transmission of partial calculation results before the model parameter calculation is completed. Specifically, the back propagation of gradients is performed in layers. When the gradient calculation of the last layer is completed, the local server room can start communication with the central supercomputing center. While sending the gradient of the last layer to the central supercomputing center, the local server room continues to perform the back propagation gradient calculation of the previous layer. By performing calculations and communications in parallel, the overall training time is significantly reduced.

[0120] Parameter update: The overlap of communication and computation leads to another problem, that is, local computer rooms with higher computing power can send gradients to the central supercomputing center faster, but they must wait for the slowest local center to send its gradients before they can proceed to the next round of model updates, resulting in a waste of computing resources. To solve this problem, the present application adopts different interaction modes according to the parameter size of the model. For models that require high training accuracy or have fewer parameters, the synchronous interaction mode is used to ensure that all local computer rooms update their parameters at the same time after each iteration. However, for large models with more parameters, the quasi-synchronous interaction mode is adopted, allowing faster local computer rooms to perform multiple parameter updates in a single training iteration. In addition, if Figure 6 As shown, in order to prevent these fast local data centers from updating too frequently and negatively affecting the model accuracy, the maximum number of updates allowed in each iteration is set to 3.

[0121] The publication number is CN118840846A, and the name is a method and storage medium for real-time early warning of abnormal conditions of long-distance highway damage. By burying optical fibers in the road section to be tested, setting a receiving end every n kilometers, and connecting to the distributed optical fiber acoustic sensing system (DAS), real-time early warning of abnormal conditions of long-distance highway damage can be achieved. During the data collection period of the DAS device, each monitoring unit automatically emits light pulses regularly, and the monitoring unit collects the original data of road vibration. After preprocessing the obtained vibration raw data, the data of multiple monitoring units are spliced ​​according to the distance to obtain the matrix image of the vibration signal of the entire highway within a period of time. The obtained vibration signal matrix image is processed and classified by an algorithm to obtain a judgment of different vibrations. These vibrations include vibrations generated by normal vehicle driving, construction vibrations, landslide vibrations, and abnormal traffic accident vibrations. Based on previous data, the DAS cloud platform compares the newly collected vibration data to achieve real-time early warning of road vibrations and ensure highway driving safety.

[0122] Although this method has been applied to highways to a certain extent, there are many problems with the use of fiber optic early warning in the railway field: (1) Railway lines are long and usually far away from urban areas, and the cost of daily maintenance is high. It is not economically feasible to lay optical fiber throughout the entire route. (2) It does not contain a wireless communication module and lacks flexibility. For railway lines in extreme environments, wireless monitoring technologies such as drones are needed to improve monitoring efficiency. (3) Fiber optic and vibration sensing technologies can only monitor the environment along the line and cannot detect and warn vehicles on the road.

[0123] In response to the above issues:

[0124] 1) This application uses wireless means instead of optical fiber, such as drones, public 5 / 6G networks, etc.

[0125] 2) This application has a wireless communication module and a flexible working mode.

[0126] 3) The air-ground-train integrated transmission part of the present application can effectively detect faults of the train itself.

[0127] The publication number is CN105844995A, and the name is Railway Line Operation and Maintenance Measurement Method Based on Vehicle-mounted LiDAR Technology. By carrying a laser radar on the train, it replaces traditional surveyors to automatically inspect the stability of the railway line and its ancillary buildings. This technology is based on laser point cloud technology. By establishing a basic control network and a target control network on the ground, the control network measurement and adjustment solution are completed, and then the solution results are constrained by point cloud segmentation to further improve the data accuracy. Finally, the constrained data is classified to obtain line element information such as rail horizontal and longitudinal section parameters.

[0128] However, this method has the following shortcomings in use: (1) The laser point cloud technology used in this technology realizes the automated detection of railway lines, but it still lacks the ability to automatically monitor the environment along the railway, especially the mountainous environment. There are still blind spots in monitoring safety risk factors. (2) This technology has proposed the deployment of additional detection devices on the train, but it cannot provide any data collection function for the operation and maintenance of the train itself. It is entirely possible to consider using the detection device to monitor the pantograph, axle and other key components of the train at the same time.

[0129] In response to the above questions:

[0130] 1) This application uses laser and high-resolution photography of the railway surroundings, especially in mountainous areas, to solve the problem of blind spots in monitoring.

[0131] 2) Many sensors are deployed on the train to detect the working conditions of the train's pantograph and axles.

[0132] This application achieves timely early warning of potential hazards by establishing a large digital twin model of key equipment and the environment along the line.

[0133] This application has the following unique features:

[0134] 1) Existing deterministic transmission research only focuses on wireless access, while TSN switches are applied to trains and the integration of TSN switches and 6G base stations is studied to achieve end-to-end deterministic transmission in heterogeneous networks.

[0135] 2) In order to reduce costs while completing multiple functions, the drone swarm includes multiple types of drones with different capacities and costs, and a learning-based self-organizing mechanism is proposed to promote collaboration among them.

[0136] 3) According to the search on the patent office website, this application is one of the earliest patents that proposed the application of large models and generative AI to high-speed railway equipment diagnosis and natural disaster prediction. Currently, there are almost no public patents related to railway digital twins and large models. From the above perspective, it is difficult to find other alternative solutions to achieve the purpose of this invention, but there should be some other solutions proposed in the specific implementation process.

[0137] This application states that in the future ultra-high-speed operation scenario, railway operation and maintenance will be more intelligent and real-time, providing further safety protection for trains. First, by establishing an integrated air-ground-vehicle transmission network, the train will realize real-time communication of big data with the maintenance base. Compared with the current GSM-R network that can only transmit voice and text, the amount of data that can be transmitted per unit time on the new network has been significantly increased. Secondly, by introducing automated inspection technologies such as drones, the level of detection of railway lines and surrounding environments can be greatly improved. Compared with traditional manual inspections, automated equipment has better performance in terms of inspection frequency, coverage and monitoring accuracy. Third, large model technology can make full use of the large amount of data accumulated by the current railway operation and maintenance, realize early warning of potential faults, and provide customized disposal solutions through generative technology, which will effectively help the staff on the train improve their emergency response capabilities.

[0138] In an exemplary embodiment, a computer device is provided, such as Figure 7 As shown, the computer device can be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store intelligent railway operation and maintenance data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an intelligent railway operation and maintenance method is implemented.

[0139] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method is implemented when the processor executes the computer program.

[0140] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above method when executed by a processor.

[0141] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the above method when executed by a processor.

[0142] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ReadOnlyMemory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (Magnetoresistive RandomAccess Memory, MRAM), ferroelectric random access memory (Ferroelectric RandomAccess Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (RandomAccess Memory, RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0143] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0144] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The present application uses specific examples to illustrate the principles and implementation methods of the present application. The above examples are only used to help understand the method and core ideas of the present application. At the same time, for those skilled in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. An intelligent railway operation and maintenance system, characterized in that: The intelligent railway operation and maintenance system comprises: Air-ground-train integrated transmission module, used to collect and transmit sensor data of key train components; The drone cluster module is used to collect and transmit environmental image data within the set range of the railway line; The digital twin big model training module is used to build a digital twin big model based on the sensor data and environmental image data, determine the integrated big model, and use the integrated big model to predict train failure types and natural disasters, and generate emergency guidance plans; the integrated big model is a neural network model.

2. The intelligent railway operation and maintenance system according to claim 1 is characterized in that: The air-ground-vehicle integrated transmission module specifically includes: a wireless terminal deployed on the train, an on-board TSN switch, a 6G base station, and a maintenance base; The vehicle-mounted TSN switch is connected to the wireless terminal via an optical fiber and is used to periodically send the collected sensor data to the wireless terminal; The wireless terminal is used to send a detection data packet to the maintenance base through the 6G base station based on different transmission modes; the transmission modes include a normal transmission mode and an emergency transmission mode; The maintenance base is used to exchange information with the wireless terminal according to the detection data packet.

3. The intelligent railway operation and maintenance system according to claim 2 is characterized in that: The drone cluster module specifically includes: a leader drone, an imaging drone and a relay drone; The leader drone is used to manage and control the entire drone cluster; The imaging drone is used to collect point cloud images; the point cloud images are the environmental image data; The relay drone is used to forward the sensor data to the 6G base station when the wireless terminal detects that the channel quality of the 6G base station is lower than a channel quality threshold.

4. The intelligent railway operation and maintenance system according to claim 1 is characterized in that: The digital twin big model training module adopts a big model training and reasoning framework based on federated learning, which specifically includes: a central supercomputing center and multiple local server rooms located in each maintenance base; The local server room is used to build a digital twin model based on the sensor data and environmental image data, and train a local neural network model; The central supercomputing center is connected to the local server room, and is used to train and aggregate multiple local neural network models to determine an integrated large model, and use the integrated large model to predict train failure types and natural disasters and generate emergency guidance plans.

5. An intelligent railway operation and maintenance method, characterized in that: The intelligent railway operation and maintenance method is applied to the intelligent railway operation and maintenance system according to any one of claims 1 to 4, and the intelligent railway operation and maintenance method comprises: Use the air-ground-train integrated transmission module to collect and transmit sensor data of key train components; Use the drone cluster module to collect and transmit environmental image data within the set range of the railway line; Using the digital twin big model training module, a digital twin big model is constructed according to the sensor data and environmental image data, an integrated big model is determined, and the integrated big model is used to predict train failure types and natural disasters, and generate emergency guidance plans; the integrated big model is a neural network model.

6. The intelligent railway operation and maintenance method according to claim 5, characterized in that: The air-ground-train integrated transmission module is used to collect and transmit sensor data of key train components, including: Using the on-board TSN switch, the collected sensor data is periodically sent to the wireless terminal; Using the wireless terminal, based on different transmission modes, through the 6G base station, the train is controlled to send a path delay detection message to a maintenance base within a set distance range; the path delay detection message is a detection data packet; the transmission mode includes a normal transmission mode and an emergency transmission mode; Selecting a detection base closest to the train according to the round-trip time of information exchange between the detection base and the wireless terminal; Control the train to send a train identifier to a detection base closest to the train, and control the detection base to feed back a station identifier to the train; Based on the site identifier, according to the channel quality of the 6G base station, the sensor data of the key components of the train are transmitted to the detection base; if the channel quality of the 6G base station is lower than the channel quality threshold, the sensor data is forwarded to the 6G base station by a relay drone, and then the 6G base station transmits the sensor data of the key components of the train to the detection base.

7. The intelligent railway operation and maintenance method according to claim 5, characterized in that: The drone cluster module is used to collect and transmit environmental image data within the set range of the railway line, including: Under the control of the team leader drone, multiple imaging drones are used to scan areas within a set range of the railway line to collect environmental image data; the areas include bridges, railway lines, and areas prone to natural disasters; Controlling the imaging drone to perform a recognition task based on the environmental image data, generate a recognition result, and send the recognition result to the team leader drone; Control the team leader drone to notify the maintenance base or staff to handle abnormal situations according to the recognition results, and generate task offloading decisions based on the environmental image data and complex tasks; the complex tasks include point cloud image stitching tasks, path planning tasks, and dynamic routing tasks; A multi-agent reinforcement learning algorithm is used to plan the flight paths of each category of drones based on the task offloading decisions; Based on the flight path, a distributed reinforcement learning routing algorithm is adopted to determine the optimal transmission path according to multiple system factors, and control the imaging drone to transmit environmental image data according to the optimal transmission path; the multiple system factors include channel status, transmission delay and link throughput.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intelligent railway operation and maintenance system described in any one of claims 5 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the intelligent railway operation and maintenance system described in any one of claims 5 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the intelligent railway operation and maintenance system described in any one of claims 5 to 7 is implemented.

Images