A system and method for compiling driving prompt data of a heavy-haul locomotive

Abstract

The present invention discloses a system and method for compiling driving prompt data of heavy-haul locomotives, including the following steps: S1, collecting multi-modal data and performing preprocessing; S2, using a cross-modal feature alignment mechanism based on Transformer for temporal alignment and correlation analysis; S3, using a gated recurrent unit to combine a three-dimensional terrain model and line features to construct a spatio-temporal model of train operation; S4, inputting the spatio-temporal model of train operation into a neuro-symbolic hybrid inference engine to dynamically analyze train operation data; S5, using a soft actor-critic algorithm to generate personalized operation suggestions; S6, constructing a causally enhanced variational neural network for equipment early warning; S7, generating real-time emergency operation suggestions and outputting an emergency treatment plan; S8, generating voice and graphic prompts and providing driving prompt information to the driver through a display screen. The present invention combines technologies such as multi-modal data, neuro-symbolic reasoning, and causal inference to realize the compilation of driving prompt data for heavy-haul locomotives.

Description

Technical Field

[0001] The present invention relates to the technical field of train operation prompting, and particularly relates to a system and method for compiling train operation prompting data of heavy-haul locomotives. Background Art

[0002] The existing train operation prompting technology for heavy-haul locomotives mainly relies on rule-based static prompting systems. Such systems usually judge the state of the locomotive by monitoring basic operation parameters such as vehicle speed, temperature, and pressure, and provide corresponding alarms or prompts. However, this monitoring method based on a single data source has obvious limitations. First of all, traditional methods mostly rely on manually set thresholds. When the operation state of the locomotive approaches or exceeds these preset values, the system will issue an alarm. Such a processing method often ignores the real-time changes and diversity of the locomotive in a complex environment. For example, during long-term operation, there may be minor changes or equipment aging phenomena, and these changes are not sufficient to trigger a warning from the traditional rule system, resulting in potential failures not being detected in time and increasing the risk of accidents.

[0003] Secondly, most of the traditional train operation prompting systems rely on single-modal data sources, which makes their response ability to multiple factors insufficient in a complex operation environment. The operation environment of heavy-haul locomotives is very complex, involving multiple factors such as terrain, weather, and vehicle performance. Single-modal data cannot comprehensively capture the mutual influence of these complex factors. For example, video monitoring can provide the view outside the carriage, but it cannot effectively capture equipment failures inside the vehicle or the operation behavior of the driver. Therefore, the existing technology has the defect of being unable to effectively integrate multi-modal information when dealing with the complex driving environment of heavy-haul locomotives, which limits the accuracy and personalization of system prediction and prompting.

[0004] In terms of intelligent decision-making, the existing technology usually relies on a rule engine for static decision-making, and lacks in-depth learning analysis of the dynamic operation trajectory and operation behavior of the locomotive. Traditional control systems usually provide fixed operation suggestions under preset conditions, and for personalized driver behavior, different operation environments, and complex operation requirements, the existing prompting systems lack flexibility. Especially in a dynamic environment, traditional methods often cannot adjust intelligently in time according to the driver's control behavior and the actual operation state of the locomotive. For example, when the locomotive enters a steep slope, the driver may need to adjust the operation strategy, and the traditional prompting system cannot provide personalized control suggestions in real time according to the terrain change and vehicle speed change.

[0005] Therefore, how to provide a system and method for compiling train operation prompting data of heavy-haul locomotives is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0006] An object of the present invention is to provide a system and method for compiling driving prompt data of a heavy-haul locomotive. The present invention combines multi-modal data processing, time series modeling, neuro-symbolic reasoning, and causal inference technologies to achieve real-time processing of complex locomotive operation data, terrain data, and driver control behaviors. Through the Transformer model for cross-modal feature alignment, the soft action algorithm for personalized control prediction, and the combination of the equipment causal relationship graph for early warning, not only the accuracy of the control prompt is improved, but also the safety and operation efficiency of the heavy-haul locomotive are significantly optimized.

[0007] A method for compiling driving prompt data of a heavy-haul locomotive according to an embodiment of the present invention includes the following steps:

[0008] S1. Collect multi-modal data and perform preprocessing to obtain preprocessed multi-modal data;

[0009] S2. Use the cross-modal feature alignment mechanism based on Transformer to perform time series alignment and correlation analysis on the preprocessed multi-modal data, extract cross-modal features in the time series, and generate a multi-dimensional feature representation;

[0010] S3. Based on the multi-dimensional feature representation, use a gated recurrent unit to perform time series modeling on the dynamic operation trajectory, and combine the three-dimensional terrain model and line features to construct a spatio-temporal model of train operation;

[0011] S4. Input the spatio-temporal model of train operation into a neuro-symbolic hybrid inference engine to perform dynamic analysis on train operation data, process explicit rule data through a symbolic logic module to generate a logic tree, and at the same time use a graph neural network to extract implicit correlation features;

[0012] S5. Based on the dynamic analysis results and the driver's historical operation data, use the soft action actor-critic algorithm to predict the control behavior, dynamically adjust the control prompt rules, and generate personalized control suggestions;

[0013] S6. Use the personalized control suggestions and real-time operation data to construct a causally enhanced variational neural network, and generate an equipment causal relationship graph through a causal inference algorithm for equipment early warning;

[0014] S7. Generate real-time emergency operation suggestions according to the equipment early warning results and operation rules, and output an emergency treatment plan;

[0015] S8. Combine the emergency treatment plan and the spatio-temporal model of train operation, use natural language generation technology to generate voice and graphic prompts, and provide driving prompt information to the driver through a display screen.

[0016] Optionally, the multi-modal data includes video data and audio data, and the preprocessing includes denoising and outlier processing.

[0017] Optionally, S2 specifically includes:

[0018] S21. Perform time slicing on the preprocessed multi-modal data, and divide the video data and audio data into several time segments according to the time window T w to generate a multi-modal time series of length L;

[0019] S22. Use a convolutional neural network to extract the spatial feature F v (t) of the video data, and use Mel-frequency cepstral coefficients to extract the frequency feature F a (t);

[0020] S23. Use a cross-modal feature alignment mechanism based on Transformer to input the spatial feature F v (t) of the video data and the frequency feature F a (t) of the audio data into the Transformer model, and generate a query matrix Q, a key matrix K, and a value matrix V through the embedding layer;

[0021] S24. Perform temporal alignment on the cross-modal features through the self-attention mechanism, and calculate the inter-modal correlation weights:

[0022]

[0023] where Attention represents the self-attention mechanism, Softmax represents the normalization function, and d k represents the dimension of the key matrix; extract the correlation features between each modality in the time series;

[0024] S25. Align the cross-modal features based on the multi-head attention mechanism, fuse the multi-modal data within the time step, calculate the global correlation features between modalities, and generate a fusion feature representation at the time step level;

[0025] S26. Further refine the aligned cross-modal fusion feature representation through the fully connected layer of the Transformer model to generate a multi-dimensional feature representation F(t) with dimensions [T, D], where T represents the number of time steps and D represents the dimension of the fused features.

[0026] Optionally, S3 specifically includes:

[0027] S31. Segment the generated multi-dimensional feature representation F(t) according to the number of time steps T, and extract the time series dynamic features as the input for time series modeling;

[0028] S32. Use a gated recurrent unit to model the time series dynamic features, and initialize the hidden state of the gated recurrent unit as a zero vector;

[0029] For each time step t, the update gate and the reset gate are calculated based on the input features and the previous hidden state:

[0030] z t =σ(W z ·[h t-1 ,f t +b z );

[0031] r t =σ(W r ·[h t-1 ,f t +b r );

[0032] Among them, z t represents the update gate, r t represents the reset gate, σ represents the activation function, W z represents the weight matrix of the update gate, W r represents the weight matrix of the reset gate, b z represents the bias term of the update gate, b r represents the bias term of the reset gate, h t-1 represents the hidden state of the previous time step, f t represents the multimodal input features of the current time step t;

[0033] Calculate the candidate hidden state and the current hidden state:

[0034]

[0035] Among them, h~ t represents the candidate hidden state, h t represents the hidden state of the current time step t, tanh represents the hyperbolic tangent function, W h represents the weight matrix of the candidate hidden state, b h represents the bias term of the candidate hidden state, ⊙ represents the element-wise multiplication operation;

[0036] S33. Generate the hidden state h t of each time step, and obtain the hidden state sequence H of the dynamic operation trajectory, which is used to characterize the dynamic operation trajectory of the train, and output the timing modeling result;

[0037] S34. Establish the terrain point cloud data according to the terrain data G(x, y, z) of the train operation area, where (x, y, z) represents the three-dimensional coordinates of the terrain, x represents the horizontal position coordinate, y represents the vertical position coordinate, and z represents the height position coordinate;

[0038] S35. Generate a comprehensive grid model by combining line feature data with terrain point cloud data, optimize the comprehensive grid model to eliminate redundant terrain information, and generate a 3D terrain model;

[0039] S36. Map the hidden state sequence H of the dynamic operation trajectory to the 3D terrain model to construct a spatio-temporal model of train operation. The spatio-temporal model of train operation includes the trajectory sequence of train operation and the corresponding 3D spatial positions.

[0040] Optionally, the specific steps of S4 are as follows:

[0041] S41. Input the generated spatio-temporal model of train operation into a neuro-symbolic hybrid inference engine. The neuro-symbolic hybrid inference engine includes a symbolic logic module, a graph neural network module, and a task shunting mechanism, which are used for dynamic analysis of train operation data;

[0042] S42. Use the symbolic logic module to process the explicit rules in the input data, extract the key rule data of train operation. The key rule data includes neutral section distance, speed limit, and gradient characteristics, construct a rule set, and generate a hierarchical symbolic logic tree based on the rule set;

[0043] S43. Recursively analyze the symbolic logic tree through the symbolic logic module, calculate the applicability of the explicit rules in combination with the input data, dynamically generate the inference results of the explicit rules, and output the rule set applicable to the current train operation state;

[0044] S44. Use the graph neural network module to extract the implicit association features in the spatio-temporal model of train operation. Construct nodes and edges with the time step features of train operation and terrain and line features to form a graph structure, and extract the correlation between time steps and the potential impact of the operation environment through the message passing mechanism of the graph;

[0045] S45. In the graph neural network module, extract the implicit association information in the train operation data, including the dynamic change trend of the operation trajectory and the potential relationship between the operation environment, through multi-layer node feature update and edge weight adjustment, and generate the analysis results of the implicit association features;

[0046] S46. Through the task shunting mechanism, dynamically allocate the inference tasks to the symbolic logic module or the graph neural network module according to the attributes and analysis requirements of the input data, integrate the inference results of the explicit rules and the analysis results of the implicit association features, and output the dynamic analysis results by the neuro-symbolic hybrid inference engine to represent the current operation state and operation requirements of the train.

[0047] Optionally, the specific steps of S5 are as follows:

[0048] S51. Input the generated dynamic analysis results and the driver's historical operation data into the soft actor-critic algorithm model. The dynamic analysis results include listing the current operating status and manipulation requirements, and the historical operation data includes the driver's operation behavior records and the corresponding operating environment characteristics.

[0049] S52. In the soft actor-critic algorithm, define the environmental state, action space, and reward function. The environmental state is denoted as S t , the action space is denoted as A t , and the reward function is denoted as R(S t , A t ):

[0050] R(S t , A t ) = -(∥S t - S target ∥ + ∥A t - A target ∥);

[0051] Where, S target represents the target state, A target represents the target action, and ∥·∥ represents the norm of the vector.

[0052] S53. Use the policy network π θ (A t | S t ) to predict the manipulation behavior according to the current environmental state, and generate the optimal manipulation action at the current time step. Among them, π θ (A t | S t ) is the policy learned by the policy network parameters θ, representing the probability distribution of executing the action A t under the state S t ;

[0053] S54. Use the critic network to evaluate the manipulation behavior generated by the policy network. The critic network updates the parameters by minimizing the loss function

[0054]

[0055] Among them, represents the loss function, represents the expected value under the state S t , action A t and immediate reward R t , γ represents the discount factor, represents the current critic network parameters, φ ′ represents the target critic network parameters, Represents the action value predicted by the current critic network, represents the maximum expected action value of all possible actions A of the target critic network at the next time step t+1; t+1

[0056] S55. Dynamically adjust the operation prompt rules based on the policy network, combine the operation requirements of the train with historical operation data, update the prompt rules to adapt to the current operating environment, and generate operation prompt content;

[0057] S56. Generate personalized operation suggestions according to the adjusted operation prompt rules.

[0058] Optionally, the S6 specifically includes:

[0059] S61. Construct a causally enhanced variational neural network according to the personalized operation suggestions and real-time operation data. The causally enhanced variational neural network is used to capture the potential causal relationship between the manipulation behavior and the device state, and input the real-time dynamic parameters and historical operation data of the train;

[0060] S62. In the causally enhanced variational neural network, identify and establish the causal relationship between devices through a causal inference algorithm, model the influence path between the device and the operation parameters, and generate a device causal relationship diagram;

[0061] S63. Combine the device causal relationship diagram with historical operation data, and use the variational inference method to infer the device state;

[0062] S64. Update the relationship weights in the device causal diagram, and optimize the device causal relationship diagram by iteratively adjusting the causal relationship between devices;

[0063] S65. Generate a device warning report based on the optimized device causal relationship diagram combined with real-time operation data.

[0064] According to an embodiment of the present invention, a compilation system for heavy-haul locomotive driving prompt data includes a hardware part and a software part. The hardware part includes a display screen, a base, a communication line, and a speaker; the software part includes:

[0065] A data acquisition module for collecting and preprocessing multi-modal data;

[0066] A data alignment and analysis module for performing cross-modal feature alignment and time-series correlation analysis to generate multi-dimensional feature representations;

[0067] A time-series modeling and terrain construction module for performing time-series modeling based on the multi-dimensional feature representations, and combining a three-dimensional terrain model and line features to construct a spatio-temporal model of train operation;

[0068] ​A neuro-symbolic reasoning module for analyzing train operation data and extracting explicit rule data and implicit association features;

[0069] An operation prediction module for predicting operation behaviors based on the analysis results and historical data and generating personalized operation suggestions;

[0070] A device warning module for generating a device causal relationship diagram through a causal inference algorithm for device warning;

[0071] An emergency handling module for generating and outputting real-time emergency operation suggestions;

[0072] A prompt information generation module for generating and outputting voice and graphic prompt information to the driver.

[0073] The beneficial effects of the present invention are as follows:

[0074] First, the present invention adopts a comprehensive analysis method of multi-modal data and can provide accurate train operation prompts in a wider range of scenarios. Compared with traditional train operation prompt systems based on single-modal data, the present invention fuses information of multiple modalities such as video data and audio data, greatly improving the adaptability to complex operation environments and the response ability to various factors. Especially under the influence of external factors such as complex terrains and weather changes, the present invention can comprehensively obtain and analyze various data sources, thereby providing more accurate and real-time train operation suggestions and ensuring the safety and efficiency of train operation.

[0075] Second, the present invention's cross-modal feature alignment mechanism based on Transformer and temporal modeling using gated recurrent units (GRUs) can dynamically track the running track of the locomotive and adjust the prompt content according to real-time changes. Traditional train operation prompt systems often rely on static rules and lack the real-time perception and response ability to the running state of the locomotive in a dynamic environment. However, the present invention can continuously evaluate and adjust operation suggestions during operation by dynamically analyzing the spatio-temporal model of train operation and combining the historical operation data of the driver, and provide personalized recommendations for different operation requirements and terrain changes. In this way, not only the accuracy of train operation prompts is improved, but also the operation flexibility and adaptability of the driver are significantly enhanced.

[0076] In terms of device warning, the present invention can capture potential causal relationships between devices and accurately give warnings by introducing a causally enhanced variational neural network and a causal inference algorithm. By constructing and optimizing the device causal relationship diagram, the fault risks of devices can be identified in advance, and detailed warning reports can be generated to help operation personnel take emergency measures in time to prevent the occurrence of major accidents. This warning method based on causal inference has higher accuracy and operability compared with traditional technologies and can significantly improve the prevention ability of the locomotive.

[0077] In addition, the neuro-symbolic hybrid inference engine adopted by the present invention combines a symbolic logic module and a graph neural network module to achieve an organic combination of explicit rules and implicit associations, enabling the driving prompt to not only have rule-based reasoning capabilities but also extract deep implicit patterns from complex dynamic data. This method enables the system to make more reasonable judgments and provide practical operation suggestions when facing unknown or complex operating environments. Through the message passing mechanism of the graph neural network, the system can dynamically adjust the associated features of the operating state, continuously optimize the driving prompt, and improve the timeliness and accuracy of the prompt.

[0078] Finally, by combining natural language generation technology, the present invention converts complex dynamic analysis results into voice and graphic prompts that are easy for drivers to understand. This design not only optimizes the expression form of the driving prompt but also improves the acceptance and execution of the prompt content by drivers, thereby enhancing the safety and efficiency of train operation. Especially in emergency situations, the system can quickly generate emergency operation suggestions and timely remind the driver through a combination of voice and graphics to ensure a rapid response in the event of an emergency and prevent accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0080] Figure 1 is the overall flowchart of a method for compiling driving prompt data of a heavy-haul locomotive proposed by the present invention;

[0081] Figure 2 is a schematic diagram of a neuro-symbolic hybrid inference engine of a method for compiling driving prompt data of a heavy-haul locomotive proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0082] Now, the present invention will be further described in detail with reference to the accompanying drawings. These drawings are all simplified schematic diagrams, only illustrating the basic structure of the present invention in a schematic manner, so they only show the components related to the present invention.

[0083] Refer to Figure 1 , a method for compiling driving prompt data of a heavy-haul locomotive, includes the following steps:

[0084] S1. Collect multi-modal data and perform preprocessing to obtain preprocessed multi-modal data;

[0085] S2. Use a cross-modal feature alignment mechanism based on Transformer to perform temporal alignment and correlation analysis on the preprocessed multi-modal data, extract cross-modal features in the time series, and generate a multi-dimensional feature representation;

[0086] S3. Based on the multi-dimensional feature representation, use a gated recurrent unit to perform temporal modeling on the dynamic operation trajectory, and combine the three-dimensional terrain model and line features to construct a spatio-temporal model of train operation;

[0087] S4. Input the spatio-temporal model of train operation into a neuro-symbolic hybrid inference engine to dynamically analyze train operation data. Process explicit rule data through a symbolic logic module to generate a logic tree, and at the same time use a graph neural network to extract implicit association features;

[0088] S5. Based on the dynamic analysis results and the driver's historical operation data, use the soft actor-critic algorithm to predict operation behaviors, dynamically adjust operation prompt rules, and generate personalized operation suggestions;

[0089] S6. Use the personalized operation suggestions and real-time operation data to construct a causally enhanced variational neural network, and generate a device causal relationship graph through a causal inference algorithm for device early warning;

[0090] S7. Generate real-time emergency operation suggestions according to the device early warning results and operation rules, and output an emergency treatment plan;

[0091] S8. Combine the emergency treatment plan and the spatio-temporal model of train operation, use natural language generation technology to generate voice and graphic prompts, and provide train operation prompt information to the driver through a display screen.

[0092] In this embodiment, the multi-modal data includes video data and audio data, and the preprocessing includes denoising and outlier processing.

[0093] In this embodiment, the specific steps of S2 are as follows:

[0094] S21. Perform time slicing on the preprocessed multi-modal data, and divide the video data and audio data into several time segments according to a time window T w to generate multi-modal time series with a length of L;

[0095] S22. Use a convolutional neural network to extract the spatial feature F v (t) of the video data, and use Mel-frequency cepstral coefficients to extract the frequency feature F a (t) of the audio data;

[0096] S23. Use a cross-modal feature alignment mechanism based on Transformer to input the spatial feature F v (t) of the video data and the frequency feature F a (t) of the audio data into the Transformer model, and generate a query matrix Q, a key matrix K, and a value matrix V through an embedding layer;

[0097] S24. Align the cross-modal features in time series through the self-attention mechanism, and calculate the correlation weights between modalities:

[0098]

[0099] where Attention represents the self-attention mechanism, Softmax represents the normalization function, and d k represents the dimension of the key matrix; extract the correlation features between each modality in the time series;

[0100] S25. Align the cross-modal features based on the multi-head attention mechanism, fuse the multi-modal data within the time step, calculate the global correlation features between modalities, and generate the fused feature representation at the time-step level;

[0101] S26. Further refine the aligned cross-modal fused feature representation through the fully connected layer of the Transformer model to generate a multi-dimensional feature representation F(t) with dimensions [T, D], where T represents the number of time steps and D represents the dimension of the fused features.

[0102] In this embodiment, the specific steps of S3 are as follows:

[0103] S31. Segment the generated multi-dimensional feature representation F(t) according to the number of time steps T, and extract the time series dynamic features as the input for time series modeling;

[0104] S32. Model the time series dynamic features using a gated recurrent unit, and initialize the hidden state of the gated recurrent unit as a zero vector;

[0105] For each time step t, calculate the update gate and reset gate based on the input feature and the previous hidden state:

[0106] z t = σ(W z · [h t-1 , f t + b z );

[0107] r t = σ(W r · [h t-1 , f t + b r );

[0108] where z t represents the update gate, r t represents the reset gate, σ represents the activation function, W z represents the weight matrix of the update gate, W r represents the weight matrix of the reset gate, bz Denote the bias term of the update gate, b r Denote the bias term of the reset gate, h t-1 Denote the hidden state at the previous time step, f t Denote the multi-modal input features at the current time step t;

[0109] Calculate the candidate hidden state and the current hidden state:

[0110]

[0111] Among them, Denote the candidate hidden state, h t Denote the hidden state at the current time step t, tanh represents the hyperbolic tangent function, W h Denote the weight matrix of the candidate hidden state, b h Denote the bias term of the candidate hidden state, ⊙ represents the element-wise multiplication operation;

[0112] S33. Generate the hidden state h at each time step t , obtain the hidden state sequence H of the dynamic operation trajectory, which is used to characterize the train's dynamic operation trajectory, and output the time series modeling result;

[0113] S34. Establish terrain point cloud data according to the terrain data G(x, y, z) of the train operation area, where (x, y, z) represents the three-dimensional coordinates of the terrain, x represents the horizontal position coordinate, y represents the vertical position coordinate, and z represents the height position coordinate;

[0114] S35. Use the line feature data to combine with the terrain point cloud data to generate a comprehensive grid model, optimize the comprehensive grid model, eliminate terrain redundant information, and generate a three-dimensional terrain model;

[0115] S36. Map the hidden state sequence H of the dynamic operation trajectory to the three-dimensional terrain model to construct a spatio-temporal model of train operation. The spatio-temporal model of train operation includes the trajectory sequence of train operation and the corresponding three-dimensional spatial positions.

[0116] In this embodiment, the specific content of S4 includes:

[0117] S41. Input the generated spatio-temporal model of train operation into the neuro-symbolic hybrid inference engine. The neuro-symbolic hybrid inference engine includes a symbolic logic module, a graph neural network module, and a task shunting mechanism, which is used to dynamically analyze the train operation data;

[0118] S42. Process the explicit rules in the input data using the symbolic logic module, extract the key rule data for train operation, where the key rule data includes the neutral section distance, speed limit, and gradient characteristics, construct a rule set, and generate a hierarchical symbolic logic tree based on the rule set;

[0119] S43. Recursively analyze the symbolic logic tree through the symbolic logic module, calculate the applicability of the explicit rules in combination with the input data, dynamically generate the inference results of the explicit rules, and output the rule set applicable to the current train operation state;

[0120] S44. Extract the implicit correlation features in the spatio-temporal model of train operation using the graph neural network module, construct nodes and edges from the time-step features of train operation and the terrain and line features to form a graph structure, and extract the correlation between time steps and the potential impact of the operation environment through the message passing mechanism of the graph;

[0121] S45. In the graph neural network module, extract the implicit correlation information in the train operation data, including the dynamic change trend of the operation trajectory and the potential relationship between the operation environment, through multi-layer node feature update and edge weight adjustment, and generate the analysis results of the implicit correlation features;

[0122] S46. Through the task shunting mechanism, dynamically allocate the inference tasks to the symbolic logic module or the graph neural network module according to the attributes and analysis requirements of the input data, integrate the inference results of the explicit rules and the analysis results of the implicit correlation features, and output the dynamic analysis results by the neuro-symbolic hybrid inference engine to represent the current train operation state and operation requirements.

[0123] In this embodiment, the S5 specifically includes:

[0124] S51. Input the generated dynamic analysis results and the historical operation data of the driver into the soft actor-critic algorithm model, where the dynamic analysis results include the current train operation state and operation requirements, and the historical operation data includes the driver's operation behavior records and the corresponding operation environment characteristics;

[0125] S52. In the soft actor-critic algorithm, define the environment state, action space, and reward function, where the environment state is represented as S t , the action space is represented as A t , and the reward function is represented as R(S t ,A t ):

[0126] R(S t ,A t ) = -(∥S t - S target ∥ + ∥A t-A target ∥);

[0127] Wherein, S target represents the target state, A target represents the target action, and ∥·∥ represents the norm of the vector;

[0128] S53. Use the policy network π θ (A t |S t ) to predict the manipulation behavior according to the current environmental state and generate the optimal manipulation action at the current time step, where π θ (A t |S t ) is the policy learned by the policy network parameters θ, representing the probability distribution of executing the action A t under the state S t ;

[0129] S54. Use the critic network to evaluate the manipulation behavior generated by the policy network. The critic network updates the parameters by minimizing the loss function

[0130]

[0131] Wherein, represents the loss function, represents the state S t , the action A t and the expected value under the immediate reward R t , γ represents the discount factor, represents the current critic network parameters, φ ′ represents the target critic network parameters, represents the action value predicted by the current critic network, represents the maximum expected action value of all possible actions A t+1 of the target critic network at the next time step t + 1;

[0132] S55. Dynamically adjust the manipulation prompt rule based on the policy network, combine the manipulation requirements of the train with the historical operation data, update the prompt rule to adapt to the current operating environment, and generate the operation prompt content;

[0133] S56. Generate personalized manipulation suggestions according to the adjusted manipulation prompt rule.

[0134] In this embodiment, the S6 specifically includes:

[0135] ​S61. Construct a causally enhanced variational neural network based on personalized manipulation suggestions and real-time operation data. The causally enhanced variational neural network is used to capture the potential causal relationship between manipulation behaviors and device states, and input the real-time dynamic parameters and historical operation data of the train.

[0136] S62. In the causally enhanced variational neural network, identify and establish the causal relationships between devices through a causal inference algorithm, model the influence paths between devices and operation parameters, and generate a device causal relationship diagram.

[0137] S63. Combine the device causal relationship diagram with historical operation data, and use the variational inference method to infer the device states.

[0138] S64. Update the relationship weights in the device causal diagram, optimize the device causal relationship diagram by iteratively adjusting the causal relationships between devices.

[0139] S65. Generate a device warning report based on the optimized device causal relationship diagram combined with real-time operation data.

[0140] Reference Figure 2 , a compilation system for driving prompt data of heavy-haul locomotives, including a hardware part and a software part. The hardware part includes a display screen, a base, a communication line, and a speaker; the software part includes:

[0141] A data acquisition module for collecting and preprocessing multimodal data.

[0142] A data alignment and analysis module for performing cross-modal feature alignment and time-series correlation analysis to generate multi-dimensional feature representations.

[0143] A time-series modeling and terrain construction module for performing time-series modeling based on multi-dimensional feature representations, and combining a three-dimensional terrain model and line features to construct a spatio-temporal model of train operation.

[0144] A neuro-symbolic reasoning module for analyzing train operation data and extracting explicit rule data and implicit association features.

[0145] An operation prediction module for predicting manipulation behaviors according to the analysis results and historical data, and generating personalized manipulation suggestions.

[0146] A device warning module for generating a device causal relationship diagram through a causal inference algorithm for device warning.

[0147] An emergency handling module for generating and outputting real-time emergency operation suggestions.

[0148] A prompt information generation module for generating and outputting voice and graphic prompt information to the driver.

[0149] Example 1:

[0150] To verify the feasibility of the present invention in implementation, the present invention is applied to a certain heavy-haul locomotive. This locomotive is currently operating on a certain railway freight line in China. The line is 1000 kilometers long and has a heavy load during the driving process, especially in mountainous areas and complex terrains. The locomotive needs to frequently perform operation adjustments during operation. In this environment, the traditional train operation prompt system only relies on simple speed and distance monitoring, and fails to fully consider the comprehensive influence of multi-faceted data such as real-time terrain changes, locomotive status, and driver's historical operations, resulting in frequent operation errors such as brake failure and excessive acceleration, affecting train operation safety and efficiency.

[0151] To improve the train operation safety of the heavy-haul locomotive on this line, this embodiment applies the dynamic analysis and prediction method based on multi-modal data proposed by the present invention. Specifically, technologies such as a cross-modal feature alignment mechanism based on Transformer, a gated recurrent unit (GRU) for time series modeling, and a neuro-symbolic hybrid reasoning engine for dynamic analysis are adopted, and combined with a real-time terrain model and locomotive dynamic parameters, personalized operation suggestions are generated, and an emergency operation plan is generated in a timely manner.

[0152] First, the operation behaviors of the driver are captured through video and audio data. After all the data is preprocessed, the Transformer model is used for time series alignment and feature extraction. Through the comprehensive analysis of cross-modal data, the system can accurately model the vehicle condition from multiple dimensions.

[0153] Secondly, the system uses a gated recurrent unit (GRU) to model the dynamic operation trajectory, and combines a three-dimensional terrain model and line features to construct a spatio-temporal model of train operation. On this basis, the neuro-symbolic hybrid reasoning engine performs dynamic analysis according to explicit rules and implicit features to generate operation suggestions and prediction results for the current train.

[0154] In addition, the system can combine the driver's historical operation data, predict possible operation behaviors through a soft actor-critic algorithm, and adjust the operation rules according to the prediction results. Finally, based on these personalized operation suggestions, the system can generate a real-time emergency operation plan and transmit the information to the driver through a display screen and voice prompt.

[0155] Table 1 Comparison of experimental data between the traditional system and the system of the present invention in train operation prompt for heavy-haul locomotives

[0156] Comparison items Traditional system System of the present invention Percentage increase (%) Operation error rate 15 times / hour 2 times / hour 86.7 Operation safety score 78 96 23.5 Responsiveness to personalized operation suggestions 85％ 98％ 13.7 Accuracy rate of braking system fault prediction 65％ 92％ 27 Accuracy rate of power system fault prediction 70％ 89％ 19 Fault warning lead time 55 minutes 20 minutes 300

[0157] By analyzing the results in Table 1 above, it can be clearly seen that the present invention has a significant improvement compared with the traditional system in multiple key performance indicators.

[0158] First, in terms of the operation error rate, the error rate of the traditional system is 15 times per hour, while that of the present invention is only 2 times per hour, with a reduction rate as high as 86.7%. This data reflects that the present invention can effectively reduce the driver's operation errors through intelligent personalized operation suggestions and dynamic analysis, thus improving the safety and reliability of driving. Reducing operation errors can not only reduce the accident rate but also improve the overall operation efficiency of the locomotive.

[0159] Secondly, the comparison of operation safety scores shows that the safety score of the present invention is 96 points, while that of the traditional system is 78 points, with an improvement rate of 23.5%. This indicates that the present invention can provide more accurate safety tips and operation suggestions in a complex operating environment, ensuring that the driver can make safer decisions based on real-time data, further enhancing the safety guarantee of train operation.

[0160] In terms of the response degree of personalized operation suggestions, the response degree of the traditional system is 85%, while the present invention can achieve a response degree of 98%, with an improvement rate of 13.7%. This result shows that the present invention can provide more accurate personalized suggestions based on the actual operating state of the train and the driver's historical operation behaviors, enabling the driver to react in a shorter time and thus optimizing train operation.

[0161] In addition, the present invention also shows significant advantages in fault prediction. The accuracy rate of power system fault prediction has been improved from 70% of the traditional system to 89% of the present invention, with an improvement rate of 19%. This indicates that by introducing advanced technologies such as causal enhanced variational neural networks and graph neural networks, the present invention can more accurately predict equipment faults, discover potential fault problems in advance, provide an important basis for fault prevention and maintenance, reduce the risk of equipment outage, and lower the maintenance cost.

[0162] Most prominently, the comparison of the fault warning lead time shows that the warning time of the traditional system is 5 minutes, while the present invention can issue a warning 20 minutes in advance, with a 300% improvement. This extension of the warning lead time enables the operator to have sufficient time to take emergency measures to avoid accidents. Especially in the face of a complex heavy-haul locomotive operating environment, the earlier warning time undoubtedly provides greater safety guarantee for the train.

[0163] It can be seen that the present invention is superior to the traditional system in multiple dimensions, especially showing significant improvements in operation error rate, operation safety, response degree of personalized operation suggestions, equipment warning accuracy rate, and warning time. By adopting intelligent technologies based on deep learning, neural networks, and causal reasoning, the present invention can effectively improve the safety and operation efficiency of heavy-haul locomotives, reduce the accident rate, and provide more reliable and intelligent support for railway transportation.

[0164] The above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes, shall be covered by the protection scope of the present invention.

Claims

1. A method for compiling driving reminder data for a heavy-load locomotive, characterized in that: The steps include: S1, collecting multimodal data and preprocessing to obtain preprocessed multimodal data; S2. Using a Transformer-based cross-modal feature alignment mechanism to perform time series alignment and correlation analysis on the pre-processed multimodal data, extract cross-modal features in the time series, and generate a multi-dimensional feature representation; S3. Based on the multi-dimensional feature representation, a gated recurrent unit is used to perform time series modeling on the dynamic running trajectory, and a spatiotemporal model of train operation is constructed by combining a three-dimensional terrain model and line characteristics; S4, inputting the spatiotemporal model of the train operation into a neural-symbolic hybrid reasoning engine, dynamically analyzing the train operation data, processing the explicit rule data through a symbolic logic module and generating a logic tree, and extracting implicit association features using a graph neural network; S5. Based on the dynamic analysis results and the driver's historical operation data, the soft-action actor-critic algorithm is used to predict the manipulation behavior, dynamically adjust the manipulation prompt rules and generate personalized manipulation suggestions; S6. Using the personalized operation suggestions and real-time operation data, construct a causal enhanced variational neural network, and generate a device causal relationship diagram through a causal inference algorithm to perform device early warning; S7. Generate real-time emergency operation suggestions based on equipment warning results and operation rules, and output emergency treatment plans; S8. Combining the emergency response plan with the spatiotemporal model of train operation, natural language generation technology is used to generate voice and graphic prompts, and the driving prompt information is provided to the driver through a display screen.

2. A method for compiling driving reminder data for a heavy-load locomotive according to claim 1, characterized in that: The multimodal data includes video data and audio data, and the preprocessing includes denoising and outlier processing.

3. The method for compiling driving reminder data for a heavy-load locomotive according to claim 1, characterized in that: The S2 specifically includes: S21, time-slicing the pre-processed multimodal data, and dividing the video data and the audio data into time windows T w Divide into several time segments to generate a multimodal time series of length L; S22. Extracting spatial features of video data using convolutional neural network F v (t), use Mel frequency cepstral coefficients to extract the frequency characteristics F of audio data a (t); S23, using the Transformer-based cross-modal feature alignment mechanism, the spatial features F of the video data are aligned v (t) and the frequency characteristics of the audio data F a (t) Input to the Transformer model and generate the query matrix Q, key matrix K and value matrix V through the embedding layer; S24. Use the self-attention mechanism to align cross-modal features in time and calculate the correlation weights between modalities: Among them, Attention represents the self-attention mechanism, Softmax represents the normalization function, and d k Represents the dimension of the key matrix; extracts the correlation features between each mode in the time series; S25. Align cross-modal features based on the multi-head attention mechanism, fuse multimodal data within the time step, calculate the global correlation features between modalities, and generate a fusion feature representation at the time step level; S26, the aligned cross-modal fusion feature representation is further refined through the fully connected layer of the Transformer model to generate a multi-dimensional feature representation F ( t ) , the dimension is [T,D], where T represents the number of time steps and D represents the feature dimension after fusion.

4. The method for compiling driving reminder data for a heavy-load locomotive according to claim 1, characterized in that: The S3 specifically includes: S31, the generated multi-dimensional feature representation F ( t) Segment by time step number T and extract the dynamic features of the time series as the input of time series modeling; S32, using a gated recurrent unit to model the dynamic characteristics of the time series, and initializing the hidden state of the gated recurrent unit to a zero vector; For each time step t, the input features and the previous hidden state are used to calculate the update gate and reset gate: z t =σ(W z ·[h t-1 ,f t ]+b z ); r t =σ(W r [h t-1 ,f t ]+b r ); Among them, z t represents the update gate, r t represents the reset gate, σ represents the activation function, W z represents the weight matrix of the update gate, W r represents the weight matrix of the reset gate, b z represents the bias term of the update gate, b r represents the bias term of the reset gate, h t-1 represents the hidden state of the previous time step, f t Represents the multimodal input features at the current time step t; Compute candidate hidden states and current hidden states: in, represents the candidate hidden state, h t represents the hidden state of the current time step t, tanh represents the hyperbolic tangent function, W h The weight matrix representing the candidate hidden state, b h represents the bias term of the candidate hidden state, ⊙ represents the element-wise multiplication operation; S33, generate the hidden state h for each time step t , obtain the hidden state sequence H of the dynamic running trajectory, which is used to characterize the dynamic running trajectory of the train and output the time series modeling results; S34, establishing terrain point cloud data according to the terrain data G(x, y, z) of the train running area, wherein (x, y, z) represents the three-dimensional coordinates of the terrain, x represents the horizontal position coordinate, y represents the vertical position coordinate, and z represents the height position coordinate; S35, generating a comprehensive grid model by combining the line feature data with the terrain point cloud data, optimizing the comprehensive grid model, eliminating redundant terrain information, and generating a three-dimensional terrain model; S36, mapping the hidden state sequence H of the dynamic running trajectory to the three-dimensional terrain model, and constructing a spatiotemporal model of train operation, wherein the spatiotemporal model of train operation includes a trajectory sequence of the running trajectory and a corresponding three-dimensional spatial position.

5. The method for compiling driving reminder data for a heavy-load locomotive according to claim 1, characterized in that: The S4 specifically includes: S41, inputting the generated spatiotemporal model of train operation into a neural-symbolic hybrid reasoning engine, wherein the neural-symbolic hybrid reasoning engine includes a symbolic logic module, a graph neural network module, and a task offloading mechanism, for dynamically analyzing train operation data; S42, using the symbolic logic module to process the explicit rules in the input data, extracting key rule data of train operation, the key rule data including phase separation distance, speed limit and slope characteristics, constructing a rule set, and generating a hierarchical symbolic logic tree based on the rule set; S43, recursively analyzing the symbolic logic tree through the symbolic logic module, calculating the applicability of the explicit rule in combination with the input data, dynamically generating an explicit rule reasoning result, and outputting a rule set applicable to the current train operation state; S44, using the graph neural network module to extract implicit correlation features in the spatiotemporal model of train operation, constructing the time step features of train operation and the terrain and line features as nodes and edges to form a graph structure, and extracting the correlation between time steps and the potential impact of the operating environment through the message passing mechanism of the graph; S45, in the graph neural network module, by updating multi-layer node features and adjusting edge weights, extracting implicit correlation information in the train operation data, including the dynamic change trend of the operation track and the potential relationship between the operation environment, and generating implicit correlation feature analysis results; S46. Through the task offloading mechanism, the reasoning tasks are dynamically allocated to the symbolic logic module or the graph neural network module according to the attributes of the input data and the analysis requirements, and the explicit rule reasoning results and the implicit association feature analysis results are integrated. The dynamic analysis results are output by the neural-symbolic hybrid reasoning engine to characterize the current operating status and operation requirements of the train.

6. The method for compiling heavy-load locomotive driving reminder data according to claim 1, characterized in that: The S5 specifically includes: S51, inputting the generated dynamic analysis result driver's historical operation data into the soft actor-critic algorithm model, wherein the dynamic analysis result includes a list of current operating states and manipulation requirements, and the historical operation data includes the driver's operating behavior records and corresponding operating environment characteristics; S52. In the soft-actor actor-critic algorithm, define the environment state, action space, and reward function, where the environment state is represented by S t , the action space is represented by A t , the reward function is expressed as R(S t ,A t ): R(S t ,A t )=-(∥S t -S target ∥+∥A t -A target ∥); Among them, S target represents the target state, A target represents the target action, ∥·∥ represents the norm of the vector; S53. Using Strategy Network π θ ( A t |S t ) predicts the manipulation behavior according to the current environment state and generates the optimal manipulation action for the current time step, where π θ (A t |S t ) is the strategy learned by the policy network parameter θ, indicating that in state S t Next, execute action A t The probability distribution of S54. Use the critic network to evaluate the manipulation behavior generated by the policy network. The critic network minimizes the loss function Update Parameters in, represents the loss function, Indicates state S t 、Action A t and instant reward R t The expected value under , γ represents the discount factor, represents the current critic network parameters, φ ′ represents the target critic network parameters, represents the action value predicted by the current critic network, represents all possible actions A of the target critic network at the next time step t+1 t+1 The maximum expected action value of S55, dynamically adjusting the operation prompt rules based on the strategy network, combining the operation requirements of the train with the historical operation data, updating the prompt rules to adapt to the current operating environment, and generating the operation prompt content; S56. Generate personalized manipulation suggestions according to the adjusted manipulation prompt rules.

7. The method for compiling driving reminder data for a heavy-load locomotive according to claim 1, characterized in that: The S6 specifically includes: S61. Constructing a causal enhancement variational neural network based on the personalized operation suggestions and real-time operation data, wherein the causal enhancement variational neural network is used to capture the potential causal relationship between the operation behavior and the equipment state, and inputs the real-time dynamic parameters and historical operation data of the train; S62. In the causal enhancement variational neural network, the causal relationship between devices is identified and established through the causal inference algorithm, the influence path between the device and the operating parameter is modeled, and the device causal relationship diagram is generated; S63, combining the equipment cause-effect relationship diagram with historical operation data, and using variational reasoning methods to infer the equipment status; S64, updating the relationship weights in the device causal relationship graph, and optimizing the device causal relationship graph by iteratively adjusting the causal relationship between devices; S65. Generate an equipment early warning report based on the optimized equipment cause-effect relationship diagram combined with real-time operation data.

8. A system for compiling driving reminder data for a heavy-loaded locomotive, executing the method for compiling driving reminder data for a heavy-loaded locomotive according to any one of claims 1 to 7, characterized in that: It includes hardware and software parts. The hardware part includes a display screen, a base, a communication line and a speaker. The software part includes: A data acquisition module, used for collecting and preprocessing multimodal data; Data alignment and analysis module, used to perform cross-modal feature alignment and time series correlation analysis to generate multi-dimensional feature representation; The time series modeling and terrain construction module is used to perform time series modeling based on multi-dimensional feature representation and to build a spatiotemporal model of train operation by combining the three-dimensional terrain model and line features; Neural symbolic reasoning module, used to analyze train operation data and extract explicit rule data and implicit association features; Operation prediction module, which is used to predict manipulation behavior based on analysis results and historical data and generate personalized manipulation suggestions; The equipment early warning module is used to generate equipment causal relationship diagrams through causal inference algorithms for equipment early warning; Emergency handling module, used to generate and output real-time emergency operation suggestions; The prompt information generation module is used to generate and output voice and graphic prompt information to the driver.

Images