System and method for compiling driving prompt data of heavy-load locomotive

Abstract

The invention discloses a compilation system and method for driving prompt data of a heavy-load locomotive. The compilation method comprises the following steps that S1, multi-modal data are collected and preprocessed; s2, performing time sequence alignment and correlation analysis by using a trans-modal feature alignment mechanism based on a Transform; s3, constructing a space-time model of train operation by utilizing a gating circulation unit in combination with the three-dimensional terrain model and the line characteristics; s4, inputting the time-space model of train operation into a neural symbol hybrid inference engine, and performing dynamic analysis on train manipulation data; s5, generating personalized manipulation suggestions by using a soft action actor-commentator algorithm; s6, constructing a causal enhanced variational neural network to carry out equipment early warning; s7, generating a real-time emergency operation suggestion and outputting an emergency processing scheme; and S8, generating voice and image-text prompts and providing driving prompt information to a driver through a display screen. According to the method, technologies of multi-modal data, neural symbol inference, causal inference and the like are combined, and compilation of the driving prompt data of the heavy-load locomotive is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of driving prompts, and in particular to a system and method for compiling driving prompt data for a heavy-load locomotive. Background Art

[0002] Existing heavy-load locomotive driving warning technology mainly relies on a rule-based static warning system. Such systems usually judge the status of the locomotive by monitoring basic operating parameters such as speed, temperature, and pressure, and provide corresponding alarms or prompts. However, this monitoring method based on a single data source has obvious limitations. First, traditional methods mostly rely on manually set thresholds. When the operating status of the locomotive approaches or exceeds these preset values, the system will issue an alarm. This processing method often ignores the real-time changes and diversity of locomotives in complex environments. For example, during long-term operation, minor changes or equipment aging may occur, and these changes are not enough to trigger warnings from traditional rule systems, resulting in potential faults not being discovered in time, increasing the risk of accidents.

[0003] Secondly, most traditional driving warning systems rely on a single-modal data source, which makes them incapable of responding to multiple factors in a complex operating environment. The operating environment of heavy-loaded locomotives is very complex, involving multiple factors such as terrain, weather, and vehicle performance. Single-modal data cannot fully capture the mutual influence of these complex factors. For example, video surveillance can provide a view outside the car, but cannot effectively capture equipment failures inside the car or the driver's operating behavior. Therefore, when dealing with the complex driving environment of heavy-loaded locomotives, the existing technology has the defect of being unable to effectively integrate multimodal information, which limits the accuracy and personalization of system predictions and prompts.

[0004] In terms of intelligent decision-making, existing technologies usually rely on rule engines for static decision-making, but lack deep learning analysis of the dynamic operation trajectory and operation behavior of the locomotive. Traditional control systems usually provide fixed operation suggestions under preset conditions, but the existing prompt system lacks flexibility for personalized driver behavior, different operating environments and complex operation requirements. Especially in dynamic environments, traditional methods often cannot make intelligent adjustments in time according to the driver's control behavior and the actual operating status of the locomotive. For example, when a locomotive enters a steep slope, the driver may need to adjust the operation strategy, but the traditional prompt system cannot provide personalized control suggestions in real time according to terrain changes and vehicle speed changes.

[0005] Therefore, how to provide a system and method for compiling heavy-load locomotive driving reminder data is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0006] One purpose of the present invention is to propose a system and method for compiling driving prompt data for heavy-duty locomotives. The present invention combines multimodal data processing, time series modeling, neural symbolic reasoning and causal inference technology to achieve real-time processing of complex locomotive operation data, terrain data and driver control behavior. Through the Transformer model to align cross-modal features, use the soft action algorithm to achieve personalized control prediction, and combine the equipment causal relationship diagram for early warning, it not only improves the accuracy of control prompts, but also significantly optimizes the safety and operation efficiency of heavy-duty locomotives.

[0007] A method for compiling driving reminder data for a heavy-load locomotive according to an embodiment of the present invention comprises the following steps:

[0008] S1, collecting multimodal data and preprocessing to obtain preprocessed multimodal data;

[0009] S2. Using a Transformer-based cross-modal feature alignment mechanism to perform time series alignment and correlation analysis on the pre-processed multimodal data, extract cross-modal features in the time series, and generate a multi-dimensional feature representation;

[0010] S3. Based on the multi-dimensional feature representation, a gated recurrent unit is used to perform time series modeling on the dynamic running trajectory, and a spatiotemporal model of train operation is constructed by combining a three-dimensional terrain model and line characteristics;

[0011] S4, inputting the spatiotemporal model of the train operation into a neural-symbolic hybrid reasoning engine, dynamically analyzing the train operation data, processing the explicit rule data through a symbolic logic module and generating a logic tree, and extracting implicit association features using a graph neural network;

[0012] S5. Based on the dynamic analysis results and the driver's historical operation data, the soft-action actor-critic algorithm is used to predict the manipulation behavior, dynamically adjust the manipulation prompt rules and generate personalized manipulation suggestions;

[0013] S6. Using the personalized operation suggestions and real-time operation data, a causal enhanced variational neural network is constructed, and a device causal relationship diagram is generated through a causal inference algorithm to perform device early warning;

[0014] S7. Generate real-time emergency operation suggestions based on equipment warning results and operation rules, and output emergency treatment plans;

[0015] S8. Combining the emergency response plan with the spatiotemporal model of train operation, natural language generation technology is used to generate voice and graphic prompts, and the driving prompt information is provided to the driver through a display screen.

[0016] Optionally, the multimodal data includes video data and audio data, and the preprocessing includes denoising and outlier processing.

[0017] Optionally, the S2 specifically includes:

[0018] S21, time-slicing the pre-processed multimodal data, and dividing the video data and the audio data into time windows T w Divide into several time segments to generate a multimodal time series of length L;

[0019] S22. Extracting spatial features of video data using convolutional neural network F v (t), use Mel frequency cepstral coefficients to extract the frequency characteristics F of audio data a (t);

[0020] S23, using the Transformer-based cross-modal feature alignment mechanism, the spatial features F of the video data are aligned v (t) and the frequency characteristics of the audio data F a (t) Input to the Transformer model and generate the query matrix Q, key matrix K and value matrix V through the embedding layer;

[0021] S24. Use the self-attention mechanism to align cross-modal features in time and calculate the correlation weights between modalities:

[0022]

[0023] Among them, Attention represents the self-attention mechanism, Softmax represents the normalization function, and d k Represents the dimension of the key matrix; extracts the correlation features between each mode in the time series;

[0024] S25. Align cross-modal features based on the multi-head attention mechanism, fuse multimodal data within the time step, calculate the global correlation features between modalities, and generate a fusion feature representation at the time step level;

[0025] S26. The aligned cross-modal fusion feature representation is further refined through the fully connected layer of the Transformer model to generate a multi-dimensional feature representation F(t) with a dimension of [T, D], where T represents the number of time steps and D represents the feature dimension after fusion.

[0026] Optionally, the S3 specifically includes:

[0027] S31, segmenting the generated multidimensional feature representation F(t) according to the number of time steps T, and extracting the dynamic features of the time series as the input of time series modeling;

[0028] S32, using a gated recurrent unit to model the dynamic characteristics of the time series, and initializing the hidden state of the gated recurrent unit to a zero vector;

[0029] For each time step t, the input features and the previous hidden state are used to calculate the update gate and reset gate:

[0030] z t =σ(W z ·[h t-1 ,f t ]+b z );

[0031] r t =σ(W r ·[h t-1 ,f t ]+b r );

[0032] Among them, z t represents the update gate, r t represents the reset gate, σ represents the activation function, W z represents the weight matrix of the update gate, W r represents the weight matrix of the reset gate, b z represents the bias term of the update gate, b r represents the bias term of the reset gate, h t-1 represents the hidden state of the previous time step, f t Represents the multimodal input features at the current time step t;

[0033] Compute candidate hidden states and current hidden states:

[0034]

[0035] Among them, h~ t represents the candidate hidden state, h t represents the hidden state of the current time step t, tanh represents the hyperbolic tangent function, W h The weight matrix representing the candidate hidden state, b h represents the bias term of the candidate hidden state, ⊙ represents the element-wise multiplication operation;

[0036] S33, generate the hidden state h for each time step t , obtain the hidden state sequence H of the dynamic running trajectory, which is used to characterize the dynamic running trajectory of the train and output the time series modeling results;

[0037] S34, establishing terrain point cloud data according to the terrain data G(x, y, z) of the train running area, wherein (x, y, z) represents the three-dimensional coordinates of the terrain, x represents the horizontal position coordinate, y represents the vertical position coordinate, and z represents the height position coordinate;

[0038] S35, generating a comprehensive grid model by combining the line feature data with the terrain point cloud data, optimizing the comprehensive grid model, eliminating redundant terrain information, and generating a three-dimensional terrain model;

[0039] S36, mapping the hidden state sequence H of the dynamic running trajectory to the three-dimensional terrain model, and constructing a spatiotemporal model of train operation, wherein the spatiotemporal model of train operation includes a trajectory sequence of the running trajectory and a corresponding three-dimensional spatial position.

[0040] Optionally, the S4 specifically includes:

[0041] S41, inputting the generated spatiotemporal model of train operation into a neural-symbolic hybrid reasoning engine, wherein the neural-symbolic hybrid reasoning engine includes a symbolic logic module, a graph neural network module, and a task offloading mechanism, for dynamically analyzing train operation data;

[0042] S42, using the symbolic logic module to process the explicit rules in the input data, extracting key rule data of train operation, the key rule data including phase separation distance, speed limit and slope characteristics, constructing a rule set, and generating a hierarchical symbolic logic tree based on the rule set;

[0043] S43, recursively analyzing the symbolic logic tree through the symbolic logic module, calculating the applicability of the explicit rule in combination with the input data, dynamically generating an explicit rule reasoning result, and outputting a rule set applicable to the current train operation state;

[0044] S44, using the graph neural network module to extract implicit correlation features in the spatiotemporal model of train operation, constructing the time step features of train operation and the terrain and line features as nodes and edges to form a graph structure, and extracting the correlation between time steps and the potential impact of the operating environment through the message passing mechanism of the graph;

[0045] S45, in the graph neural network module, by updating multi-layer node features and adjusting edge weights, extracting implicit correlation information in the train operation data, including the dynamic change trend of the operation track and the potential relationship between the operation environment, and generating implicit correlation feature analysis results;

[0046] S46. Through the task offloading mechanism, the reasoning tasks are dynamically allocated to the symbolic logic module or the graph neural network module according to the attributes of the input data and the analysis requirements, and the explicit rule reasoning results and the implicit association feature analysis results are integrated. The dynamic analysis results are output by the neural-symbolic hybrid reasoning engine to characterize the current operating status and operation requirements of the train.

[0047] Optionally, the S5 specifically includes:

[0048] S51, inputting the generated dynamic analysis result driver's historical operation data into the soft actor-critic algorithm model, wherein the dynamic analysis result includes a list of current operating states and manipulation requirements, and the historical operation data includes the driver's operating behavior records and corresponding operating environment characteristics;

[0049] S52. In the soft-actor actor-critic algorithm, define the environment state, action space, and reward function, where the environment state is represented by S t , the action space is represented by A t , the reward function is expressed as R(S t ,A t ):

[0050] R(S t ,A t )=-(∥S t -S target ∥+∥A t -A target ∥);

[0051] Among them, S target represents the target state, A target represents the target action, ∥·∥ represents the norm of the vector;

[0052] S53. Using Strategy Network π θ (A t |S t ) predicts the manipulation behavior according to the current environment state and generates the optimal manipulation action for the current time step, where π θ (A t |S t ) is the strategy learned by the policy network parameter θ, indicating that in state S t Next, execute action A t The probability distribution of

[0053] S54. Use the critic network to evaluate the manipulation behavior generated by the policy network. The critic network minimizes the loss function Update Parameters

[0054]

[0055] in, represents the loss function, Indicates state S t 、Action A t and instant reward R t The expected value under , γ represents the discount factor, represents the current critic network parameters, φ ′ represents the target critic network parameters, represents the action value predicted by the current critic network, represents all possible actions A of the target critic network at the next time step t+1 t+1 The maximum expected action value of

[0056] S55, dynamically adjusting the operation prompt rules based on the strategy network, combining the operation requirements of the train with the historical operation data, updating the prompt rules to adapt to the current operating environment, and generating the operation prompt content;

[0057] S56. Generate personalized manipulation suggestions according to the adjusted manipulation prompt rules.

[0058] Optionally, the S6 specifically includes:

[0059] S61. Constructing a causal enhancement variational neural network based on the personalized operation suggestions and real-time operation data, wherein the causal enhancement variational neural network is used to capture the potential causal relationship between the operation behavior and the equipment state, and inputs the real-time dynamic parameters and historical operation data of the train;

[0060] S62. In the causal enhancement variational neural network, the causal relationship between devices is identified and established through the causal inference algorithm, the influence path between the device and the operating parameter is modeled, and the device causal relationship diagram is generated;

[0061] S63, combining the equipment cause-effect relationship diagram with historical operation data, and using variational reasoning methods to infer the equipment status;

[0062] S64, updating the relationship weights in the device causal relationship graph, and optimizing the device causal relationship graph by iteratively adjusting the causal relationship between devices;

[0063] S65. Generate an equipment early warning report based on the optimized equipment cause-effect relationship diagram combined with real-time operation data.

[0064] A system for compiling driving prompt data for a heavy-loaded locomotive according to an embodiment of the present invention includes a hardware part and a software part, wherein the hardware part includes a display screen, a base, a communication line and a speaker; and the software part includes:

[0065] A data acquisition module, used to collect and pre-process multimodal data;

[0066] Data alignment and analysis module, used to perform cross-modal feature alignment and time series correlation analysis to generate multi-dimensional feature representation;

[0067] The time series modeling and terrain construction module is used to perform time series modeling based on multi-dimensional feature representation and to build a spatiotemporal model of train operation by combining the three-dimensional terrain model and line features;

[0068] Neural symbolic reasoning module, used to analyze train operation data and extract explicit rule data and implicit association features;

[0069] Operation prediction module, which is used to predict manipulation behavior based on analysis results and historical data and generate personalized manipulation suggestions;

[0070] The equipment early warning module is used to generate equipment causal relationship diagrams through causal inference algorithms for equipment early warning;

[0071] Emergency handling module, used to generate and output real-time emergency operation suggestions;

[0072] The prompt information generation module is used to generate and output voice and graphic prompt information to the driver.

[0073] The beneficial effects of the present invention are:

[0074] First, the present invention adopts a comprehensive analysis method of multimodal data, which can provide accurate driving tips in a wider range of scenarios. Compared with the traditional driving prompt system based on single-modal data, the present invention greatly improves the adaptability to complex operating environments and the ability to respond to multiple factors by integrating information in multiple modes such as video data and audio data. Especially under the influence of external factors such as complex terrain and weather changes, the present invention can comprehensively obtain and analyze various data sources, thereby providing more accurate and real-time driving suggestions to ensure the safety and efficiency of train operation.

[0075] Secondly, the present invention is based on the Transformer's cross-modal feature alignment mechanism and the timing modeling of the gated recurrent unit (GRU), which can dynamically track the locomotive's running trajectory and adjust the prompt content according to real-time changes. Traditional driving prompt systems often rely on static rules and lack the ability to perceive and respond to the locomotive's operating status in a dynamic environment in real time. The present invention dynamically analyzes the spatiotemporal model of train operation and combines the driver's historical operation data to continuously evaluate and adjust the operating suggestions during operation, and make personalized recommendations for different operating requirements and terrain changes. In this way, not only the accuracy of driving prompts is improved, but also the driver's operational flexibility and adaptability are significantly enhanced.

[0076] In terms of equipment early warning, the present invention can capture the potential causal relationship between equipment and provide accurate early warning by introducing causal enhancement variational neural network and causal inference algorithm. By constructing and optimizing the equipment causal relationship graph, the failure risk of equipment can be identified in advance, and a detailed early warning report can be generated to help operators take emergency measures in time to prevent major accidents. This early warning method based on causal inference has higher accuracy and operability than traditional technologies, and can significantly improve the prevention capability of locomotives.

[0077] In addition, the neural-symbolic hybrid reasoning engine adopted by the present invention realizes the organic combination of explicit rules and implicit associations by combining the symbolic logic module and the graph neural network module, so that the driving prompts not only have the ability of rule-based reasoning, but also can extract deep implicit laws from complex dynamic data. This method enables the system to make more reasonable judgments when facing unknown or complex operating environments, and provide control suggestions that meet the actual situation. Through the message transmission mechanism of the graph neural network, the system can dynamically adjust the correlation characteristics of the operating status, continuously optimize the driving prompts, and improve the timeliness and accuracy of the prompts.

[0078] Finally, the present invention converts complex dynamic analysis results into voice and graphic prompts that are easy for drivers to understand by combining natural language generation technology. This design not only optimizes the expression of driving prompts, but also improves the driver's acceptance and execution of the prompt content, thereby improving the safety and efficiency of train operations. Especially in emergency situations, the system can quickly generate emergency operation suggestions and promptly remind the driver through a combination of voice and graphic, ensuring that a quick response can be made in emergencies to prevent accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0080] Figure 1 This is an overall flow chart of a method for compiling driving reminder data for a heavy-load locomotive proposed by the present invention;

[0081] Figure 2 A schematic diagram of a neural-symbolic hybrid reasoning engine for a method of compiling heavy-load locomotive driving prompt data proposed in the present invention. DETAILED DESCRIPTION

[0082] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0083] refer to Figure 1 A method for compiling driving reminder data for a heavy-load locomotive comprises the following steps:

[0084] S1, collecting multimodal data and preprocessing to obtain preprocessed multimodal data;

[0085] S2. Using a Transformer-based cross-modal feature alignment mechanism to perform time series alignment and correlation analysis on the pre-processed multimodal data, extract cross-modal features in the time series, and generate a multi-dimensional feature representation;

[0086] S3. Based on the multi-dimensional feature representation, a gated recurrent unit is used to perform time series modeling on the dynamic running trajectory, and a spatiotemporal model of train operation is constructed by combining a three-dimensional terrain model and line characteristics;

[0087] S4, inputting the spatiotemporal model of the train operation into a neural-symbolic hybrid reasoning engine, dynamically analyzing the train operation data, processing the explicit rule data through a symbolic logic module and generating a logic tree, and extracting implicit association features using a graph neural network;

[0088] S5. Based on the dynamic analysis results and the driver's historical operation data, the soft-action actor-critic algorithm is used to predict the manipulation behavior, dynamically adjust the manipulation prompt rules and generate personalized manipulation suggestions;

[0089] S6. Using the personalized operation suggestions and real-time operation data, a causal enhanced variational neural network is constructed, and a device causal relationship diagram is generated through a causal inference algorithm to perform device early warning;

[0090] S7. Generate real-time emergency operation suggestions based on equipment warning results and operation rules, and output emergency treatment plans;

[0091] S8. Combining the emergency response plan with the spatiotemporal model of train operation, natural language generation technology is used to generate voice and graphic prompts, and the driving prompt information is provided to the driver through a display screen.

[0092] In this implementation, the multimodal data includes video data and audio data, and the preprocessing includes denoising and outlier processing.

[0093] In this implementation, S2 specifically includes:

[0094] S21, time-slicing the pre-processed multimodal data, and dividing the video data and the audio data into time windows T w Divide into several time segments to generate a multimodal time series of length L;

[0095] S22. Extracting spatial features of video data using convolutional neural network F v (t), use Mel frequency cepstral coefficients to extract the frequency characteristics F of audio data a (t);

[0096] S23, using the Transformer-based cross-modal feature alignment mechanism, the spatial features F of the video data are aligned v (t) and the frequency characteristics of the audio data F a (t) Input to the Transformer model and generate the query matrix Q, key matrix K and value matrix V through the embedding layer;

[0097] S24. Use the self-attention mechanism to align cross-modal features in time and calculate the correlation weights between modalities:

[0098]

[0099] Among them, Attention represents the self-attention mechanism, Softmax represents the normalization function, and d k Represents the dimension of the key matrix; extracts the correlation features between each mode in the time series;

[0100] S25. Align cross-modal features based on the multi-head attention mechanism, fuse multimodal data within the time step, calculate the global correlation features between modalities, and generate a fusion feature representation at the time step level;

[0101] S26. The aligned cross-modal fusion feature representation is further refined through the fully connected layer of the Transformer model to generate a multi-dimensional feature representation F(t) with a dimension of [T, D], where T represents the number of time steps and D represents the feature dimension after fusion.

[0102] In this implementation, S3 specifically includes:

[0103] S31, segmenting the generated multidimensional feature representation F(t) according to the number of time steps T, and extracting the dynamic features of the time series as the input of time series modeling;

[0104] S32, using a gated recurrent unit to model the dynamic characteristics of the time series, and initializing the hidden state of the gated recurrent unit to a zero vector;

[0105] For each time step t, the input features and the previous hidden state are used to calculate the update gate and reset gate:

[0106] z t =σ(W z ·[h t-1 ,f t ]+b z );

[0107] r t =σ(W r ·[h t-1 ,f t ]+b r );

[0108] Among them, z t represents the update gate, r t represents the reset gate, σ represents the activation function, W z represents the weight matrix of the update gate, W r represents the weight matrix of the reset gate, bz represents the bias term of the update gate, b r represents the bias term of the reset gate, h t-1 represents the hidden state of the previous time step, f t Represents the multimodal input features at the current time step t;

[0109] Compute candidate hidden states and current hidden states:

[0110]

[0111] in, represents the candidate hidden state, h t represents the hidden state of the current time step t, tanh represents the hyperbolic tangent function, W h The weight matrix representing the candidate hidden state, b h represents the bias term of the candidate hidden state, ⊙ represents the element-wise multiplication operation;

[0112] S33, generate the hidden state h for each time step t , obtain the hidden state sequence H of the dynamic running trajectory, which is used to characterize the dynamic running trajectory of the train and output the time series modeling results;

[0113] S34, establishing terrain point cloud data according to the terrain data G(x, y, z) of the train running area, wherein (x, y, z) represents the three-dimensional coordinates of the terrain, x represents the horizontal position coordinate, y represents the vertical position coordinate, and z represents the height position coordinate;

[0114] S35, generating a comprehensive grid model by combining the line feature data with the terrain point cloud data, optimizing the comprehensive grid model, eliminating redundant terrain information, and generating a three-dimensional terrain model;

[0115] S36, mapping the hidden state sequence H of the dynamic running trajectory to the three-dimensional terrain model, and constructing a spatiotemporal model of train operation, wherein the spatiotemporal model of train operation includes a trajectory sequence of the running trajectory and a corresponding three-dimensional spatial position.

[0116] In this implementation manner, the S4 specifically includes:

[0117] S41, inputting the generated spatiotemporal model of train operation into a neural-symbolic hybrid reasoning engine, wherein the neural-symbolic hybrid reasoning engine includes a symbolic logic module, a graph neural network module, and a task offloading mechanism, for dynamically analyzing train operation data;

[0118] S42, using the symbolic logic module to process the explicit rules in the input data, extracting key rule data of train operation, the key rule data including phase separation distance, speed limit and slope characteristics, constructing a rule set, and generating a hierarchical symbolic logic tree based on the rule set;

[0119] S43, recursively analyzing the symbolic logic tree through the symbolic logic module, calculating the applicability of the explicit rule in combination with the input data, dynamically generating an explicit rule reasoning result, and outputting a rule set applicable to the current train operation state;

[0120] S44, using the graph neural network module to extract implicit correlation features in the spatiotemporal model of train operation, constructing the time step features of train operation and the terrain and line features as nodes and edges to form a graph structure, and extracting the correlation between time steps and the potential impact of the operating environment through the message passing mechanism of the graph;

[0121] S45, in the graph neural network module, by updating multi-layer node features and adjusting edge weights, extracting implicit correlation information in the train operation data, including the dynamic change trend of the operation track and the potential relationship between the operation environment, and generating implicit correlation feature analysis results;

[0122] S46. Through the task offloading mechanism, the reasoning tasks are dynamically allocated to the symbolic logic module or the graph neural network module according to the attributes of the input data and the analysis requirements, and the explicit rule reasoning results and the implicit association feature analysis results are integrated. The dynamic analysis results are output by the neural-symbolic hybrid reasoning engine to characterize the current operating status and operation requirements of the train.

[0123] In this implementation manner, S5 specifically includes:

[0124] S51, inputting the generated dynamic analysis result driver's historical operation data into the soft actor-critic algorithm model, wherein the dynamic analysis result includes a list of current operating states and manipulation requirements, and the historical operation data includes the driver's operating behavior records and corresponding operating environment characteristics;

[0125] S52. In the soft-actor actor-critic algorithm, define the environment state, action space, and reward function, where the environment state is represented by S t , the action space is represented by A t , the reward function is expressed as R(S t ,A t ):

[0126] R(S t ,A t )=-(∥S t -S target ∥+∥A t-A target ∥);

[0127] Among them, S target represents the target state, A target represents the target action, ∥·∥ represents the norm of the vector;

[0128] S53. Using Strategy Network π θ (A t |S t ) predicts the manipulation behavior according to the current environment state and generates the optimal manipulation action for the current time step, where π θ (A t |S t ) is the strategy learned by the policy network parameter θ, indicating that in state S t Next, execute action A t The probability distribution of

[0129] S54. Use the critic network to evaluate the manipulation behavior generated by the policy network. The critic network minimizes the loss function Update Parameters

[0130]

[0131] in, represents the loss function, Indicates state S t 、Action A t and instant reward R t The expected value under , γ represents the discount factor, represents the current critic network parameters, φ ′ represents the target critic network parameters, represents the action value predicted by the current critic network, represents all possible actions A of the target critic network at the next time step t+1 t+1 The maximum expected action value of

[0132] S55, dynamically adjusting the operation prompt rules based on the strategy network, combining the operation requirements of the train with the historical operation data, updating the prompt rules to adapt to the current operating environment, and generating the operation prompt content;

[0133] S56. Generate personalized manipulation suggestions according to the adjusted manipulation prompt rules.

[0134] In this implementation manner, S6 specifically includes:

[0135] S61. Constructing a causal enhancement variational neural network based on the personalized operation suggestions and real-time operation data, wherein the causal enhancement variational neural network is used to capture the potential causal relationship between the operation behavior and the equipment state, and inputs the real-time dynamic parameters and historical operation data of the train;

[0136] S62. In the causal enhancement variational neural network, the causal relationship between devices is identified and established through the causal inference algorithm, the influence path between the device and the operating parameter is modeled, and the device causal relationship diagram is generated;

[0137] S63, combining the equipment cause-effect relationship diagram with historical operation data, and using variational reasoning methods to infer the equipment status;

[0138] S64, updating the relationship weights in the device causal relationship graph, and optimizing the device causal relationship graph by iteratively adjusting the causal relationship between devices;

[0139] S65. Generate an equipment early warning report based on the optimized equipment cause-effect relationship diagram combined with real-time operation data.

[0140] refer to Figure 2 , a system for compiling driving prompt data for heavy-duty locomotives, comprising a hardware part and a software part, wherein the hardware part comprises a display screen, a base, a communication line and a speaker; the software part comprises:

[0141] A data acquisition module, used to collect and pre-process multimodal data;

[0142] Data alignment and analysis module, used to perform cross-modal feature alignment and time series correlation analysis to generate multi-dimensional feature representation;

[0143] The time series modeling and terrain construction module is used to perform time series modeling based on multi-dimensional feature representation and to build a spatiotemporal model of train operation by combining the three-dimensional terrain model and line features;

[0144] Neural symbolic reasoning module, used to analyze train operation data and extract explicit rule data and implicit association features;

[0145] Operation prediction module, which is used to predict manipulation behavior based on analysis results and historical data and generate personalized manipulation suggestions;

[0146] The equipment early warning module is used to generate equipment causal relationship diagrams through causal inference algorithms for equipment early warning;

[0147] Emergency handling module, used to generate and output real-time emergency operation suggestions;

[0148] The prompt information generation module is used to generate and output voice and graphic prompt information to the driver.

[0149] Embodiment 1:

[0150] In order to verify the feasibility of the present invention in implementation, the present invention is applied to a heavy-load locomotive. The locomotive is currently running on a railway freight line in China, with a total length of 1,000 kilometers. The load is heavy during driving, especially in mountainous areas and complex terrains, and the locomotive needs to be frequently adjusted during operation. In this environment, the traditional driving prompt system only relies on simple speed and distance monitoring, and fails to fully consider the comprehensive impact of multiple data such as real-time terrain changes, locomotive status, and driver's historical operations, resulting in frequent operating errors such as brake failure and over-acceleration, affecting driving safety and efficiency.

[0151] In order to improve the driving safety of heavy-load locomotives on this line, this embodiment applies the dynamic analysis and prediction method based on multimodal data proposed in the present invention. Specifically, it adopts the Transformer-based cross-modal feature alignment mechanism, the gated recurrent unit (GRU) for time series modeling, and the neural symbolic hybrid reasoning engine for dynamic analysis, and combines the real-time terrain model and locomotive dynamic parameters to generate personalized operation suggestions and timely generate emergency operation plans.

[0152] First, the driver's operating behavior is captured through video and audio data. After all data is preprocessed, the Transformer model is used for time series alignment and feature extraction. Through comprehensive analysis of cross-modal data, the system can accurately model the vehicle condition from multiple dimensions.

[0153] Secondly, the system uses the gated recurrent unit (GRU) to model the dynamic running trajectory, and combines the three-dimensional terrain model and line characteristics to build a spatiotemporal model of train operation. On this basis, the neural symbolic hybrid reasoning engine performs dynamic analysis based on explicit rules and implicit features to generate operational recommendations and prediction results for the current train.

[0154] In addition, the system can combine the driver's historical operation data, predict possible manipulation behaviors through the soft actor-critic algorithm, and adjust the manipulation rules according to the prediction results. Finally, based on these personalized manipulation suggestions, the system can generate real-time emergency operation plans and convey the information to the driver through display screens and voice prompts.

[0155] Table 1 Comparison of experimental data between the conventional system and the system of the present invention in driving prompts for heavy-loaded locomotives

[0156] Compare Projects Traditional systems System of the present invention Improvement percentage (%) Operation error rate 15 times / hour 2 times / hour 86.7 Operational safety score 78 96 23.5 Personalized control suggestion responsiveness 85％ 98％ 13.7 Braking system fault prediction accuracy 65％ 92％ 27 Power system fault prediction accuracy 70％ 89％ 19 Fault warning lead time 55 minutes 20 minutes 300

[0157] By analyzing the results in Table 1 above, it can be clearly seen that the present invention has significant improvements in multiple key performance indicators compared to the traditional system.

[0158] First, in terms of the operating error rate, the traditional system has an error rate of 15 times per hour, while the error rate of the present invention is only 2 times per hour, a reduction of up to 86.7%. This data reflects that the present invention can effectively reduce the driver's operating errors through intelligent personalized control suggestions and dynamic analysis, thereby improving the safety and reliability of driving. Reducing operating errors can not only reduce the accident rate, but also improve the overall operating efficiency of the locomotive.

[0159] Secondly, the comparison of operational safety scores shows that the safety score of the present invention is 96 points, while the traditional system is 78 points, an increase of 23.5%. This shows that the present invention can provide more accurate safety tips and control suggestions in complex operating environments, thereby ensuring that drivers can make safer decisions based on real-time data, further improving the safety of train operations.

[0160] In terms of personalized operation suggestion responsiveness, the traditional system has a responsiveness of 85%, while the present invention can achieve a responsiveness of 98%, an improvement of 13.7%. This result shows that the present invention can provide more accurate personalized suggestions based on the actual operating status of the train and the driver's historical operating behavior, allowing the driver to respond in a shorter time and thus optimize driving operations.

[0161] In addition, the present invention also demonstrates significant advantages in fault prediction. The accuracy of power system fault prediction is increased from 70% of the traditional system to 89% of the present invention, an increase of 19%. This shows that the present invention can more accurately predict equipment failures and detect potential fault problems in advance by introducing advanced technologies such as causal enhancement variational neural networks and graph neural networks, providing an important basis for fault prevention and maintenance, reducing the risk of equipment downtime and reducing maintenance costs.

[0162] The most outstanding is the comparison of the advance time of fault warning. The warning time of the traditional system is 5 minutes, while the present invention can issue a warning 20 minutes in advance, which is an increase of 300%. The extension of the advance warning time allows operators to have sufficient time to take emergency measures to avoid accidents. Especially in the face of complex heavy-load locomotive operating environment, the advance warning time undoubtedly provides greater protection for train safety.

[0163] It can be seen that the present invention is superior to the traditional system in many dimensions, especially in terms of operation error rate, operation safety, personalized control suggestion responsiveness, equipment warning accuracy and warning time. By adopting intelligent technology based on deep learning, neural network and causal reasoning, the present invention can effectively improve the safety and operation efficiency of heavy-load locomotives, reduce the accident rate, and provide more reliable and intelligent support for railway transportation.

[0164] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for compiling driving reminder data for a heavy-load locomotive, characterized in that: The steps include: S1, collecting multimodal data and preprocessing to obtain preprocessed multimodal data; S2. Using a Transformer-based cross-modal feature alignment mechanism to perform time series alignment and correlation analysis on the pre-processed multimodal data, extract cross-modal features in the time series, and generate a multi-dimensional feature representation; S3. Based on the multi-dimensional feature representation, a gated recurrent unit is used to perform time series modeling on the dynamic running trajectory, and a spatiotemporal model of train operation is constructed by combining a three-dimensional terrain model and line characteristics; S4, inputting the spatiotemporal model of the train operation into a neural-symbolic hybrid reasoning engine, dynamically analyzing the train operation data, processing the explicit rule data through a symbolic logic module and generating a logic tree, and extracting implicit association features using a graph neural network; S5. Based on the dynamic analysis results and the driver's historical operation data, the soft-action actor-critic algorithm is used to predict the manipulation behavior, dynamically adjust the manipulation prompt rules and generate personalized manipulation suggestions; S6. Using the personalized operation suggestions and real-time operation data, a causal enhanced variational neural network is constructed, and a device causal relationship diagram is generated through a causal inference algorithm to perform device early warning; S7. Generate real-time emergency operation suggestions based on equipment warning results and operation rules, and output emergency treatment plans; S8. Combining the emergency response plan with the spatiotemporal model of train operation, natural language generation technology is used to generate voice and graphic prompts, and the driving prompt information is provided to the driver through a display screen.

2. A method for compiling driving reminder data for a heavy-load locomotive according to claim 1, characterized in that: The multimodal data includes video data and audio data, and the preprocessing includes denoising and outlier processing.

3. The method for compiling driving reminder data for a heavy-load locomotive according to claim 1, characterized in that: The S2 specifically includes: S21, time-slicing the pre-processed multimodal data, and dividing the video data and the audio data into time windows T w Divide into several time segments to generate a multimodal time series of length L; S22. Extracting spatial features of video data using convolutional neural network F v (t), use Mel frequency cepstral coefficients to extract the frequency characteristics F of audio data a (t); S23, using the Transformer-based cross-modal feature alignment mechanism, the spatial features F of the video data are aligned v (t) and the frequency characteristics of the audio data F a (t) Input to the Transformer model and generate the query matrix Q, key matrix K and value matrix V through the embedding layer; S24. Use the self-attention mechanism to align cross-modal features in time and calculate the correlation weights between modalities: Among them, Attention represents the self-attention mechanism, Softmax represents the normalization function, and d k Represents the dimension of the key matrix; extracts the correlation features between each mode in the time series; S25. Align cross-modal features based on the multi-head attention mechanism, fuse multimodal data within the time step, calculate the global correlation features between modalities, and generate a fusion feature representation at the time step level; S26, the aligned cross-modal fusion feature representation is further refined through the fully connected layer of the Transformer model to generate a multi-dimensional feature representation F ( t ) , the dimension is [T,D], where T represents the number of time steps and D represents the feature dimension after fusion.

4. The method for compiling driving reminder data for a heavy-load locomotive according to claim 1, characterized in that: The S3 specifically includes: S31, the generated multi-dimensional feature representation F ( t) Segment by time step number T and extract the dynamic features of the time series as the input of time series modeling; S32, using a gated recurrent unit to model the dynamic characteristics of the time series, and initializing the hidden state of the gated recurrent unit to a zero vector; For each time step t, the input features and the previous hidden state are used to calculate the update gate and reset gate: z t =σ(W z ·[h t-1 ,f t ]+b z ); r t =σ(W r [h t-1 ,f t ]+b r ); Among them, z t represents the update gate, r t represents the reset gate, σ represents the activation function, W z represents the weight matrix of the update gate, W r represents the weight matrix of the reset gate, b z represents the bias term of the update gate, b r represents the bias term of the reset gate, h t-1 represents the hidden state of the previous time step, f t Represents the multimodal input features at the current time step t; Compute candidate hidden states and current hidden states: in, represents the candidate hidden state, h t represents the hidden state of the current time step t, tanh represents the hyperbolic tangent function, W h The weight matrix representing the candidate hidden state, b h represents the bias term of the candidate hidden state, ⊙ represents the element-wise multiplication operation; S33, generate the hidden state h for each time step t , obtain the hidden state sequence H of the dynamic running trajectory, which is used to characterize the dynamic running trajectory of the train and output the time series modeling results; S34, establishing terrain point cloud data according to the terrain data G(x, y, z) of the train running area, wherein (x, y, z) represents the three-dimensional coordinates of the terrain, x represents the horizontal position coordinate, y represents the vertical position coordinate, and z represents the height position coordinate; S35, generating a comprehensive grid model by combining the line feature data with the terrain point cloud data, optimizing the comprehensive grid model, eliminating redundant terrain information, and generating a three-dimensional terrain model; S36, mapping the hidden state sequence H of the dynamic running trajectory to the three-dimensional terrain model, and constructing a spatiotemporal model of train operation, wherein the spatiotemporal model of train operation includes a trajectory sequence of the running trajectory and a corresponding three-dimensional spatial position.

5. The method for compiling driving reminder data for a heavy-load locomotive according to claim 1, characterized in that: The S4 specifically includes: S41, inputting the generated spatiotemporal model of train operation into a neural-symbolic hybrid reasoning engine, wherein the neural-symbolic hybrid reasoning engine includes a symbolic logic module, a graph neural network module, and a task offloading mechanism, for dynamically analyzing train operation data; S42, using the symbolic logic module to process the explicit rules in the input data, extracting key rule data of train operation, the key rule data including phase separation distance, speed limit and slope characteristics, constructing a rule set, and generating a hierarchical symbolic logic tree based on the rule set; S43, recursively analyzing the symbolic logic tree through the symbolic logic module, calculating the applicability of the explicit rule in combination with the input data, dynamically generating an explicit rule reasoning result, and outputting a rule set applicable to the current train operation state; S44, using the graph neural network module to extract implicit correlation features in the spatiotemporal model of train operation, constructing the time step features of train operation and the terrain and line features as nodes and edges to form a graph structure, and extracting the correlation between time steps and the potential impact of the operating environment through the message passing mechanism of the graph; S45, in the graph neural network module, by updating multi-layer node features and adjusting edge weights, extracting implicit correlation information in the train operation data, including the dynamic change trend of the operation track and the potential relationship between the operation environment, and generating implicit correlation feature analysis results; S46. Through the task offloading mechanism, the reasoning tasks are dynamically allocated to the symbolic logic module or the graph neural network module according to the attributes of the input data and the analysis requirements, and the explicit rule reasoning results and the implicit association feature analysis results are integrated. The dynamic analysis results are output by the neural-symbolic hybrid reasoning engine to characterize the current operating status and operation requirements of the train.

6. The method for compiling heavy-load locomotive driving reminder data according to claim 1, characterized in that: The S5 specifically includes: S51, inputting the generated dynamic analysis result driver's historical operation data into the soft actor-critic algorithm model, wherein the dynamic analysis result includes a list of current operating states and manipulation requirements, and the historical operation data includes the driver's operating behavior records and corresponding operating environment characteristics; S52. In the soft-actor actor-critic algorithm, define the environment state, action space, and reward function, where the environment state is represented by S t , the action space is represented by A t , the reward function is expressed as R(S t ,A t ): R(S t ,A t )=-(∥S t -S target ∥+∥A t -A target ∥); Among them, S target represents the target state, A target represents the target action, ∥·∥ represents the norm of the vector; S53. Using Strategy Network π θ ( A t |S t ) predicts the manipulation behavior according to the current environment state and generates the optimal manipulation action for the current time step, where π θ (A t |S t ) is the strategy learned by the policy network parameter θ, indicating that in state S t Next, execute action A t The probability distribution of S54. Use the critic network to evaluate the manipulation behavior generated by the policy network. The critic network minimizes the loss function Update Parameters in, represents the loss function, Indicates state S t 、Action A t and instant reward R t The expected value under , γ represents the discount factor, represents the current critic network parameters, φ ′ represents the target critic network parameters, represents the action value predicted by the current critic network, represents all possible actions A of the target critic network at the next time step t+1 t+1 The maximum expected action value of S55, dynamically adjusting the operation prompt rules based on the strategy network, combining the operation requirements of the train with the historical operation data, updating the prompt rules to adapt to the current operating environment, and generating the operation prompt content; S56. Generate personalized manipulation suggestions according to the adjusted manipulation prompt rules.

7. The method for compiling driving reminder data for a heavy-load locomotive according to claim 1, characterized in that: The S6 specifically includes: S61. Constructing a causal enhancement variational neural network based on the personalized operation suggestions and real-time operation data, wherein the causal enhancement variational neural network is used to capture the potential causal relationship between the operation behavior and the equipment state, and inputs the real-time dynamic parameters and historical operation data of the train; S62. In the causal enhancement variational neural network, the causal relationship between devices is identified and established through the causal inference algorithm, the influence path between the device and the operating parameter is modeled, and the device causal relationship diagram is generated; S63, combining the equipment cause-effect relationship diagram with historical operation data, and using variational reasoning methods to infer the equipment status; S64, updating the relationship weights in the device causal relationship graph, and optimizing the device causal relationship graph by iteratively adjusting the causal relationship between devices; S65. Generate an equipment early warning report based on the optimized equipment cause-effect relationship diagram combined with real-time operation data.

8. A system for compiling driving reminder data for a heavy-loaded locomotive, executing the method for compiling driving reminder data for a heavy-loaded locomotive according to any one of claims 1 to 7, characterized in that: It includes hardware and software parts. The hardware part includes a display screen, a base, a communication line and a speaker. The software part includes: A data acquisition module, used to collect and pre-process multimodal data; Data alignment and analysis module, used to perform cross-modal feature alignment and time series correlation analysis to generate multi-dimensional feature representation; The time series modeling and terrain construction module is used to perform time series modeling based on multi-dimensional feature representation and to build a spatiotemporal model of train operation by combining the three-dimensional terrain model and line features; Neural symbolic reasoning module, used to analyze train operation data and extract explicit rule data and implicit association features; Operation prediction module, which is used to predict manipulation behavior based on analysis results and historical data and generate personalized manipulation suggestions; The equipment early warning module is used to generate equipment causal relationship diagrams through causal inference algorithms for equipment early warning; Emergency handling module, used to generate and output real-time emergency operation suggestions; The prompt information generation module is used to generate and output voice and graphic prompt information to the driver.

Images