Fault diagnosis method based on train multi-modal operation data and related equipment

Abstract

The embodiment of the application discloses a fault diagnosis method based on train multi-modal operation data and related equipment, the method comprises the following steps: obtaining the multi-modal operation data of the train and the real-time operation condition parameters corresponding to the multi-modal operation data; synchronously aligning the multi-modal operation data according to time to obtain multi-modal synchronous data, inputting the multi-modal synchronous data and the real-time operation condition parameters into a preset feature interaction network to obtain a multi-modal joint feature vector and weight data of the multi-modal joint feature vector; obtaining preliminary diagnosis evidence based on the multi-modal joint feature vector, and fusing the preliminary diagnosis evidence by using a confidence factor corresponding to the weight data to obtain a final diagnosis result.

Description

Technical Field

[0001] This invention belongs to the field of intelligent operation and maintenance and condition monitoring technology for rail transit vehicles, specifically relating to a fault diagnosis method and related equipment based on multimodal train operation data. Background Technology

[0002] As rail transit systems develop towards high speed, heavy load, and intelligence, the requirements for train operation safety and maintenance efficiency are increasing. Fault diagnosis technology has evolved from offline analysis based on single signals to online intelligent diagnosis relying on multimodal data fusion.

[0003] In existing technologies, multimodal fusion has been widely applied to train condition monitoring and fault diagnosis. Based on the decision logic of their fusion weights, existing methods can be mainly summarized as follows: Attention-based Feature Fusion (AFF): This method belongs to the "data-driven" paradigm. Its core lies in automatically learning the intrinsic correlations between different modal features using deep neural networks (such as attention mechanisms) and generating fusion weights accordingly. The fusion weights of this method are entirely derived from the statistical characteristics of the input data itself, failing to explicitly incorporate the physical operating conditions (operating conditions) of the train as a decision-making basis. Fixed Model Based on Operating Condition Selection (MSM): This method belongs to the "model switching" paradigm. Its core lies in pre-training multiple independent diagnostic models for different operating conditions (such as low speed, medium speed, and high speed). During online application, the system calls the corresponding pre-trained model for diagnosis based on the real-time determined operating condition category. Although this method considers operating conditions at a macro level, the fusion rules within each sub-model are fixed and unchanging, potentially leading to discontinuous decision-making when switching between different operating condition regions.

[0004] However, the fault characteristics of key train components (such as traction motors and bearings) are strongly correlated with real-time operating conditions such as speed and load. When dealing with complex and ever-changing operating conditions, the weight allocation of the AFF method may rely excessively on the statistical regularities within the data. When operating conditions change drastically, this data correlation may become disconnected from the physical mechanism, leading to inaccurate weight allocation. The MSM method, on the other hand, may experience performance fluctuations during the transition phase of continuous changes in operating conditions due to model switching mismatch or decision jumps. Summary of the Invention

[0005] In view of this, the present invention provides a fault diagnosis method and related equipment based on train multimodal operation data, which is used to solve the problem of unstable performance of existing technologies under all operating conditions (especially when parameters such as speed and load change drastically) due to the decoupling of the fusion weight adjustment strategy from the external physical operating environment (operating conditions) and excessive reliance on the internal statistical correlation of data.

[0006] To achieve one or more of the above objectives or other objectives, in a first aspect, the present invention proposes a fault diagnosis method based on train multimodal operation data, comprising: acquiring train multimodal operation data and real-time operating condition parameters corresponding to the multimodal operation data, wherein the multimodal operation data includes at least: discrete state data generated by the train control network, continuous monitoring data collected by physical sensors deployed on the train, and monitoring data recorded by an image device for train actions, and the real-time operating condition parameters include at least the train's operating speed and traction load level;

[0007] The multimodal operation data is synchronized and aligned according to time to obtain multimodal synchronization data. The multimodal synchronization data and the real-time operation parameters are input into a preset feature interaction network to obtain a multimodal joint feature vector and weight data for constructing the multimodal joint feature vector. Preliminary diagnostic evidence is obtained based on the multimodal joint feature vector and the preset classifier, and the preliminary diagnostic evidence is fused using the confidence factor corresponding to the weight data to obtain the final diagnostic result.

[0008] Optionally, the step of inputting multimodal synchronization data and the real-time operating condition parameters into a preset feature interaction network to obtain a multimodal joint feature vector and weight data for constructing the multimodal joint feature vector includes: Multimodal synchronization data is input into different feature processing branches of the preset feature interaction network according to modal type to obtain feature data of different modes. The preset feature interaction network includes at least a first feature processing branch for processing vibration or current continuous signals, a second feature processing branch for processing image or video data, and a third feature processing branch for processing discrete state data of train network. The real-time operating condition parameters are input into the dynamic weight fusion module of the preset feature interaction network to obtain weight data for constructing the multimodal joint feature vector. The feature data of different modalities are input into the dynamic weight fusion module, so that the dynamic weight fusion module fuses the feature data of different modalities according to the weight data to obtain a multimodal joint feature vector.

[0009] Optionally, the dynamic weight fusion module includes: a dynamic weight allocation layer and a dynamic feature interaction layer. The dynamic weight allocation layer is used to calculate the weight data for constructing the multimodal joint feature vector based on the real-time operating condition parameters using a weight calculation formula. The mathematical representation of the weight calculation formula is as follows:

[0010] in, The speed is one of the real-time operating condition parameters. This refers to the load level in the real-time operating condition parameters. and These are the trainable weight matrices corresponding to the speed and load levels, respectively. For the bias term related to mode m, The preset nonlinear activation function is used. It is a normalized exponential function; The dynamic feature interaction layer is used to fuse feature data of different modalities according to the weight data to obtain a multimodal joint feature vector.

[0011] Optionally, the step of inputting feature data from different modalities into the dynamic weight fusion module, so that the dynamic weight fusion module fuses the feature data from different modalities according to the weight data to obtain a multimodal joint feature vector, includes: The feature data of different modalities are input into the dynamic weight fusion module so that the dynamic weight fusion module maps the feature data of different modalities to the same complex vector space, thereby obtaining the mapping values ​​of the feature data of different modalities. The dynamic weight fusion module fuses the mapping values ​​according to the weight data to obtain a multimodal joint feature vector.

[0012] Optionally, before the step of fusing the preliminary diagnostic evidence using confidence factors corresponding to the weighted data, the method further includes: The confidence factor corresponding to the weighted data is calculated based on the weighted data and the confidence factor calculation formula. The mathematical representation of the confidence factor calculation formula is as follows:

[0013] in, This is an adjustable attenuation coefficient.

[0014] Optionally, the method further includes: A diagnostic report is generated based on the final diagnostic results and the preset fault knowledge graph, including the fault type, fault location, and handling strategy.

[0015] Secondly, the present invention provides a fault diagnosis device based on train multimodal operation data, the device comprising: The data acquisition module is used to acquire multimodal operation data of the train and real-time operating condition parameters corresponding to the multimodal operation data. The multimodal operation data includes at least: discrete state data generated by the train control network, continuous monitoring data collected by physical sensors deployed on the train, and monitoring data recorded by image equipment for train actions. The real-time operating condition parameters include at least the train's operating speed and traction load level. The data fusion module is used to perform time-based synchronization and alignment processing on the multimodal operation data to obtain multimodal synchronization data. The multimodal synchronization data and the real-time operation parameters are input into a preset feature interaction network to obtain a multimodal joint feature vector and weight data for constructing the multimodal joint feature vector. The fault diagnosis module is used to obtain preliminary diagnostic evidence based on the multimodal joint feature vector and a preset classifier, and to fuse the preliminary diagnostic evidence using the confidence factor corresponding to the weight data to obtain the final diagnostic result.

[0016] Thirdly, this application provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the fault diagnosis method based on train multimodal operation data as described above.

[0017] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the fault diagnosis method based on train multimodal operation data as described above.

[0018] Implementing the embodiments of the present invention will have the following beneficial effects: A trainable weight mapping model is constructed with the real-time physical condition of the train as the only external driving signal. This fundamentally transforms the generation logic of the fusion weights from "implicit, passive adjustment" that depends on the internal statistical correlation of the data to "explicit, active mapping" that is directly driven by the external physical environment. This couples the entire fusion diagnostic strategy with the real physical laws of equipment operation, improving accuracy and robustness. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] in: Figure 1 This is a flowchart of a fault diagnosis method based on train multimodal operation data provided in an embodiment of this application; Figure 2 This is a flowchart of another fault diagnosis method based on train multimodal operation data provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of a fault diagnosis device based on train multimodal operation data provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a storage medium provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] like Figure 1 As shown in the figure, this application provides a fault diagnosis method based on train multimodal operation data, including: S101. Acquire multimodal operation data of the train and real-time operating condition parameters corresponding to the multimodal operation data, wherein the multimodal operation data includes at least: discrete state data generated by the train control network, continuous monitoring data collected by physical sensors deployed on the train, and monitoring data recorded by image equipment for train actions, and the real-time operating condition parameters include at least the train's operating speed and traction load level. S102. The multimodal operation data is synchronized and aligned according to time to obtain multimodal synchronization data. The multimodal synchronization data and the real-time operation parameters are input into a preset feature interaction network to obtain a multimodal joint feature vector and weight data for constructing the multimodal joint feature vector. S103. Based on the multimodal joint feature vector and the preset classifier, preliminary diagnostic evidence is obtained, and the confidence factor corresponding to the weight data is used to fuse the preliminary diagnostic evidence to obtain the final diagnostic result.

[0023] For example, such as Figure 2As shown, a fault diagnosis method based on train multimodal operation data uses real-time physical operating condition parameters of the train as external driving variables. Through a learnable weight mapping model, it dynamically generates fusion weights for multimodal features, thereby achieving adaptive coordination between the diagnostic strategy and the physical operating environment. Specifically, it includes: Control messages are acquired and parsed through the train backbone network to form discrete state data; continuous signals such as vibration and current from key rotating components (e.g., traction motors) are collected through physical sensors; and visual data on the operation process and equipment status are acquired through onboard image / video equipment. Timestamp alignment is performed based on the network synchronization clock, and preprocessing such as filtering, noise reduction, and format standardization is carried out.

[0024] Deep feature extraction of multimodal data: Feature extraction is performed using adaptive techniques based on the physical characteristics of different modal data.

[0025] Continuous signal characteristics ( ): The vibration and current signals are transformed by time-frequency to obtain the time-frequency spectrum, which is then input into a CNN neural network to extract high-dimensional features.

[0026] Visual features ( , ): Using a pre-trained visual model, operation sequence and state semantic features are extracted from driver's cab video and equipment appearance video, respectively.

[0027] Discrete state characteristics ( Based on a predefined component association topology, relevant network signals are filtered and encoded to construct structured feature vectors.

[0028] The extracted feature vector is then compared with the train's real-time operating parameters (speed) used as an external driving signal. and load settings The network, which shares features with other systems, performs key operations: Dynamic weight generation: Based on operating parameters and Adaptive fusion weights for each modality are generated by calculating a trainable weight mapping model. .

[0029] Feature interaction fusion: Utilizing The features represented by complex numbers are weighted, interacted, and fused, with the complex amplitudes dynamically modulated by operating parameters to embed operating information into the feature layer. The final output is a joint feature vector F that integrates physical operating information and has stronger discriminative power.

[0030] Step 4: Closed-loop decision-making: To maintain consistency in decision-making logic, the adaptive mechanism is extended from the front end to the back end.

[0031] The joint feature F is fed into the classifier to obtain preliminary diagnostic evidence for each modality.

[0032] Introducing weights for integration with the front end The associated confidence modulation factor weights the evidence.

[0033] Based on improved evidence theory, weighted evidence is fused, and the fusion result is matched and reasoned with a predefined fault knowledge graph to output the final diagnostic conclusion, forming a closed-loop adaptive process from perception to decision-making.

[0034] In one possible implementation, the step of inputting multimodal synchronization data and the real-time operating condition parameters into a preset feature interaction network to obtain a multimodal joint feature vector and weight data for constructing the multimodal joint feature vector includes: Multimodal synchronization data is input into different feature processing branches of the preset feature interaction network according to modal type to obtain feature data of different modes. The preset feature interaction network includes at least a first feature processing branch for processing vibration or current continuous signals, a second feature processing branch for processing image or video data, and a third feature processing branch for processing discrete state data of train network. The real-time operating condition parameters are input into the dynamic weight fusion module of the preset feature interaction network to obtain weight data for constructing the multimodal joint feature vector. The feature data of different modalities are input into the dynamic weight fusion module, so that the dynamic weight fusion module fuses the feature data of different modalities according to the weight data to obtain a multimodal joint feature vector.

[0035] In one possible implementation, the dynamic weight fusion module includes: a dynamic weight allocation layer and a dynamic feature interaction layer. The dynamic weight allocation layer is used to calculate weight data for constructing the multimodal joint feature vector based on the real-time operating condition parameters using a weight calculation formula. The mathematical representation of the weight calculation formula is as follows:

[0036] in, The speed is one of the real-time operating condition parameters. This refers to the load level in the real-time operating condition parameters. and These are the trainable weight matrices corresponding to the speed and load levels, respectively. For the bias term related to mode m, The preset nonlinear activation function is used. It is a normalized exponential function; The dynamic feature interaction layer is used to fuse feature data of different modalities according to the weight data to obtain a multimodal joint feature vector.

[0037] In one possible implementation, the step of inputting feature data of different modalities into the dynamic weight fusion module, so that the dynamic weight fusion module fuses the feature data of different modalities according to the weight data to obtain a multimodal joint feature vector, includes: The feature data of different modalities are input into the dynamic weight fusion module so that the dynamic weight fusion module maps the feature data of different modalities to the same complex vector space, thereby obtaining the mapping values ​​of the feature data of different modalities. The dynamic weight fusion module fuses the mapping values ​​according to the weight data to obtain a multimodal joint feature vector.

[0038] In one possible implementation, prior to the step of fusing the preliminary diagnostic evidence using confidence factors corresponding to the weighted data, the method further includes: The confidence factor corresponding to the weighted data is calculated based on the weighted data and the confidence factor calculation formula. The mathematical representation of the confidence factor calculation formula is as follows:

[0039] in, This is an adjustable attenuation coefficient.

[0040] In one possible implementation, the method further includes: A diagnostic report is generated based on the final diagnostic results and the preset fault knowledge graph, including the fault type, fault location, and handling strategy.

[0041] In one possible implementation, such as Figure 3 As shown, the present invention provides a fault diagnosis device based on train multimodal operation data, the device comprising: The data acquisition module 201 is used to acquire multimodal operation data of the train and real-time operating condition parameters corresponding to the multimodal operation data. The multimodal operation data includes at least: discrete state data generated by the train control network, continuous monitoring data collected by physical sensors deployed on the train, and monitoring data recorded by image equipment for train actions. The real-time operating condition parameters include at least the train's operating speed and traction load level. The data fusion module 202 is used to perform time-based synchronization and alignment processing on the multimodal operation data to obtain multimodal synchronization data. The multimodal synchronization data and the real-time operation parameters are input into a preset feature interaction network to obtain a multimodal joint feature vector and weight data for constructing the multimodal joint feature vector. The fault diagnosis module 203 is used to obtain preliminary diagnostic evidence based on the multimodal joint feature vector and a preset classifier, and to fuse the preliminary diagnostic evidence using the confidence factor corresponding to the weight data to obtain the final diagnostic result.

[0042] In one possible implementation, such as Figure 4 As shown in the figure, this application provides an electronic device 300, including: a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it performs the following: acquiring multimodal operation data of a train and real-time operating condition parameters corresponding to the multimodal operation data, wherein the multimodal operation data includes at least: discrete state data generated by a train control network, continuous monitoring data collected by physical sensors deployed on the train, and monitoring data recorded by an image device for train actions, and the real-time operating condition parameters include at least the train's operating speed and traction load level; performing time-synchronized alignment processing on the multimodal operation data to obtain multimodal synchronization data; inputting the multimodal synchronization data and the real-time operating condition parameters into a preset feature interaction network to obtain a multimodal joint feature vector and weight data for constructing the multimodal joint feature vector; obtaining preliminary diagnostic evidence based on the multimodal joint feature vector and a preset classifier, and fusing the preliminary diagnostic evidence using confidence factors corresponding to the weight data to obtain a final diagnostic result.

[0043] In one possible implementation, such as Figure 5 As shown in the figure, this application embodiment provides a computer-readable storage medium 400, on which a computer program 411 is stored. When the computer program 411 is executed by a processor, it performs the following: acquiring multimodal operation data of a train and real-time operating condition parameters corresponding to the multimodal operation data, wherein the multimodal operation data includes at least: discrete state data generated by a train control network, continuous monitoring data collected by physical sensors deployed on the train, and monitoring data recorded by an image device for train actions, and the real-time operating condition parameters include at least the train's operating speed and traction load level; performing time-synchronized alignment processing on the multimodal operation data to obtain multimodal synchronization data; inputting the multimodal synchronization data and the real-time operating condition parameters into a preset feature interaction network to obtain a multimodal joint feature vector and weight data for constructing the multimodal joint feature vector; obtaining preliminary diagnostic evidence based on the multimodal joint feature vector and a preset classifier, and fusing the preliminary diagnostic evidence using confidence factors corresponding to the weight data to obtain a final diagnostic result.

[0044] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0045] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0046] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0047] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0048] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0049] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

[0050] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A fault diagnosis method based on train multimodal operation data, characterized in that, include: The train's multimodal operation data and the corresponding real-time operating condition parameters are acquired. The multimodal operation data includes at least: discrete state data generated by the train control network, continuous monitoring data collected by physical sensors deployed on the train, and monitoring data recorded by image equipment for train actions. The real-time operating condition parameters include at least the train's operating speed and traction load level. The multimodal operation data is synchronized and aligned according to time to obtain multimodal synchronization data. The multimodal synchronization data and the real-time operation parameters are input into a preset feature interaction network to obtain a multimodal joint feature vector and weight data for constructing the multimodal joint feature vector. Preliminary diagnostic evidence is obtained based on the multimodal joint feature vector and the preset classifier, and the preliminary diagnostic evidence is fused using the confidence factor corresponding to the weight data to obtain the final diagnostic result.

2. The fault diagnosis method based on train multimodal operation data as described in claim 1, characterized in that, The step of inputting multimodal synchronization data and real-time operating condition parameters into a preset feature interaction network to obtain a multimodal joint feature vector and weight data for constructing the multimodal joint feature vector includes: Multimodal synchronization data is input into different feature processing branches of the preset feature interaction network according to modal type to obtain feature data of different modes. The preset feature interaction network includes at least a first feature processing branch for processing vibration or current continuous signals, a second feature processing branch for processing image or video data, and a third feature processing branch for processing discrete state data of train network. The real-time operating condition parameters are input into the dynamic weight fusion module of the preset feature interaction network to obtain weight data for constructing the multimodal joint feature vector. The feature data of different modalities are input into the dynamic weight fusion module, so that the dynamic weight fusion module fuses the feature data of different modalities according to the weight data to obtain a multimodal joint feature vector.

3. The fault diagnosis method based on train multimodal operation data as described in claim 2, characterized in that, The dynamic weight fusion module includes a dynamic weight allocation layer and a dynamic feature interaction layer. The dynamic weight allocation layer is used to calculate the weight data for constructing the multimodal joint feature vector based on the real-time operating condition parameters and a weight calculation formula. The mathematical representation of the weight calculation formula is as follows: in, The speed is one of the real-time operating condition parameters. This refers to the load level in the real-time operating condition parameters. and These are the trainable weight matrices corresponding to the speed and load levels, respectively. For the bias term related to mode m, For the preset nonlinear activation function, It is a normalized exponential function; The dynamic feature interaction layer is used to fuse feature data of different modalities according to the weight data to obtain a multimodal joint feature vector.

4. The fault diagnosis method based on train multimodal operation data as described in claim 2, characterized in that, The step of inputting feature data from different modalities into the dynamic weight fusion module, so that the dynamic weight fusion module fuses the feature data from different modalities according to the weight data to obtain a multimodal joint feature vector, includes: The feature data of different modalities are input into the dynamic weight fusion module so that the dynamic weight fusion module maps the feature data of different modalities to the same complex vector space, thereby obtaining the mapping values ​​of the feature data of different modalities. The dynamic weight fusion module fuses the mapping values ​​according to the weight data to obtain a multimodal joint feature vector.

5. The fault diagnosis method based on train multimodal operation data as described in claim 1, characterized in that, Before the step of fusing the preliminary diagnostic evidence using confidence factors corresponding to the weighted data, the method further includes: The confidence factor corresponding to the weighted data is calculated based on the weighted data and the confidence factor calculation formula. The mathematical representation of the confidence factor calculation formula is as follows: in, This is an adjustable attenuation coefficient.

6. The fault diagnosis method based on train multimodal operation data as described in claim 1, characterized in that, The method further includes: A diagnostic report is generated based on the final diagnostic results and the preset fault knowledge graph, including the fault type, fault location, and handling strategy.

7. A fault diagnosis device based on train multimodal operation data, characterized in that, The device includes: The data acquisition module is used to acquire multimodal operation data of the train and real-time operating condition parameters corresponding to the multimodal operation data. The multimodal operation data includes at least: discrete state data generated by the train control network, continuous monitoring data collected by physical sensors deployed on the train, and monitoring data recorded by image equipment for train actions. The real-time operating condition parameters include at least the train's operating speed and traction load level. The data fusion module is used to perform time-based synchronization and alignment processing on the multimodal operation data to obtain multimodal synchronization data. The multimodal synchronization data and the real-time operation parameters are input into a preset feature interaction network to obtain a multimodal joint feature vector and weight data for constructing the multimodal joint feature vector. The fault diagnosis module is used to obtain preliminary diagnostic evidence based on the multimodal joint feature vector and a preset classifier, and to fuse the preliminary diagnostic evidence using the confidence factor corresponding to the weight data to obtain the final diagnostic result.

8. An electronic device, characterized in that, include: The system includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the fault diagnosis method based on train multimodal operation data as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, performs the steps of the fault diagnosis method based on train multimodal operation data as described in any one of claims 1 to 6.

Images