A train parts abnormality probability prediction method based on multimodal data

Abstract

The present invention relates to the technical field of train abnormality prediction, and discloses a method for predicting the abnormality probability of train parts based on multimodal data, comprising: obtaining historical working data of parts to pre-build a conventional function, constructing a static intervention function according to the installation position of the parts and historical multimodal data, constructing a dynamic intervention function of the parts according to the historical multimodal data of the parts, calling the corresponding static intervention function according to the actual installation position and the duration of a special environment, judging whether a special event is in progress according to the actual multimodal data and calling the corresponding dynamic intervention function, and obtaining a prediction function according to the static intervention function, the dynamic intervention function and the conventional function, wherein the prediction function is specifically a function of abnormality probability changing with time, and taking the actual process parameters, actual installation position and actual multimodal data of the parts as inputs of the prediction function to obtain the future predicted failure probability, which is used to infer the future predicted failure probability when the number of samples is insufficient.

Description

Technical Field

[0001] The present invention relates to the technical field of train anomaly prediction, and in particular to a method for predicting the probability of train component anomalies based on multimodal data. Background Art

[0002] With the advancement of rail transit construction, as a multi-disciplinary complex integrated linkage system, the safety and reliability of rail transit facilities and equipment is the basis for the normal operation of the system, and is crucial to ensuring the safety of urban rail transit operations. The performance of key systems or components of the train is directly related to the safety and reliability of vehicle operation. Therefore, timely prediction of faults is the key to ensuring train safety.

[0003] At present, when predicting the abnormal probability of train parts, the information of each part is usually obtained to infer its approximate service life, and then the abnormal probability is inferred. However, this method is limited by the number of samples. When there are sufficient samples, more accurate calculation results can be obtained. However, when the number of samples is insufficient, since the abnormal probability is not affected by a single variable, more accurate prediction data cannot be obtained. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a train component abnormality probability prediction method based on multimodal data, which is used to combine multiple variables to infer the future predicted failure probability when the sample size is insufficient, and the prediction accuracy is high.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A train component abnormality probability prediction method based on multimodal data, comprising:

[0007] Get the historical working data of the parts and pre-build the regular functions.

[0008] Construct static intervention functions based on the installation locations of components and historical multimodal data.

[0009] Construct dynamic intervention functions for components based on their historical multimodal data.

[0010] Calling a corresponding static intervention function according to the actual installation location and the duration of the special environment, judging whether it is in a special event according to the actual multimodal data and calling a corresponding dynamic intervention function,

[0011] A prediction function is obtained according to the static intervention function, the dynamic intervention function and the conventional function, wherein the prediction function is specifically a function of the abnormal probability changing with time,

[0012] The actual process parameters, actual installation locations, and actual multimodal data of components are used as inputs of the prediction function to obtain future predicted failure probabilities.

[0013] Furthermore, the steps of constructing the conventional function include:

[0014] Acquiring the historical working data, wherein the historical working data includes process parameters, model parameters, damage type and service life;

[0015] The historical work data is used as nodes, the corresponding relationships between the historical work data are used as associations, and the probability of the corresponding relationships is used as attributes to establish a knowledge graph.

[0016] Features are extracted from the knowledge graph, and a Bayesian network structure is designed to calculate the conditional probability relationship between various factors to obtain the conventional function.

[0017] Furthermore, the step of constructing the static intervention function according to the installation location includes:

[0018] Get the installation position of the component to define as position information,

[0019] Obtain the conventional predicted abnormal probability of the parts based on the historical working data and the conventional function,

[0020] The actual abnormal occurrence time of the component is obtained and compared with the conventional predicted abnormal probability to obtain a static error, and the correlation between the installation position and the static error is obtained through training, that is, the static intervention function is obtained.

[0021] Furthermore, the step of constructing the dynamic intervention function according to the historical multimodal data includes:

[0022] Acquire historical multimodal data of components, wherein the historical multimodal data includes historical working temperature, historical working humidity, historical working image information, and historical working audio information.

[0023] Obtain the conventional predicted abnormal probability of the parts based on the historical working data and the conventional function,

[0024] The actual abnormal occurrence time of the component is obtained and compared with the conventional predicted abnormal probability to obtain a dynamic error, and the correlation between the historical multimodal data and the dynamic error is obtained through training, that is, the dynamic intervention function is obtained.

[0025] Furthermore, the actual multimodal data is collected based on a multimodal sensor, and the actual multimodal data includes an actual working temperature parameter and an actual working humidity parameter. Whether the environment is in a special environment is determined based on the actual multimodal data, and the static intervention function is corrected based on the duration of the special environment.

[0026] Furthermore, the static intervention function is configured with a function update strategy, and the function update strategy includes

[0027] Get the current working image information at a preset update time interval.

[0028] Identify and determine whether the current working image information contains abnormal features, such as rust, surface damage and discoloration,

[0029] If there are abnormal features, the static intervention function is modified.

[0030] Furthermore, a function learning strategy is also configured, and the function learning strategy includes

[0031] Determine the correlation between environmental variables and the abnormality rate of parts with different process parameters. If, under specific environmental variables, the abnormality rate of parts with specific process parameters is greater than the preset abnormality threshold, then modify the static intervention function of the part under the environmental variables.

[0032] Furthermore, the determination of the special event includes:

[0033] Obtain real-time working sound information, real-time working image information and real-time working video information,

[0034] determining whether the real-time working sound information contains abnormal sound information, determining whether the real-time working image information contains abnormal image information, determining whether the real-time working video information contains abnormal video information,

[0035] When abnormal information appears in two of the real-time working sound information, the real-time working image information and the real-time working video information, it is determined that a special event occurs.

[0036] Beneficial effects of the present invention:

[0037] The present invention uses conventional functions, static intervention functions, and dynamic intervention functions to obtain conventional abnormal probabilities of similar components, and then uses static intervention functions to eliminate static errors, such as static errors caused by installation position and working environment, and uses dynamic intervention functions to eliminate dynamic errors, such as dynamic errors caused by special events. This setting can quickly establish and obtain a prediction function when parameter conditions are insufficient, and can obtain higher accuracy. Its prediction accuracy can be gradually improved as the number of parameters increases. In addition, multimodal sensors are used in this application to collect multimodal data. Multimodal data facilitates comprehensive acquisition of the actual working status of components, improves data reliability, and facilitates timely discovery of abnormal operating conditions of components, and is also used for abnormal probability prediction and abnormal judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the flow chart of the prediction method in the present invention;

[0039] Figure 2 It is a schematic diagram of the steps for constructing a conventional function in the present invention;

[0040] Figure 3 It is a schematic diagram of the steps for constructing the static intervention function in the present invention;

[0041] Figure 4 It is a schematic diagram of the steps for constructing the dynamic intervention function in the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] It should be noted that when a component is referred to as being "fixed to" another component, it may be directly on the other component or there may also be a component centered. When a component is considered to be "connected to" another component, it may be directly connected to the other component or there may also be a component centered. When a component is considered to be "set on" another component, it may be directly set on the other component or there may also be a component centered. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0045] like Figure 1 As shown, a train component abnormality probability prediction method based on multimodal data in this embodiment includes:

[0046] Get the historical working data of the parts and pre-build the regular functions.

[0047] Construct static intervention functions based on the installation locations of components and historical multimodal data.

[0048] Construct dynamic intervention functions for components based on their historical multimodal data.

[0049] Calling a corresponding static intervention function according to the actual installation location and the duration of the special environment, judging whether it is in a special event according to the actual multimodal data and calling a corresponding dynamic intervention function,

[0050] A prediction function is obtained according to the static intervention function, the dynamic intervention function and the conventional function, wherein the prediction function is specifically a function of the abnormal probability changing with time,

[0051] The actual process parameters, actual installation locations, and actual multimodal data of components are used as inputs of the prediction function to obtain future predicted failure probabilities.

[0052] The present invention uses conventional functions, static intervention functions, and dynamic intervention functions to obtain the conventional abnormality probability of similar components, and then uses the static intervention function to eliminate static errors, such as static errors caused by installation position and working environment, and uses the dynamic intervention function to eliminate dynamic errors, such as dynamic errors caused by special events. This setting can quickly establish and obtain a prediction function when parameter conditions are insufficient, and can obtain higher accuracy. As the number of parameters increases, its prediction accuracy can be gradually improved.

[0053] For further information, please refer to Figure 2 , the construction steps of the conventional function include

[0054] Acquiring the historical working data, wherein the historical working data includes process parameters, model parameters, damage type and service life;

[0055] The historical work data is used as nodes, the corresponding relationships between the historical work data are used as associations, and the probability of the corresponding relationships is used as attributes to establish a knowledge graph.

[0056] Features are extracted from the knowledge graph, and a Bayesian network structure is designed to calculate the conditional probability relationship between various factors to obtain the conventional function.

[0057] The knowledge graph can quickly obtain the association between historical work data and obtain the association results, and then use the Bayesian network structure to calculate the prediction probability under various conditions, that is, to obtain the conventional function. Considering the lack of data, the installation status of parts and the working environment are not distinguished in the conventional function. Dynamic intervention function and static intervention function are used to eliminate errors.

[0058] For further information, please refer to Figure 3 Considering that the same component will be subjected to different loads at different installation positions, and therefore the probability of its failure is also different, the steps of constructing the static intervention function according to the installation position include:

[0059] Get the installation position of the component to define as position information,

[0060] Obtain the conventional predicted abnormal probability of the parts based on the historical working data and the conventional function,

[0061] The actual abnormal occurrence time of the component is obtained and compared with the conventional predicted abnormal probability to obtain a static error, and the correlation between the installation position and the static error is obtained through training, that is, the static intervention function is obtained.

[0062] For example, from the above, we can get that the screw of type A has a function whose abnormal probability changes with time, that is, a regular function, and then different static intervention functions are introduced according to the actual installation position of the screw of type A.

[0063] For further information, please refer to Figure 4 The steps of constructing the dynamic intervention function according to the historical multimodal data include:

[0064] Acquire historical multimodal data of components, wherein the historical multimodal data includes historical working temperature, historical working humidity, historical working image information, and historical working audio information.

[0065] Obtain the conventional predicted abnormal probability of the parts based on the historical working data and the conventional function,

[0066] The actual abnormal occurrence time of the component is obtained and compared with the conventional predicted abnormal probability to obtain a dynamic error, and the correlation between the historical multimodal data and the dynamic error is obtained through training, that is, the dynamic intervention function is obtained.

[0067] The setting of the dynamic intervention function mainly considers the dynamic errors caused by the external environment, such as the dynamic errors caused by the differences in the operating scenarios. In this application, the dynamic intervention function is obtained by using historical multimodal data, and the actual working data of the components is collected for calculation, so that the prediction accuracy is higher.

[0068] Furthermore, the actual multimodal data is collected based on the multimodal sensor, and the actual multimodal data includes actual working temperature parameters and actual working humidity parameters. Whether it is in a special environment is determined according to the actual multimodal data, and the static intervention function is corrected according to the duration of the special environment. This setting is mainly to take into account that the influence of environmental factors not only affects the dynamic error. For example, when the high temperature and high humidity environment lasts for a certain period of time, when the external conditions of high temperature and high humidity are eliminated, the dynamic error is also eliminated. However, it has already affected the possibility of future abnormalities of components, so the called static intervention function can change.

[0069] Furthermore, the static intervention function is configured with a function update strategy, and the function update strategy includes

[0070] Get the current working image information at a preset update time interval.

[0071] Identify and determine whether the current working image information contains abnormal features, such as rust, surface damage and discoloration,

[0072] If there are abnormal features, the static intervention function is modified.

[0073] In this embodiment, this setting is mainly to take into account that the actual state of the component will affect the probability of future abnormalities. For example, when the component is rusted, the risk of damage is higher, so the static intervention function called by it can change.

[0074] Furthermore, a function learning strategy is also configured, and the function learning strategy includes

[0075] Determine the correlation between environmental variables and the abnormality rate of parts with different process parameters. If, under specific environmental variables, the abnormality rate of parts with specific process parameters is greater than the preset abnormality threshold, then correct the static intervention function of the part under the specific environmental variables.

[0076] In this embodiment, the function learning strategy is mainly based on the consideration that the amount of data initially established for the static intervention function may be insufficient, so it cannot fully cover all influencing factors. The setting of the function learning strategy can make up for this deficiency, and can autonomously determine the environmental variable parameters that are highly correlated with the abnormal rate in subsequent operations, and correct the parameters. For example, if it is found that the screws of model B are easily damaged at temperature T, the dynamic intervention function of temperature T can be corrected.

[0077] Furthermore, the determination of the special event includes:

[0078] Obtain real-time working sound information, real-time working image information and real-time working video information,

[0079] determining whether the real-time working sound information contains abnormal sound information, determining whether the real-time working image information contains abnormal image information, determining whether the real-time working video information contains abnormal video information,

[0080] When abnormal information appears in two of the real-time working sound information, the real-time working image information and the real-time working video information, it is determined that a special event occurs.

[0081] Considering that damage or scratches to other components may cause structural deviations, this deviation does not directly lead to failures, but may affect the probability of future failures, so special events are set. For example, the wheel is under relatively large pressure, and the impact during operation causes deviations in roundness. Multimodal sensors can be used to obtain various types of information, including sound, images, videos, temperature, humidity, etc. Special events can be derived from the above data. For example, when judging whether there is an abnormal condition of the wheel, the working sound can be combined with the video information to judge, identify the wheel working audio and judge whether there is abnormal information. If it is judged that there is abnormal information in the wheel working audio, the wheel video information can be called to judge the wheel movement state and then confirm it again. The abnormal condition of the wheel can be judged in combination with the wheel video information, and the dynamic intervention function can be called according to the abnormal condition to correct the abnormal error.

[0082] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A train component abnormality probability prediction method based on multimodal data, characterized by: include Get the historical working data of the parts and pre-build the regular functions. Construct static intervention functions based on the installation locations of components and historical multimodal data. Construct dynamic intervention functions for components based on their historical multimodal data. Calling a corresponding static intervention function according to the actual installation location and the duration of the special environment, obtaining actual multimodal data, determining whether a special event is present according to the actual multimodal data and calling a corresponding dynamic intervention function, A prediction function is obtained according to the static intervention function, the dynamic intervention function and the conventional function, wherein the prediction function is specifically a function of the abnormal probability changing with time, Taking actual process parameters, actual installation locations, and actual multimodal data of components as inputs of the prediction function to obtain future predicted failure probabilities; The construction steps of the conventional function include Acquiring the historical working data, wherein the historical working data includes process parameters, model parameters, damage type and service life; The historical work data is used as nodes, the corresponding relationships between the historical work data are used as associations, and the probability of the corresponding relationships is used as attributes to establish a knowledge graph. Extract features from the knowledge graph, design a Bayesian network structure to calculate the conditional probability relationship between various factors, and obtain the conventional function; The steps of constructing the static intervention function according to the installation location include: Get the installation position of the component to define as position information, Obtain the conventional predicted abnormal probability of the parts based on the historical working data and the conventional function, The actual abnormal occurrence time of the component is obtained and compared with the conventional predicted abnormal probability to obtain a static error, and the correlation between the installation position and the static error is obtained by training, that is, the static intervention function is obtained; The steps of constructing the dynamic intervention function according to the historical multimodal data include: Acquire historical multimodal data of components, wherein the historical multimodal data includes historical working temperature, historical working humidity, historical working image information, and historical working audio information. Obtain the conventional predicted abnormal probability of the parts based on the historical working data and the conventional function, The actual abnormal occurrence time of the component is obtained and compared with the conventional predicted abnormal probability to obtain a dynamic error, and the correlation between the historical multimodal data and the dynamic error is obtained through training, that is, the dynamic intervention function is obtained.

2. The method for predicting the abnormal probability of train parts based on multimodal data according to claim 1 is characterized in that: The actual multimodal data is collected based on a multimodal sensor, the actual multimodal data includes an actual working temperature parameter and an actual working humidity parameter, whether the environment is in a special environment is determined according to the actual multimodal data, and the static intervention function is corrected according to the duration of the special environment.

3. The method for predicting the abnormal probability of train parts based on multimodal data according to claim 1, characterized in that: The static intervention function is configured with a function update strategy, and the function update strategy includes Get the current working image information at a preset update time interval. Identify and determine whether the current working image information contains abnormal features, such as rust, surface damage and discoloration, If there are abnormal features, the static intervention function is modified.

4. The method for predicting the abnormal probability of train parts based on multimodal data according to claim 1 is characterized in that: A function learning strategy is also configured, and the function learning strategy includes Determine the correlation between environmental variables and the abnormality rate of parts with different process parameters. If, under specific environmental variables, the abnormality rate of parts with specific process parameters is greater than the preset abnormality threshold, then correct the static intervention function of the part under the specific environmental variables.

5. The method for predicting the abnormal probability of train parts based on multimodal data according to claim 1 is characterized in that: The determination of the special event includes Obtain real-time working sound information, real-time working image information and real-time working video information, determining whether the real-time working sound information contains abnormal sound information, determining whether the real-time working image information contains abnormal image information, determining whether the real-time working video information contains abnormal video information, When abnormal information appears in two of the real-time working sound information, the real-time working image information and the real-time working video information, it is determined that a special event occurs.

Images