Fault prediction method, related device, equipment and storage medium

Abstract

The invention discloses a fault prediction method, a related device, equipment and a storage medium, and the method comprises the steps: obtaining the collection data, about a plurality of modes, of to-be-tested equipment in an operation process; adjusting respective attention parameters of the plurality of modes based on respective jump detection results of the data acquired by the plurality of modes; based on respective attention parameters of the plurality of modes, obtaining target data features of the to-be-tested device; performing prediction based on the target data features to obtain a fault prediction result of the to-be-tested equipment; wherein the fault prediction result at least comprises the possibility that the to-be-tested equipment has a plurality of preset fault types. According to the scheme, the real-time performance and accuracy of equipment fault prediction can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a fault prediction method and related devices, equipment and storage media. Background Art

[0002] Fault detection is an essential part of equipment operation and maintenance. Fault detection of equipment helps to find faults as early as possible and then eliminate them to ensure the safe operation of the equipment as much as possible.

[0003] At present, equipment failures are mainly discovered through manual detection. This method is difficult to detect faults in real time in some scenarios (such as automobile engine fault detection, etc.), and the detection accuracy depends on manual experience. With the development of artificial intelligence technology, fault prediction is also carried out through artificial intelligence models. However, the prediction accuracy is also relatively limited. In view of this, how to improve the real-time and accuracy of equipment fault prediction has become an urgent problem to be solved. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a fault prediction method and related devices, equipment and storage media, which can improve the real-time and accuracy of equipment fault prediction.

[0005] In order to solve the above-mentioned technical problems, the first aspect of the present application provides a fault prediction method, including: obtaining collected data about several modes of the device under test during operation; adjusting the attention parameters of the several modes based on the jump detection results of the collected data of the several modes; obtaining the target data characteristics of the device under test based on the attention parameters of the several modes; making predictions based on the target data characteristics to obtain fault prediction results of the device under test; wherein the fault prediction results at least include the possibility that the device under test has several preset fault types.

[0006] In order to solve the above-mentioned technical problems, the second aspect of the present application provides a fault prediction device, including: a data acquisition module, a parameter adjustment module, a feature acquisition module and a result prediction module, the data acquisition module is used to obtain the collected data about several modes of the device under test during operation; the parameter adjustment module is used to adjust the attention parameters of several modes based on the jump detection results of the collected data of several modes; the feature acquisition module is used to obtain the target data characteristics of the device under test based on the attention parameters of several modes; the result prediction module is used to make predictions based on the target data characteristics to obtain the fault prediction results of the device under test; wherein the fault prediction results at least include the possibility that the device under test has several preset fault types.

[0007] In order to solve the above technical problems, the third aspect of the present application provides an electronic device, which comprises at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor is used to execute the program instructions to implement the fault prediction method in the above first aspect.

[0008] In order to solve the above technical problem, the fourth aspect of the present application provides a computer-readable storage medium, which stores program instructions that can be executed by a processor, and the program instructions are used to implement the fault prediction method of the first aspect.

[0009] The above scheme obtains the collected data of several modes of the device under test during operation, and adjusts the attention parameters of several modes based on the jump detection results of the collected data of several modes, and then obtains the target data characteristics of the device under test based on the attention parameters of the several modes, and predicts based on the target data characteristics to obtain the fault prediction results of the device under test, and the fault prediction results at least include the possibility that the device under test has several preset fault types. Therefore, on the one hand, by predicting the fault of the collected data during the operation of the device under test without relying on manual inspection, it is helpful to predict the fault of the device under test in real time during operation. On the other hand, since the corresponding attention parameters are adjusted by referring to the jump detection results of various modal data in the fault prediction process, the attention to the relevant modal can be increased in the fault prediction process, and different attention to different modalities can be adopted to assist fault prediction, which helps to adaptively reduce the attention to the modal data that is not helpful for fault prediction and enhance the attention to the modal data that is helpful for fault prediction in the fault prediction process. Therefore, the real-time and accuracy of equipment fault prediction can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flow chart of an embodiment of a fault prediction method of the present application; Figure 2 It is a schematic diagram of the framework of an embodiment of a fault prediction device of the present application; Figure 3 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application; Figure 4 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0011] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0012] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0013] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the fragment " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0014] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the fault prediction method of the present application. Specifically, it may include the following steps: Step S11: Acquire collected data on several modes during the operation of the device under test.

[0015] In an implementation scenario, the device to be tested can be specifically set according to the application scenario, application industry, etc. For example, in a driving scenario or the automotive industry, the device to be tested may include but is not limited to an engine; or in a power generation scenario, the device to be tested may include but is not limited to a generator set. Other scenarios or industries can be deduced in the same way, and no further examples are given here.

[0016] In an implementation scenario, the several modalities may include but are not limited to: image, sound, vibration, temperature, etc., and the specific contents of the several modalities are not limited here.

[0017] In a specific implementation scenario, taking the driving scenario or the automotive industry as an example, the device to be tested may be an engine, and a camera device may be set in the engine compartment to at least take an image of the engine to obtain acquisition data of the image modality, which may be referred to as image data in order to distinguish the acquisition data of other modalities. It should be noted that real-time image analysis can help to timely discover and locate potential problems and improve maintenance efficiency.

[0018] In a specific implementation scenario, taking the driving scenario or the automotive industry as an example, the device to be tested can be an engine, and a vibration sensor can be set in the engine compartment to at least collect vibrations of the engine to obtain the collected data of the vibration mode, which can be called vibration data in order to distinguish the collected data of other modes. It should be noted that the vibration sensor can monitor the subtle vibrations of the engine, and can distinguish between normal wear and abnormal vibrations, such as bearing failures, piston knocking, etc. By analyzing this data in real time, possible mechanical failures can be warned in advance, reducing maintenance costs and downtime.

[0019] In a specific implementation scenario, taking the driving scenario or the automotive industry as an example, the device to be tested can be an engine, and a temperature sensor can be set in the engine compartment to collect the temperature of at least the engine to obtain the collected data of the temperature mode, which can be called temperature data in order to distinguish the collected data of other modes. It should be noted that the temperature of various parts of the engine can be monitored through the temperature sensor to identify overheating or temperature abnormalities, such as cooling system failure, incomplete combustion, etc., to detect temperature abnormalities in time, prevent the engine from being seriously damaged due to overheating, and ensure the safe operation of the equipment.

[0020] In a specific implementation scenario, taking the driving scenario or the automotive industry as an example, the device to be tested may be an engine, and a sound pickup device may be set in the engine compartment to collect sound from the engine at least, so as to obtain the collected data of the sound mode, which may be referred to as audio data in order to distinguish the collected data of other modes. It should be noted that by analyzing the audio data, abnormal conditions during the operation of the engine can be discovered in a timely manner, providing important clues for subsequent diagnosis.

[0021] It should be noted that the above examples are only several possible examples of obtaining several types of modal acquisition data when the engine is used as the device under test in driving scenarios or the automotive industry. The specific methods of obtaining several types of modal acquisition data for the device under test in other scenarios or industries are not limited, and no examples are given here. For example, in driving scenarios or the automotive industry, pressure sensors can also be set to detect the pressure of the oil circuit and the gas circuit to determine leakage or blockage, such as oil pump failure, fuel injector blockage, etc. Through real-time pressure monitoring, potential system failures can be warned to avoid equipment failures or accidents caused by abnormal pressure.

[0022] Step S12: Based on the jump detection results of the data collected in the several modes, adjust the attention parameters of the several modes respectively.

[0023] Specifically, in the embodiment of the present disclosure, the fault prediction result can be obtained by the fault prediction model, and the fault prediction model can at least include an attention network. On this basis, based on the jump detection results of each of the several modal acquisition data, the initial data features of each of the several modal acquisition data can be extracted respectively, and then after the initial data features of each of the several modal acquisition data are input into the fault prediction model, the attention parameters output by the attention network are obtained. In other words, the "attention parameter" in the embodiment of the present disclosure is not a fixed value, but is adaptively adjusted according to the input, specifically, after the input, it is obtained by the output of the attention network in the fault prediction model. The above method extracts the initial data features based on the jump detection results, and then obtains the attention parameters output by the attention network after the initial data features are input into the fault prediction model, which enables the attention network to adaptively output the attention parameters according to the jump detection results, which helps to reduce the attention to the modal data without jumps and enhance the attention to the modal data with jumps.

[0024] In one implementation scenario, during the training of the fault prediction model, the attention network can learn the potential relationship between the attention parameters and the jump detection results, so that after the initial data features obtained based on the jump detection results are input into the fault prediction model, the attention network can adaptively output the corresponding attention parameters according to the jump detection results reflected by the initial data features. It should be noted that the specific structure of the attention network and the specific implementation of the attention mechanism can refer to the technical details of the attention mechanism (i.e., the Attention mechanism), which will not be repeated here. In addition, the training process of the fault prediction model can refer to the subsequent related description of the embodiments of the present disclosure, which will not be repeated here.

[0025] In an implementation scenario, before feature extraction, the collected data can be preprocessed, such as data cleaning, noise reduction, etc. In the data cleaning stage, outliers, missing values, duplicate values, etc. in the data can be identified and eliminated to ensure the integrity and accuracy of the data. In the data noise reduction stage, various filtering and smoothing techniques can be used to remove noise and interference in the data and improve the signal-to-noise ratio and stability of the data.

[0026] In one implementation scenario, for image data, convolutional neural networks, Transformers, etc. can be set in the fault prediction model to extract features from image data to automatically learn spatial features and frequency domain features, which helps to identify details in visual information, such as component damage or oil leakage, etc., and for audio data, recurrent neural networks, long short-term memory networks, and one-dimensional convolutional neural networks can be set in the fault prediction model to extract features from audio data to extract abnormal noise or change trends in sound, and for vibration data, convolutional neural networks, autoencoders, Fourier transforms, etc. can be set in the fault prediction model to extract features from vibration data to identify abnormal vibrations, such as bearing failures or piston knocks, etc. Other modal data (such as temperature data, pressure data, etc.) can be deduced in the same way, and examples are not given one by one here.

[0027] In one implementation scenario, the jump detection result includes at least the jump time point, jump amplitude, jump frequency, etc. Then, in the process of extracting the initial data features, the jump features of the mode to which the collected data belongs can be extracted based on the jump detection result of the collected data, and the time domain features of the mode to which the collected data belongs can be extracted based on the local data of the collected data within the preset time limit at the jump time point, and the frequency domain features of the mode to which the collected data belongs can be extracted based on the collected data. On this basis, the initial data features of the mode to which the collected data belongs can be obtained based on at least the jump features, time domain features and frequency domain features of the mode to which the collected data belongs. The above method combines the jump features, time domain features and frequency domain features to obtain the initial data, and can extract features from as many dimensions as possible in combination with the data jump situation, which helps to improve the richness and accuracy of the initial data features.

[0028] In a specific implementation scenario, the transition feature can be constructed based on transition-related features such as transition timing, transition amplitude, and transition frequency.

[0029] In a specific implementation scenario, local data can be selected within a preset time limit (e.g., 1 second, 2 seconds, etc.) of the jump point in the collected data to analyze the data trend changes, periodicity, etc. before and after the jump point as time domain features.

[0030] In a specific implementation scenario, the collected data may be transformed into a frequency domain (eg, Fourier transform, etc.) to extract frequency domain features. It should be noted that the frequency domain features may include frequency components related to jumps.

[0031] In a specific implementation scenario, in addition to the above-mentioned jump features, time domain features, and frequency domain features, other features can also be extracted. Exemplarily, basic features can also be extracted, such as basic information such as mean, variance, maximum value, minimum value, etc. can be statistically analyzed from the collected data to construct basic features. Of course, the above example is only a possible way to extract basic features, and other possible features are not limited thereto.

[0032] In a specific implementation scenario, for the collected data of any mode, after extracting the characteristic information including but not limited to the above-mentioned jump characteristics, time domain characteristics, frequency domain characteristics, etc., the above-mentioned characteristic information can be integrated to obtain the initial data characteristics of the mode to which the collected data belongs, so as to realize multi-level feature fusion, so that the mutation situation in the data can be understood as comprehensively as possible during the subsequent fault prediction, which helps to assist in the prediction and identification of abnormal situations.

[0033] In one implementation scenario, before obtaining the attention parameters output by the attention network, it is also possible to obtain the target weight parameters of the several types of modal acquisition data based on the jump detection results of the several types of modal acquisition data, and then obtain the weighted data features of the several types of modal acquisition data based on the target weight parameters and initial data features of the several types of modal acquisition data. On this basis, the attention parameters output by the attention network can be obtained after the weighted data features of the several types of modalities are input into the fault prediction model. The above method adaptively determines the target weight parameters of each mode through the jump detection results to weight the initial data features, which helps to further reduce the attention to the modal data without jumps and enhance the attention to the modal data with jumps.

[0034] In a specific implementation scenario, after obtaining the jump detection result, it is possible to obtain the weight adjustment coefficients of the respective modes based on the jump detection results of the respective modes of the acquired data, and then obtain the target weight parameters of the respective modes of the acquired data based on the weight adjustment coefficients of the respective modes and the weight parameters to be adjusted. Exemplarily, when the jump detection result of any mode represents a larger jump amplitude and a higher jump frequency, the weight adjustment coefficient of the corresponding mode is larger. Conversely, when the jump detection result of any mode represents a smaller jump amplitude and a lower jump frequency, the weight adjustment coefficient of the corresponding mode is smaller. The specific value of the weight adjustment coefficient is not limited here. In addition, the weight parameter to be adjusted of any mode can be specifically the current weight parameter of the corresponding mode, or it can be the initial weight parameter of the corresponding mode, which is not limited here. For example, when it is determined that the weight adjustment coefficient of a certain mode is 0.3, the target weight parameter of the mode can be set to 0.3 multiplied by the weight parameter to be adjusted of the mode (such as the current weight parameter, the initial weight parameter, etc.), and other cases can be deduced by analogy, and examples are not given one by one here. In the above method, the weight adjustment coefficient is first determined by the jump detection result, and then the weight adjustment coefficient is used to adjust the weight parameter to be adjusted to obtain the target weight parameter. It can adaptively reduce the attention to the non-jump modal data and enhance the attention to the jump modal data according to the jump detection result.

[0035] In a specific implementation scenario, in order to further improve the accuracy of weight setting, a feedback adjustment mechanism for weight setting can also be established. For example, after obtaining the fault prediction result, the target detection result of the device under test can be output, and the target detection result at least includes the target fault type of the device under test, and the target detection result is obtained based on the fault prediction result. For details, please refer to the following related description, which will not be repeated here. On this basis, the weight distribution mechanism can be adjusted in response to the feedback result of the target detection result, and the weight distribution mechanism is used to determine the target weight parameter based on the jump detection result. For example, in the case where the feedback result represents that the target detection result deviates greatly from the actual fault type of the device under test, the adjustment strength of the target weight parameter determined based on the jump detection result can be increased; or, in the case where the feedback result represents that the target detection result deviates slightly from the actual fault type of the device under test, the adjustment strength of the target weight parameter determined based on the jump detection result can be slightly adjusted. Of course, the above examples are only several possible examples of adjusting the weight distribution mechanism, and other possible adjustment methods of the weight distribution mechanism are not limited thereto, and no examples are given one by one here. The above method, according to the feedback result of the target detection result, adjusts the weight distribution mechanism, which can further improve the accuracy of weight setting.

[0036] Step S13: Based on the attention parameters of the respective modes, the target data features of the device under test are obtained.

[0037] Specifically, as mentioned above, the fault prediction result can be obtained by the fault prediction model, and the fault prediction model at least includes an attention network, and the attention parameters can be obtained by the attention network. On this basis, after obtaining the attention parameters, the attention features of several modalities can be obtained based on the input data features of the attention network and the respective attention parameters of the several modalities, and then the target data features can be obtained based on the attention features of the several modalities. It should be noted that the specific process of obtaining the attention features based on the attention parameters and the input data features of the attention network can refer to the technical details of the attention mechanism, which will not be repeated here. In addition, as a possible example, after obtaining the attention features of several modalities, the attention features of several modalities can be fused by methods such as splicing, addition, weighted summation, etc. to obtain the target data features.

[0038] Step S14: Predicting based on the target data characteristics to obtain a fault prediction result of the device under test.

[0039] In the disclosed embodiment, the fault prediction result includes at least the possibility that the device under test has several preset fault types. As a possible example, the fault prediction result may at least specifically include the probability values ​​of the device under test having several preset fault types. For example, N preset fault types may be set according to the specific type of the device under test, and the fault prediction result may specifically include the probability values ​​of the device under test having N preset fault types. As another possible example, in addition to the probability values ​​of the aforementioned preset fault types, the fault prediction result may also include the probability values ​​of unknown fault types. For example, according to the specific type of the device under test, N+1 fault types may be set, including N preset fault types and 1 unknown fault type. The above examples are only several possible examples of predicted fault results, and do not limit the specific content of the predicted fault results, and will not be given one by one here.

[0040] In one implementation scenario, for each preset fault type, the risk detection result of the device under test regarding the preset fault type can be determined based on the probability value of the device under test regarding the preset fault type and the probability threshold group of the preset fault type, and then based on the risk detection results of the device under test regarding various preset fault types, the target detection result of the device under test can be obtained comprehensively. It should be noted that the probability threshold group of any preset fault type can at least include a minimum probability threshold for determining the occurrence of the corresponding preset fault type. In addition, as mentioned above, the target detection result of the device under test can at least include the target fault type of the device under test. The above method determines the risk detection result of the device under test regarding the corresponding preset fault type based on the probability threshold group of each preset fault type and the probability value of the device under test predicted to occur the corresponding preset fault type, and comprehensively obtains the target detection result of the device under test, which can improve the comprehensiveness of fault detection of the device under test.

[0041] In a specific implementation scenario, the probability threshold group of a preset fault type may specifically include: probability thresholds of the preset fault type at different risk levels, and the risk detection result of the device under test regarding the preset fault type includes: whether the device under test has the preset fault type, and when the device under test has the preset fault type, the risk level of the device under test regarding the preset fault type. Exemplarily, for a certain preset fault type, its probability threshold group can be set to include: a probability threshold of 0.4 at low risk, a probability threshold of 0.6 at medium risk, and a probability threshold of 0.8 at high risk. On this basis, if the probability value of the device under test predicted to have the preset fault type is lower than 0.4, it can be determined that the risk detection result of the device under test regarding the preset fault type includes no risk of the preset fault type occurring. If the probability value of the device under test predicted to have the preset fault type is between 0.4 and 0.6, it can be determined that the risk detection result of the device under test regarding the preset fault type includes a low risk of the preset fault type occurring. If the probability value of the device under test predicted to have the preset fault type is between 0.6 and 0.8, it can be determined that the risk detection result of the device under test regarding the preset fault type includes a medium risk of the preset fault type occurring. If the probability value of the device under test predicted to have the preset fault type is above 0.8, it can be determined that the risk detection result of the device under test regarding the preset fault type includes a high risk of the preset fault type occurring. Of course, the above example is only a possible example in the actual application process, and the specific values ​​of the probability thresholds of different risk levels are not limited here. In the above method, the probability threshold group of the preset fault type includes: the probability thresholds of the preset fault type at different risk levels, and the risk detection result of the device under test for the preset fault type includes: whether the device under test has the preset fault type, and when the device under test has the preset fault type, the risk level of the device under test for the preset fault type, which can not only realize the fault type detection, but also realize the detection of the risk level of various fault types, which helps to improve the richness of fault prediction.

[0042] In a specific implementation scenario, the risk detection results of the device under test regarding various preset fault types may be combined to obtain a target detection result of the device under test.

[0043] In one implementation scenario, in addition to fault warning through the fault prediction model, the collected data itself can also be detected for abnormalities through the warning threshold. For example, based on the jump detection results of the collected data of several modes, the warning thresholds of several modes can be adjusted, and then the collected data of the same mode can be detected respectively based on at least the warning thresholds of several modes to obtain the abnormal detection results of several modes. The above method, which adaptively sets the warning threshold through the jump detection results and performs abnormality detection on the collected data, helps to adaptively detect abnormalities in the collected data according to the specific situation.

[0044] In a specific implementation scenario, an initial jump detection threshold can be set. When the collected data of any mode changes beyond the initial jump detection threshold, the collected data of the mode can be marked as having a jump event. On this basis, as mentioned above, jump detection results such as jump amplitude can be calculated. Of course, other types of values ​​can also be included, such as the duration of the jump, etc. For details, please refer to the above-mentioned related description, which will not be repeated here.

[0045] In a specific implementation scenario, taking the jump detection result including the jump amplitude as an example, when the jump amplitude of the collected data of a certain mode is larger, its warning threshold can be increased to increase the sensitivity to abnormal situations, and when the jump amplitude of the collected data of a certain mode is smaller, its warning threshold can be lowered to reduce false alarms. Of course, the above examples are only possible examples of adaptively adjusting the warning threshold according to the jump detection results, and other possible adjustment methods are not limited thereto, and no examples are given here one by one.

[0046] In a specific implementation scenario, after obtaining the corresponding warning thresholds of each modality, the collected data of each modality can be detected for abnormalities to obtain abnormal detection results. For example, for temperature data, the collected temperature data can be compared with the warning thresholds set adaptively above. If it is not within the numerical range of the warning threshold, the abnormal detection result of the temperature data can be determined, including detection as abnormal. Other modalities can be deduced in the same way, and examples are not given one by one here.

[0047] In an implementation scenario, as mentioned above, the fault prediction result can be obtained by the fault prediction model. As a possible example, before using the fault prediction model for fault prediction, the fault prediction model can be trained first. Specifically, the sample acquisition data of several modes of the sample device during operation can be obtained first, and the sample acquisition data can be at least marked with the sample fault type, and then based on the sample jump detection results of the sample acquisition data of several modes, the target learning rate of each of the several modes is determined, and the sample acquisition data is processed based on the fault prediction model to obtain at least the predicted probability value of several preset fault types in the sample device. For details, please refer to the relevant description of obtaining the fault prediction result mentioned above, which will not be repeated here. On this basis, the network parameters of the fault prediction model can be adjusted according to the target learning rate based on the training loss measured by the sample fault type and the predicted probability value. Exemplarily, the training loss can be obtained by measuring the sample fault type and the predicted probability value using the cross entropy loss function. For details, please refer to the technical details of the cross entropy loss function, which will not be repeated here. It should be noted that the learning rate is a hyperparameter in deep learning, which is used to control the update speed and direction of the weight parameters of the model during the training process. For details, please refer to the technical details of the learning rate, which will not be repeated here. In the above method, the target learning rate of various modes is determined in combination with the sample jump detection results during the training process of the fault prediction model, so that various modal data can be trained according to the target learning rate, which can force the fault prediction model to adapt to the mutation data faster during the training process.

[0048] In one implementation scenario, in order to reduce latency, the above-mentioned computing process can be implemented through edge computing. As a possible example, taking the driving scenario or the automotive industry as an example, the computing resources of edge computing may include but are not limited to: an on-board computing unit responsible for processing sensor device data (such as an electronic control unit ECU, a dedicated computer, etc.), an integrated processor or computing platform for real-time analysis and processing of data streams (such as an on-board processing module, an embedded system, etc.), a roadside unit for receiving and processing real-time data from the vehicle (such as a computing device set up on the roadside), and a cloud service for further analysis and processing. Of course, the above-mentioned computing resources are only a few possible examples in the actual application process, and the specific types of computing resources are not limited thereto.

[0049] The above scheme obtains the collected data of several modes of the device under test during operation, and adjusts the attention parameters of several modes based on the jump detection results of the collected data of several modes, and then obtains the target data characteristics of the device under test based on the attention parameters of the several modes, and predicts based on the target data characteristics to obtain the fault prediction results of the device under test, and the fault prediction results at least include the possibility that the device under test has several preset fault types. Therefore, on the one hand, by predicting the fault of the collected data during the operation of the device under test without relying on manual inspection, it is helpful to predict the fault of the device under test in real time during operation. On the other hand, since the corresponding attention parameters are adjusted by referring to the jump detection results of various modal data in the fault prediction process, the attention to the relevant modal can be increased in the fault prediction process, and different attention to different modalities can be adopted to assist fault prediction, which helps to adaptively reduce the attention to the modal data that is not helpful for fault prediction and enhance the attention to the modal data that is helpful for fault prediction in the fault prediction process. Therefore, the real-time and accuracy of equipment fault prediction can be improved.

[0050] See also Figure 2 , Figure 2 It is a schematic diagram of the framework of an embodiment of the fault prediction device of the present application. The fault prediction device 20 includes: a data acquisition module 21, a parameter adjustment module 22, a feature acquisition module 23 and a result prediction module 24. The data acquisition module 21 is used to obtain the collected data about several modes of the device under test during operation; the parameter adjustment module 22 is used to adjust the attention parameters of several modes based on the jump detection results of the collected data of several modes; the feature acquisition module 23 is used to obtain the target data characteristics of the device under test based on the attention parameters of several modes; the result prediction module 24 is used to predict based on the target data characteristics to obtain the fault prediction result of the device under test; wherein the fault prediction result at least includes the possibility that the device under test has several preset fault types.

[0051] In the above scheme, the fault prediction device 20 obtains the collected data of several modes during the operation of the device under test, and adjusts the attention parameters of several modes based on the jump detection results of the collected data of several modes, and then obtains the target data characteristics of the device under test based on the attention parameters of the several modes, and predicts based on the target data characteristics to obtain the fault prediction result of the device under test, and the fault prediction result at least includes the possibility that the device under test has several preset fault types. Therefore, on the one hand, by predicting the fault of the collected data during the operation of the device under test without relying on manual inspection, it is helpful to predict the fault of the device under test in real time during the operation. On the other hand, since the corresponding attention parameters are adjusted by referring to the jump detection results of various modal data in the fault prediction process, the attention to the relevant modal can be increased in the fault prediction process, and different attention to different modalities is adopted to assist fault prediction, which helps to adaptively reduce the attention to the modal data that is not helpful for fault prediction and enhance the attention to the modal data that is helpful for fault prediction in the fault prediction process. Therefore, the real-time and accuracy of equipment fault prediction can be improved.

[0052] In some disclosed embodiments, a fault prediction result is obtained by a fault prediction model, and the fault prediction model at least includes an attention network. The parameter adjustment module 22 includes an initial extraction submodule for extracting initial data features of each of the several types of modal acquisition data based on respective jump detection results of the several types of modal acquisition data. The parameter adjustment module 22 includes a parameter acquisition submodule for acquiring attention parameters output by the attention network after the initial data features of each of the several types of modal acquisition data are input into the fault prediction model.

[0053] In some disclosed embodiments, the jump detection result includes at least a jump timing point, a jump amplitude and a jump frequency. The initial extraction submodule includes an extraction unit for extracting the jump characteristics of the modality to which the collected data belongs based on the jump detection result of the collected data, and extracting the time domain characteristics of the modality to which the collected data belongs based on the local data of the collected data within a preset time limit at the jump timing point, and extracting the frequency domain characteristics of the modality to which the collected data belongs based on the collected data; the initial extraction submodule includes a combination unit for obtaining the initial data characteristics of the modality to which the collected data belongs based on at least the jump characteristics, time domain characteristics and frequency domain characteristics of the modality to which the collected data belongs.

[0054] In some disclosed embodiments, the parameter adjustment module 22 includes a weight acquisition submodule, which is used to obtain target weight parameters of several types of modal acquisition data based on the jump detection results of each of the several types of modal acquisition data; the parameter adjustment module 22 is specifically used to obtain weighted data features of several types of modal acquisition data based on the target weight parameters and initial data features of each of the several types of modal acquisition data, and after the weighted data features of each of the several types of modal acquisition data are input into the fault prediction model, obtain the attention parameters output by the attention network.

[0055] In some disclosed embodiments, the weight acquisition submodule includes a coefficient determination unit, which is used to obtain the weight adjustment coefficients of several types of modal acquisition data based on the jump detection results of each of the several types of modal acquisition data; the weight acquisition submodule includes a parameter determination unit, which is used to obtain the target weight parameters of several types of modal acquisition data based on the weight adjustment coefficients of each of the several types of modal acquisition data and the weight parameters to be adjusted.

[0056] In some disclosed embodiments, the weight acquisition submodule includes a result output unit for outputting a target detection result of the device under test; wherein the target detection result includes at least a target fault type of the device under test, and the target detection result is obtained based on a fault prediction result; the weight acquisition submodule includes a mechanism adjustment unit for adjusting a weight allocation mechanism in response to feedback results of the target detection result; wherein the weight allocation mechanism is used to determine a target weight parameter based on a jump detection result.

[0057] In some disclosed embodiments, a fault prediction result is obtained by a fault prediction model, and the fault prediction model at least includes an attention network, and the attention parameters are obtained by the attention network. The feature acquisition module 23 includes an attention feature acquisition submodule, which is used to obtain the attention features of several modalities based on the input data features of the attention network and the respective attention parameters of the several modalities; the feature acquisition module 23 includes a target data feature acquisition submodule, which is used to obtain the target data features based on the attention features of the several modalities.

[0058] In some disclosed embodiments, the fault prediction device 20 includes a threshold adjustment module for adjusting the warning thresholds of several modal acquisition data based on the jump detection results of the several modal acquisition data; the fault prediction device 20 includes a data detection module for detecting the same modal acquisition data based on at least the warning thresholds of the several modal acquisition data to obtain abnormal detection results of the several modal acquisition data.

[0059] In some disclosed embodiments, the fault prediction result at least specifically includes probability values ​​of several preset fault types existing in the device under test, and the fault prediction device 20 includes a risk determination module for determining, for each preset fault type, a risk detection result of the device under test regarding a preset fault type based on the probability value of the device under test regarding the preset fault type and a probability threshold group of the preset fault type; the fault prediction device 20 includes a result synthesis module for synthesizing the target detection result of the device under test based on the risk detection results of the device under test regarding various preset fault types.

[0060] In some disclosed embodiments, a probability threshold group for a preset fault type includes: probability thresholds for the preset fault type at different risk levels, and a risk detection result of the device under test regarding the preset fault type includes: whether the device under test has the preset fault type, and when the device under test has the preset fault type, the risk level of the device under test regarding the preset fault type.

[0061] In some disclosed embodiments, a fault prediction result is obtained by a fault prediction model, and the fault prediction device 20 includes a sample acquisition module for acquiring sample collection data about several modes during the operation of the sample device; wherein the sample collection data is at least marked with a sample fault type; the fault prediction device 20 includes a learning rate determination module for determining the target learning rate of each of the several modes based on the sample jump detection results of the sample collection data of the several modes; the fault prediction device 20 includes a probability prediction module for processing the sample collection data based on the fault prediction model to at least obtain the predicted probability values ​​of the sample devices having several preset fault types; the fault prediction device 20 includes a parameter adjustment module for adjusting the network parameters of the fault prediction model according to the target learning rate based on the training loss measured by the sample fault type and the predicted probability value.

[0062] See also Figure 3 , Figure 3: is a schematic diagram of a framework of an embodiment of an electronic device of the present application. The electronic device 30 includes a memory 31 and a processor 32 coupled to each other, the memory 31 at least stores program instructions, and the processor 32 is used to execute the program instructions to implement the steps in any of the above-mentioned fault prediction method embodiments. For details, please refer to the aforementioned disclosed embodiments, which will not be repeated here. As a possible example, the electronic device 30 may include but is not limited to a server, etc., and the specific type of the electronic device 30 is not limited here. In addition, the electronic device 30 may also include but is not limited to a camera device (not shown), a temperature sensor (not shown), a vibration sensor (not shown), a sound pickup device (not shown) and other sensor devices that are connected to the processor 32 for communication. Among them, the camera device is used to capture an image of the device under test at least during the operation of the device under test, the temperature sensor is used to measure the temperature of the device under test at least during the operation of the device under test, the vibration sensor is used to collect vibrations of the device under test at least during the operation of the device under test, and the sound pickup device is used to collect sound of the device under test at least during the operation of the device under test. The above examples are only several possible examples of the electronic device 30 that may include sensor devices, and the specific types of sensor devices are not limited thereto, and examples are not given one by one here.

[0063] Specifically, the processor 32 is used to control itself and the memory 31 to implement the steps in any of the above-mentioned fault prediction method embodiments. The processor 32 can also be called a CPU (Central Processing Unit). The processor 32 may be an integrated circuit chip with signal processing capabilities. The processor 32 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 32 can be implemented by an integrated circuit chip.

[0064] In the above scheme, the electronic device 30 obtains the collected data of several modes of the device under test during operation, and adjusts the attention parameters of several modes based on the jump detection results of the collected data of several modes, and then obtains the target data characteristics of the device under test based on the attention parameters of the several modes, and predicts based on the target data characteristics to obtain the fault prediction result of the device under test, and the fault prediction result at least includes the possibility that the device under test has several preset fault types. Therefore, on the one hand, by predicting the fault of the collected data during the operation of the device under test without relying on manual inspection, it is helpful to predict the fault of the device under test in real time during operation. On the other hand, since the corresponding attention parameters are adjusted by referring to the jump detection results of various modal data in the fault prediction process, the attention to the relevant modal can be increased in the fault prediction process, and different attention levels can be taken for different modalities to assist fault prediction, which helps to adaptively reduce the attention to the modal data that is not helpful for fault prediction and enhance the attention to the modal data that is helpful for fault prediction in the fault prediction process. Therefore, the real-time and accuracy of equipment fault prediction can be improved.

[0065] See also Figure 4 , Figure 4 1 is a schematic diagram of a framework of an embodiment of a computer-readable storage medium 40 of the present application. The computer-readable storage medium 40 stores program instructions 41 that can be executed by a processor, and the program instructions 41 are used to implement the steps in any of the above-mentioned fault prediction method embodiments.

[0066] In the above scheme, the computer-readable storage medium 40 obtains the collected data of several modes during the operation of the device under test, and adjusts the attention parameters of several modes based on the jump detection results of the collected data of several modes, and then obtains the target data characteristics of the device under test based on the attention parameters of the several modes, and predicts based on the target data characteristics to obtain the fault prediction result of the device under test, and the fault prediction result at least includes the possibility that the device under test has several preset fault types. Therefore, on the one hand, by predicting the fault of the collected data during the operation of the device under test without relying on manual inspection, it is helpful to predict the fault of the device under test in real time during the operation. On the other hand, since the corresponding attention parameters are adjusted by referring to the jump detection results of various modal data in the fault prediction process, the attention to the relevant modal can be increased in the fault prediction process, so that different attention levels are taken for different modalities to assist fault prediction, which helps to adaptively reduce the attention to the modal data that is not helpful for fault prediction and enhance the attention to the modal data that is helpful for fault prediction in the fault prediction process. Therefore, the real-time and accuracy of equipment fault prediction can be improved.

[0067] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0068] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0069] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0070] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0071] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0072] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0073] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A fault prediction method, characterized in that: include: Obtain the collected data of several modes during the operation of the device under test; Adjusting the attention parameters of the respective modalities based on the jump detection results of the collected data of the respective modalities; Based on the attention parameters of the respective modalities, obtaining target data features of the device under test; Predictions are made based on the target data features to obtain fault prediction results of the device under test; wherein the fault prediction results at least include the possibility that the device under test has several preset fault types.

2. The method according to claim 1, characterized in that The fault prediction result is obtained by a fault prediction model, and the fault prediction model at least includes an attention network. The attention parameters of the several modes are adjusted based on the jump detection results of the collected data of the several modes, including: Based on the jump detection results of the collected data of the several modes, respectively extracting the initial data features of the collected data of the several modes; After the initial data features of the collected data of the several modes are respectively input into the fault prediction model, the attention parameters output by the attention network are obtained.

3. The method according to claim 2, characterized in that The jump detection result at least includes a jump time point, a jump amplitude and a jump frequency. The initial data features of the collected data of the several modes are extracted based on the jump detection results of the collected data of the several modes, respectively, including: Based on the jump detection result of the collected data, the jump feature of the mode to which the collected data belongs is extracted, and based on the local data of the collected data within a preset time limit at the jump time point, the time domain feature of the mode to which the collected data belongs is extracted, and based on the collected data, the frequency domain feature of the mode to which the collected data belongs is extracted; Initial data features of the modality to which the collected data belongs are obtained based at least on the jump features, time domain features and frequency domain features of the modality to which the collected data belongs.

4. The method according to claim 2, characterized in that: Before obtaining the attention parameter output by the attention network, the method further includes: Based on the jump detection results of the collected data in the several modes, obtaining the target weight parameters of the collected data in the several modes; The obtaining of the attention parameters output by the attention network comprises: Based on the target weight parameters and initial data features of the collected data of the several modes, respectively, weighted data features of the collected data of the several modes are obtained; After the weighted data features of the collected data of the several modes are respectively input into the fault prediction model, the attention parameters output by the attention network are obtained.

5. The method according to claim 4, characterized in that The obtaining of the target weight parameters of the collected data in the several modes based on the jump detection results of the collected data in the several modes comprises: Based on the jump detection results of the collected data of the several modes, obtaining the weight adjustment coefficients of the several modes respectively; Based on the weight adjustment coefficients of the respective modes and the weight parameters to be adjusted, target weight parameters of the respective collected data of the respective modes are obtained.

6. The method according to claim 4, characterized in that The method further comprises: Outputting a target detection result of the device under test; wherein the target detection result at least includes a target fault type of the device under test, and the target detection result is obtained based on the fault prediction result; In response to the feedback result of the target detection result, a weight allocation mechanism is adjusted; wherein the weight allocation mechanism is used to determine the target weight parameter based on the jump detection result.

7. The method according to claim 1, characterized in that The fault prediction result is obtained by a fault prediction model, and the fault prediction model at least includes an attention network, the attention parameter is obtained by the attention network, and the target data feature of the device under test is obtained based on the attention parameters of each of the several modes, including: Based on the input data features of the several modalities in the attention network and the respective attention parameters, obtaining the respective attention features of the several modalities; Based on the attention features of each of the several modalities, the target data features are obtained.

8. The method according to claim 1, characterized in that The method further comprises: Adjusting the warning thresholds of the plurality of modes based on the jump detection results of the collected data of the plurality of modes; The collected data of the same modality are detected respectively based on at least the respective early warning thresholds of the several modalities to obtain the respective abnormality detection results of the several modalities.

9. The method according to claim 1, characterized in that: The fault prediction result at least specifically includes probability values ​​of the device under test having several preset fault types respectively. After the prediction is performed based on the target data feature to obtain the fault prediction result of the device under test, the method further includes: For each of the preset fault types, based on the probability value of the device under test with respect to the preset fault type and the probability threshold group of the preset fault type, determine the risk detection result of the device under test with respect to the preset fault type; Based on the risk detection results of the device under test for various preset fault types, a target detection result of the device under test is comprehensively obtained.

10. The method according to claim 9, characterized in that The probability threshold group of the preset fault type includes: the probability thresholds of the preset fault type at different risk levels, and the risk detection result of the device under test regarding the preset fault type includes: whether the device under test has the preset fault type, and when the device under test has the preset fault type, the risk level of the device under test regarding the preset fault type.

11. The method according to claim 1, characterized in that: The fault prediction result is obtained by a fault prediction model, and the training steps of the fault prediction model include: Acquire sample acquisition data of several modes during the operation of the sample device; wherein the sample acquisition data is at least marked with the sample fault type; Determining target learning rates for the respective modalities based on sample jump detection results of the respective sample acquisition data of the respective modalities; Processing the sample collection data based on the fault prediction model to obtain at least predicted probability values ​​of the sample devices respectively having the several preset fault types; Based on the training loss measured by the sample fault type and the predicted probability value, the network parameters of the fault prediction model are adjusted according to the target learning rate.

12. A fault prediction device, characterized in that: include: A data acquisition module is used to acquire data collected about several modes during the operation of the device under test; A parameter adjustment module, configured to adjust the attention parameters of each of the several modalities based on the jump detection results of each of the collected data of the several modalities; A feature acquisition module, used to obtain target data features of the device under test based on the attention parameters of each of the several modalities; The result prediction module is used to make predictions based on the target data features to obtain a fault prediction result of the device under test; wherein the fault prediction result at least includes the possibility that the device under test has several preset fault types.

13. An electronic device, characterized in that: The method comprises at least a memory and a processor coupled to each other, wherein the memory at least stores program instructions, and the processor is used to execute the program instructions to implement the fault prediction method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the fault prediction method according to any one of claims 1 to 11.

Images