Fault detection method and device, nonvolatile storage medium and electronic equipment

Abstract

The invention discloses a fault detection method and device, a nonvolatile storage medium and electronic equipment. The method comprises the following steps: acquiring first audio data, acquired by audio acquisition equipment, of sound source equipment and second audio data output by a prediction model, and determining a difference value between the first audio data and the second audio data; when the difference value exceeds a preset threshold value, determining that the sound source equipment has a fault; determining the similarity between the first audio data and historical fault audio data in a fault database under the condition of determining that the sound source equipment has the fault; and determining target fault data from the historical fault audio data according to the similarity, and determining the fault type of the sound source equipment according to the fault type corresponding to the target fault data. According to the invention, the technical problem that the equipment fault is not found timely and inaccurately due to low manual inspection efficiency is solved.

Description

Technical Field

[0001] The present application relates to the field of coal mine equipment fault detection, and specifically, to a fault detection method, device, non-volatile storage medium and electronic equipment. Background Art

[0002] At present, in production management, coal mining enterprises mainly use manual on-site inspections to detect equipment faults. They regularly conduct on-site inspections and knocking to check the conditions of their transmission equipment, chains and other products. The discovery of faults mainly depends on the auditory experience and observation of on-site workers. Due to the low efficiency of manual inspections, equipment faults are not discovered in a timely and accurate manner.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a fault detection method, device, non-volatile storage medium and electronic device to at least solve the technical problem of untimely and inaccurate detection of equipment faults due to low efficiency of manual inspection.

[0005] According to one aspect of an embodiment of the present application, a fault detection method is provided, including: obtaining first audio data of a sound source device collected by an audio collection device and second audio data output by a prediction model, and determining a difference value between the first audio data and the second audio data, wherein the prediction model is trained based on historical audio data and is used to predict the audio data of the sound source device in each working time period, and the working states of the sound source device in each working time period are not exactly the same; when the difference value exceeds a preset threshold, determining that the sound source device has a fault; when it is determined that the sound source device has a fault, determining the similarity between the first audio data and the historical fault audio data in a fault database; determining target fault data from the historical fault audio data based on the similarity, and determining the fault type of the sound source device based on the fault type corresponding to the target fault data, wherein the working time period corresponding to the historical fault audio data is the same as the working time period corresponding to the first audio data, and the target fault data is the historical fault audio data in the fault database that has the highest similarity to the first audio data.

[0006] Optionally, determining the fault type of the sound source device based on the fault type corresponding to the target fault data includes: when the similarity of the target fault data is greater than a preset threshold, determining the fault type of the sound source device to be the fault type corresponding to the target fault data; when the similarity of the target fault data is less than a preset threshold, determining the fault type of the sound source device to be a newly added fault type.

[0007] Optionally, before obtaining the second audio data output by the prediction model, the method also includes: obtaining historical audio data, extracting feature data of the historical audio data through Mel-frequency cepstral coefficients; constructing a time series based on the feature data, wherein the time series is divided into multiple groups, and the multiple groups of time series correspond to different working time periods; training a basic model based on the time series to obtain a prediction model, wherein the basic model is a pre-trained long short-term memory network model.

[0008] Optionally, the sound source device includes a first device and a second device, wherein the historical device audio data used for training the prediction model is the historical device audio data of the first device; obtaining the second audio data output by the prediction model includes: when the sound source device is the first device, using the data output by the prediction model as the second audio data; when the sound source device is the second device, obtaining the data output by the prediction model, and converting the data into second audio data based on a feature change factor, wherein the feature change factor is used to represent the mapping relationship between the audio data of the first device and the audio data of the second device.

[0009] Optionally, the characteristic change factor is determined in the following manner: obtaining third audio data of the first device, segmenting the third audio data to obtain a first data frame of the third audio data, wherein the first data frame is arranged according to a time series; performing feature extraction on the first data frame to obtain a first feature vector sequence; performing mean processing on multiple groups of first feature vector sequences to obtain a standardized first feature vector sequence; obtaining fourth audio data of the second device, segmenting the fourth audio data to obtain a second data frame of the fourth audio data, wherein the second data frame is arranged according to a time series; performing feature extraction on the second data frame to obtain a second feature vector sequence; performing mean processing on multiple groups of second feature vector sequences to obtain a standardized second feature vector sequence; and determining the characteristic change factor of the first device and the second device based on the discrete relationship between the standardized first feature vector sequence and the standardized second feature vector sequence.

[0010] Optionally, determining the similarity between the first audio data and the historical fault audio data in the fault database includes: obtaining a first spectrum graph of the first audio data and a second spectrum graph of the historical fault audio data; extracting feature points in the first spectrum graph as first coordinate points in a unified coordinate system, and extracting feature points in the second spectrum graph as second coordinate points in the unified coordinate system, wherein the feature points include peak points and valley points, and the first coordinate points and the second coordinate points correspond one to one; calculating the distance between the first coordinate point in the first spectrum graph and the second coordinate point corresponding to the first coordinate point in the second spectrum graph, and determining the distance between the first spectrum graph and the second spectrum graph based on the distance between the first coordinate point in the first spectrum graph and the second coordinate point corresponding to the first coordinate point in the second spectrum graph, wherein the value of the distance between the first spectrum graph and the second spectrum graph is the similarity between the first audio data and each historical fault audio data.

[0011] Optionally, the prediction model is set in a sensor, wherein the sensor is used to collect data and to perform local real-time fault detection based on the predicted audio data output by the prediction model, and the sensor supports detection in an offline state.

[0012] Optionally, after determining that the fault type of the sound source device is a newly added fault type, the method also includes: sending the first audio data to a manual identification platform, the manual identification platform being used to display the first audio data of the fault type that requires manual identification to the target object; obtaining the fault type returned by the manual identification platform, and updating the fault database.

[0013] Optionally, the audio acquisition device includes a microphone array, and obtaining the first audio data of the sound source device collected by the audio acquisition device includes: when the working amplitude of the device is less than or equal to a preset amplitude threshold, obtaining the first audio data of the sound source device collected by the microphone array arranged by a near-field model; when the working amplitude of the device is greater than the preset amplitude threshold, obtaining the first audio data of the sound source device collected by the microphone array arranged by a far-field model.

[0014] According to another aspect of an embodiment of the present application, a fault detection device is also provided, including: a first acquisition module, used to acquire first audio data of a sound source device collected by an audio acquisition device and second audio data output by a prediction model, and determine a difference value between the first audio data and the second audio data, wherein the prediction model is obtained based on historical audio data training, and is used to predict the audio data of the sound source device in each working time period, and the working states of the sound source device in each working time period are not completely the same; a first determination module, used to determine that the sound source device has a fault when the difference value exceeds a preset threshold; a second determination module, used to determine the similarity between the first audio data and the historical fault audio data in a fault database when it is determined that the sound source device has a fault; a third determination module, used to determine target fault data from the historical fault audio data based on the similarity, and determine the fault type of the sound source device based on the fault type corresponding to the target fault data, wherein the working time period corresponding to the historical fault audio data is the same as the working time period corresponding to the first audio data, and the target fault data is the historical fault audio data in the fault database that has the highest similarity to the first audio data.

[0015] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, in which a program is stored, wherein when the program is running, a device where the non-volatile storage medium is located is controlled to execute a fault detection method.

[0016] According to another aspect of an embodiment of the present application, there is further provided an electronic device, including: a memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the fault detection method is executed when the program is run.

[0017] According to another aspect of an embodiment of the present application, a computer program product is further provided, including a computer program, and the computer program implements the fault detection method when executed by a processor.

[0018] In an embodiment of the present application, first audio data of a sound source device collected by an audio collection device and second audio data output by a prediction model are obtained, and a difference value between the first audio data and the second audio data is determined, wherein the prediction model is obtained based on historical audio data training and is used to predict the audio data of the sound source device in each working time period, and the working states of the sound source device in each working time period are not exactly the same; when the difference value exceeds a preset threshold, it is determined that the sound source device has a fault; when it is determined that the sound source device has a fault, the similarity between the first audio data and the historical fault audio data in a fault database is determined; based on the similarity, target fault data is determined from the historical fault audio data, and based on the target fault data corresponding to the target fault data, the target fault data is obtained. The fault type determines the fault type of the sound source equipment, wherein the working time period corresponding to the historical fault audio data is the same as the working time period corresponding to the first audio data, and the target fault data is the historical fault audio data in the fault database that has the highest similarity to the first audio data. The pre-trained prediction model is trained by the historical audio data, and the collected device audio data is compared with the predicted audio data. When the difference is large, it is determined that there is a fault, and the data is compared with the fault audio data stored in the fault database to determine the fault type, thereby achieving the purpose of automated fault detection, thereby realizing the technical effect of efficient and accurate fault detection, and further solving the technical problem of untimely and inaccurate detection of equipment faults due to low efficiency of manual inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 is a structural schematic diagram of a computer terminal provided according to an embodiment of the present application;

[0021] Figure 2 It is a flowchart of a fault detection method provided according to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of a transfer learning process provided according to an embodiment of the present application;

[0023] Figure 4 It is a structural schematic diagram of a fault detection device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0027] MFCC (Mel frequency cepstral coefficients): Mel frequency is based on the hearing characteristics of the human ear, and it has a nonlinear correspondence with the Hz frequency. The Mel frequency cepstral coefficient (MFCC) uses this relationship to calculate the Hz spectrum characteristics. It is mainly used for audio data feature extraction and reducing the operation dimension.

[0028] In related technologies, the discovery of fault problems mainly relies on on-site manual auditory experience and observation. Some scenes use camera scanning for image analysis to find cracks and abnormal points in visible light. There are the following disadvantages:

[0029] (1) The timeliness of discovering equipment failures is poor, and human subjective judgment is prone to misjudgment and omission;

[0030] (2) Abnormalities in the internal parts of the equipment cannot be directly observed, and hidden dangers are easily overlooked.

[0031] In order to solve the above problems, relevant solutions are provided in the embodiments of the present application, which are described in detail below.

[0032] According to an embodiment of the present application, a method embodiment of a fault detection method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a fault detection method. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0034] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0035] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the fault detection method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned fault detection method is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0038] In the above operating environment, the present application embodiment provides a fault detection method, such as Figure 2 As shown, the method comprises the following steps:

[0039] Step S202, taking the first audio data of the sound source device collected by the audio collection device and the second audio data output by the prediction model, and determining the difference value between the first audio data and the second audio data, wherein the prediction model is trained based on historical audio data and is used to predict the audio data of the sound source device in each working time period, and the working status of the sound source device in each working time period is not exactly the same.

[0040] Optionally, the audio acquisition device includes a microphone array, and obtaining the first audio data of the sound source device collected by the audio acquisition device includes: when the working amplitude of the device is less than or equal to a preset amplitude threshold, obtaining the first audio data of the sound source device collected by the microphone array arranged by a near-field model; when the working amplitude of the device is greater than the preset amplitude threshold, obtaining the first audio data of the sound source device collected by the microphone array arranged by a far-field model.

[0041] Optionally, a microphone array is deployed to collect sound data of coal mine machinery and equipment. A multi-channel two-dimensional microphone array is used to collect sound from coal mine equipment. For small and medium-sized coal mine equipment and equipment with a small working amplitude (less than or equal to a preset amplitude threshold), a near-field model is used to collect equipment sound; for larger coal mine equipment (greater than a preset amplitude threshold) and equipment with a large working amplitude, a far-field model is used to collect equipment sound; microphone array technology is used to collect sound, that is, more than one microphone is used to pick up sound. By using multiple microphones, we can record spatial sound field information, locate sound sources, collect sound fields, achieve better noise reduction performance, and make up for the shortcomings of single-channel sound pickup.

[0042] A microphone array is an array formed by a group of omnidirectional or directional microphones located at different positions in space and arranged in a certain shape. It is a device for spatial sampling of spatially propagating sound signals, and the collected signals contain their spatial position information. According to the distance between the sound source and the microphone array, the array can be divided into a near-field model and a far-field model. According to the topological structure of the microphone array, it can be divided into a linear array, a planar array, a volume array, etc. Correspondingly, according to the dimensionality of the microphone array, it can be divided into one-dimensional, two-dimensional and three-dimensional microphone arrays.

[0043] A one-dimensional microphone array, i.e. a linear microphone array, has its elements centered on the same straight line. Depending on whether the spacing between adjacent elements is the same, it can be divided into a uniform linear array (ULA) and a nested linear array. The uniform linear array is the simplest array topology, with equal distances between elements, consistent phases and sensitivities. The nested linear array can be viewed as a superposition of several groups of uniform linear arrays, and is a special type of non-uniform array. A linear array can only obtain the horizontal angular information of the signal.

[0044] A two-dimensional microphone array, also known as a planar microphone array, has its elements centered on a plane. According to the geometric shape of the array, it can be divided into an equilateral triangle array, a T-shaped array, a uniform circular array, a uniform square array, a coaxial circular array, a circular or rectangular array, etc. A planar array can obtain the horizontal azimuth and vertical azimuth information of the signal.

[0045] A three-dimensional microphone array, or a stereo microphone array, has its elements distributed in a three-dimensional space. According to the three-dimensional shape of the array, it can be divided into a tetrahedron array, a cube array, a rectangular array, a spherical array, etc. A stereo array can obtain the three-dimensional information of the horizontal azimuth, vertical azimuth, and distance between the sound source and the reference point of the microphone array.

[0046] For the sound collection of coal mine equipment, a multi-channel two-dimensional microphone array can be selected. Compared with a one-dimensional array, a two-dimensional array can provide more spatial resolution, thereby better locating the sound source. Compared with a three-dimensional array, a two-dimensional array has a lower processing complexity and a more compact mechanical structure, so it is more suitable for sound collection in space-constrained environments such as coal mines. At the same time, a two-dimensional microphone array can also provide sufficient signal information to meet the acoustic analysis needs in the coal mine field.

[0047] Alternatively, the near-field model considers the sound wave as a spherical wave, which takes into account the amplitude difference between the signals received by the microphone array elements; the far-field model considers the sound wave as a plane wave, which ignores the amplitude difference between the signals received by each array element and approximately assumes that there is a simple time delay relationship between the received signals.

[0048] Optionally, before obtaining the second audio data output by the prediction model, the method also includes: obtaining historical audio data, extracting feature data of the historical audio data through Mel-frequency cepstral coefficients; constructing a time series based on the feature data, wherein the time series is divided into multiple groups, and the multiple groups of time series correspond to different working time periods; training a basic model based on the time series to obtain a prediction model, wherein the basic model is a pre-trained long short-term memory network model.

[0049] Optionally, collect audio data from different devices during full-load operation, and extract features of the audio data through MFCC (Mel-frequency cepstral coefficients) to obtain spectrum information based on time series. Use the sound spectrum information to analyze and compare complex sound characteristics through Mel parameters, and use quantitative characteristics to characterize the characteristics of simulated objects. Use multiple sets of spectrum information based on time series to train long short-term memory models (LSTM models) to obtain a time-based prediction model. Continuously collect audio, regularly update the training model, and accumulate an acoustic sample library.

[0050] Optionally, the working status of the sound source device in each working time period is not exactly the same, that is, the sound source device in each working time period has different working status, resulting in different characteristics of the fault audio. The production status is divided into three stages according to the working time period: preparation stage, formal stage and ending stage. With 24 hours a day as a cycle, 0-4 hours is the preparation stage, 4-20 hours is the formal stage, and 20-24 hours is the ending stage. The fault types manually determined in the past and their corresponding audio information are matched to different production states, saved in the fault library, and switched to the corresponding production states in chronological order for monitoring;

[0051] In actual work, the equipment can go through multiple production states during the whole day, that is, the movement time periods of different production states are fixed every day. For example, the production state can be divided into: preparation stage, formal stage and ending stage. With 24 hours a day as a cycle, 0-4 hours is the preparation stage, 4-20 is the formal stage, and 20-24 is the ending stage. The historically acquired fault audio data is further converted to the corresponding production state in a certain order for classification. The same fault audio data may correspond to different fault types in different production states. The historically manually determined fault types and their corresponding audio information, the corresponding production states are obtained and saved in the fault library. The above division of production states is a simple and direct way, and there may be other more detailed divisions.

[0052] Optionally, the fault types include equipment initialization abnormality and startup running-in abnormality in the initial stage, overload, overheating and wear problems in the formal stage, and cleaning and fatigue abnormality in the final stage.

[0053] Optionally, in order to solve the difficulties in sample collection caused by the harsh on-site environment in industrial monitoring in the coal field, feature-based transfer learning technology is adopted to reduce the large demand for early data accumulation in the project and reduce the difficulty of implementing data collection: the basic model includes a set of baseline feature versions that are trained based on the existing data accumulation of industrial monitoring in previous years and can be provided to subsequent projects as the starting point for training. The number of features, audio frame length and related data all adopt unified industry standards. The embodiment of the present application provides a new artificial intelligence training method, which can quickly model new tasks based on the feature model of old projects and achieve a higher accuracy rate.

[0054] Optionally, a migration algorithm is used to start training with the accumulated model parameters of different machine models in the same field that have been trained in the past as a basis, the underlying model features are all retained through program control, and the top few connection layers are trained with a small amount of newly collected field data; specifically, a historical LSTM prediction model that has been trained in the same or similar field with the same or different machine models is obtained, the first several layers are set as fixed layers, and the latter several layers are set as training layers, the training layers include the top several connection layers, the historical LSTM model is retrained with the actual collected data, the model parameters of the fixed layers are fixed during training, and only the network parameters of the training layers are adjusted, and finally a trained LSTM model is obtained.

[0055] Through experiments, it is found that the use of transfer learning algorithm has a further rapid convergence of accuracy improvement compared with previous methods, and the required training data is greatly reduced (reduced by about 10 times). This method simplifies the cost of early data collection and the requirements for data volume, reduces the time and cost of previous collection and training, and can complete the data collection process in a very short time (about 1 hour). After the migration model training, all work is reduced to 48 hours, and the core training data is reduced from 200,000 to 15,000-20,000 records, reducing the training data requirements by more than 10 times, greatly improving the training efficiency and reducing the dependence on data collection. The transfer learning method, that is, not training the model from scratch, but starting training with the existing model parameters of different machine models in the same field that have been accumulated in the past as the basis, retaining all the underlying model features of the basic model through program control, and only using a small amount of newly collected field data to train the top few connection layers, so as to achieve the purpose of reducing the dependence on the total amount of data by using the knowledge model accumulation of other projects, and at the same time directing the final results to the prediction model of the new machine model we need. The end result is that the amount of data required for the new machine model is reduced by about 10 times, and the subsequent training time is shortened by about 30%. Figure 3 As shown:

[0056] Specifically, as mentioned above, the long short-term memory LSTM model obtains a historical LSTM prediction model (i.e., a basic model) that has been trained in the same or similar field on the same or different machine models. The prediction model has a multi-layer network structure. The first several layers can be set as fixed layers, and the following several layers can be set as training layers. The training layer can include the top several connection layers. The historical LSTM model is then retrained with the actual collected data. During the actual training process, the model parameters of the fixed layer are fixed, and only the network parameters of the training layer are adjusted to finally obtain a trained LSTM model (i.e., a prediction model).

[0057] Optionally, determining audio data conversion rules under full load and non-full load conditions of the equipment based on historical audio data and its spectrum information based on time series, and using the conversion rules to predict subsequent non-full load equipment operation data;

[0058] For example, the load can be the weight of the coal carried by the equipment. The spectrum graph P1 of the equipment under full load working state can be collected, and then the spectrum graph P2 of the equipment under non-full load working state (such as 80%) can be obtained. By comparing the conversion relationship between P1 and P2, the conversion relationship after the equipment changes from full load to non-full load can be determined. Of course, in fact, the above conversion relationship will be different for different equipment and different working links of the same equipment. Therefore, different conversion relationships can be set according to different equipment and different working periods of the equipment.

[0059] In order to accurately detect faults, specific conversion relationships are determined according to the equipment type and working period to ensure the comprehensiveness and accuracy of the monitoring system. The significance of this conversion is that modeling is based on audio data under full load, that is, the training of the prediction model mainly relies on the acoustic samples of the equipment under maximum load. The audio data of the non-full load state can be derived from the full load data through the known conversion relationship. This can reduce the demand for data collection under non-full load conditions, simplify the data accumulation process, and save costs and time. At the same time, since the prediction model only needs to be trained for full load data, the training of the model is more centralized and efficient, and the expected accuracy can be achieved faster. In addition, through this conversion, the monitoring of different working periods can be standardized. Even if the equipment load changes, the audio data collected in real time can be converted into a feature representation equivalent to the full load state through the pre-set conversion relationship, and a unified fault detection analysis can be performed. This not only improves the consistency and comparability of monitoring, but also helps to quickly identify faults under different working conditions, take preventive measures as early as possible, and avoid monitoring blind spots caused by equipment load changes, thereby significantly improving the stability and safety of coal mine machinery and equipment operation.

[0060] In the technical solution provided in step S202, the sound source device includes a first device and a second device, wherein the historical device audio data used in the prediction model training is the historical device audio data of the first device; obtaining the second audio data output by the prediction model includes: when the sound source device is the first device, using the data output by the prediction model as the second audio data; when the sound source device is the second device, obtaining the data output by the prediction model, and converting the data into second audio data based on a feature change factor, wherein the feature change factor is used to represent the mapping relationship between the audio data of the first device and the audio data of the second device.

[0061] Optionally, the characteristic change factor is determined in the following manner: obtaining third audio data of the first device, segmenting the third audio data to obtain a first data frame of the third audio data, wherein the first data frame is arranged according to a time series; performing feature extraction on the first data frame to obtain a first feature vector sequence; performing mean processing on multiple groups of first feature vector sequences to obtain a standardized first feature vector sequence; obtaining fourth audio data of the second device, segmenting the fourth audio data to obtain a second data frame of the fourth audio data, wherein the second data frame is arranged according to a time series; performing feature extraction on the second data frame to obtain a second feature vector sequence; performing mean processing on multiple groups of second feature vector sequences to obtain a standardized second feature vector sequence; and determining the characteristic change factor of the first device and the second device based on the discrete relationship between the standardized first feature vector sequence and the standardized second feature vector sequence.

[0062] Optionally, audio data of different machines in normal operation is obtained, and ambient audio in the audio data is removed using noise processing technology. One machine is determined to be a benchmark machine (i.e., the first device), and the characteristic change factor of each other different machine relative to the benchmark machine is determined through the audio changes between other different machines (i.e., the second device) and the benchmark machine, that is, the conversion relationship between the audio of the two machines; the audio data of other machines are predicted through the prediction model trained with the historical data of the benchmark machine combined with the characteristic change factor.

[0063] Optionally, the characteristic variation factor is determined as follows:

[0064] 1. Obtain multiple groups of normal audio data (third audio data) of the benchmark machine, that is, audio data of multiple days, segment the audio data of each day, and obtain multiple groups of data frames based on time series for each day (first feature vector sequence);

[0065] 2. Extracting feature vectors from the data frame to obtain feature vectors (first feature vector sequence), wherein the feature vectors represent audio state values ​​of the data frame;

[0066] 3. Perform mean processing on multiple groups of feature vectors based on time series corresponding to multiple days at the same time point to obtain a feature vector sequence a that is normalized for the benchmark machine;

[0067] 4. Perform the above steps on other machines in the same way to obtain the feature vector sequence b (second feature vector sequence) of other machines;

[0068] 5. Based on the discrete relationship between the feature vector sequence a and the feature vector sequence b, determine the feature change factor converted from the predicted data of the benchmark machine to other machines. Specifically, first calculate the Euclidean distance d(a,b) between the feature vector sequences a and b. If the distance value is small, use a linear model such as linear regression to describe this relationship; if the distance value is large, use a nonlinear model such as polynomial regression or support vector machine to describe it; then, through model training and parameter optimization, such as using cross-validation and other methods to determine the best conversion model, the characterization parameters of the obtained model are the feature change factors; at the same time, a test data set is needed to verify the accuracy of the model, and the conversion effect is evaluated based on indicators such as the mean square error and R square of the converted data; finally, use the trained model to predict the new feature vector sequence, and convert the predicted data of the benchmark machine to the feature vector sequence of other machines to achieve effective conversion between feature vectors.

[0069] Step S204: When the difference value exceeds a preset threshold, it is determined that the sound source device has a fault.

[0070] Optionally, when the equipment is working at full load, the working conditions of each device in a specified time period (such as the whole day) change in a similar manner, that is, the working conditions between 8:00 and 10:00 on the first day and between 8:00 and 10:00 on the second day are similar. Therefore, by obtaining spectrum information based on time series, the working conditions of different devices under full load can be determined. When it deviates from this similar state, it indicates that there is a problem.

[0071] Therefore, multiple groups of spectral information based on time series can be obtained to train long-short term memory models (LSTM) models (prediction models), thereby obtaining a time-based audio prediction model. Then, the prediction model can be used to predict the subsequent audio information of each day in real time, and then the faulty audio and the subsequent fault type can be determined by comparing the predicted data with the actual sampled data.

[0072] Optionally, the trained prediction model is used to predict the predicted audio data for each subsequent day, and based on the comparison between the audio data collected in real time (first audio data) and the predicted data (second audio data), the part whose difference exceeds a preset value is regarded as abnormal data.

[0073] Specifically, using the MFCC feature extraction method to compare the difference values ​​of audio data is a widely used technology in the field of audio processing. The core idea of ​​this method is to convert the audio signal into a set of eigenvalues, namely MFCC (Mel-Frequency Cepstral Coefficients), which can capture the key information in the audio signal and are very sensitive to small changes in the audio signal.

[0074] First, the audio signal is pre-emphasized, a technique that boosts the energy of high frequencies because the human ear is less sensitive to high frequencies than to low frequencies. Then, the pre-emphasized signal is split into a series of short time frames, each of which is considered an independent sample for analysis.

[0075] On the basis of framing, each frame signal is subjected to a fast Fourier transform (FFT) to obtain the power spectrum of the frame. Subsequently, the power spectrum is passed through a set of Mel filters, which simulate the way the human ear perceives sounds of different frequencies and convert the power spectrum into a Mel frequency energy spectrum. The design of the Mel filter bank is based on the nonlinear perception characteristics of the human ear to sound frequencies, so that the obtained energy spectrum is more in line with the actual hearing effect of the human ear.

[0076] Next, the Mel frequency energy spectrum is subjected to discrete cosine transform (DCT) to obtain MFCC features. DCT is a mathematical tool that converts signals from the time domain to the frequency domain, which can extract the most important frequency components in the signal. MFCC features are actually the first few coefficients of the DCT result of the Mel frequency energy spectrum, which contain the most discernible information in the audio signal.

[0077] Finally, by comparing the MFCC features of the two audio segments, the difference between them can be calculated. Common comparison methods include Euclidean distance and cosine similarity. Euclidean distance measures the straight-line distance between two sets of feature values ​​in multidimensional space, while cosine similarity measures the angle between two sets of feature values, which is very useful when comparing directional similarities. Through these comparisons, we can quantify the similarity or difference between two audio segments to determine whether they belong to the same category or whether there are significant differences. This method has a wide range of applications in many fields such as speech recognition, audio classification, and anomaly detection.

[0078] Optionally, predict the audio information of subsequent days, and then determine the faulty audio by comparing the predicted data (second audio data) with the actual sampled data (first audio data), and mark the audio data as normal data and abnormal data;

[0079] Step S206: When it is determined that the sound source device has a fault, determine the similarity between the first audio data and the historical fault audio data in the fault database.

[0080] In the technical solution provided in step S206, determining the similarity between the first audio data and the historical fault audio data in the fault database includes: obtaining a first spectrum graph of the first audio data and a second spectrum graph of the historical fault audio data; extracting feature points in the first spectrum graph as first coordinate points in a unified coordinate system, and extracting feature points in the second spectrum graph as second coordinate points in the unified coordinate system, wherein the feature points include peak points and valley points, and the first coordinate points and the second coordinate points correspond one to one; calculating the distance between the first coordinate point in the first spectrum graph and the second coordinate point corresponding to the first coordinate point in the second spectrum graph, and determining the distance between the first spectrum graph and the second spectrum graph based on the distance between the first coordinate point in the first spectrum graph and the second coordinate point corresponding to the first coordinate point in the second spectrum graph, wherein the value of the distance between the first spectrum graph and the second spectrum graph is the similarity between the first audio data and each historical fault audio data.

[0081] Optionally, determine the time period in which the abnormal data is located, and compare the abnormal data with the audio in the fault library corresponding to the time period to determine the similarity between the first audio data and the historical fault audio data in the fault database. The similarity is calculated as follows: convert the obtained abnormal data into a spectrum graph, perform image similarity analysis with the spectrum graph in the fault library, place the two sets of spectrum graphs in a unified coordinate system, calculate the distance between the corresponding coordinate points in the two spectrum graphs, and use the Euclidean distance or Mann distance calculation method to determine the distance between the two spectrum graphs, thereby obtaining the corresponding similarity.

[0082] Specifically, in a unified coordinate system, the corresponding coordinate points in the two spectrum graphs can be determined by the following method:

[0083] First, divide the spectrum graph into a grid, and each grid point corresponds to a coordinate point;

[0084] Then extract the characteristic points in the spectrum graph, such as peak points, valley points, and other points whose corresponding coordinates in the two graphs;

[0085] Then use Euclidean distance or Manhattan distance to calculate the distance between corresponding coordinate points in two spectrum graphs.

[0086] Step S208, determine the target fault data from the historical fault audio data based on the similarity, and determine the fault type of the sound source device based on the fault type corresponding to the target fault data, wherein the working time period corresponding to the historical fault audio data is the same as the working time period corresponding to the first audio data, and the target fault data is the historical fault audio data with the highest similarity to the first audio data in the fault database.

[0087] In the technical solution provided in step S208, determining the fault type of the sound source device based on the fault type corresponding to the target fault data includes: when the similarity of the target fault data is greater than a preset threshold, determining the fault type of the sound source device as the fault type corresponding to the target fault data; when the similarity of the target fault data is less than a preset threshold, determining the fault type of the sound source device as a newly added fault type.

[0088] Optionally, after determining that the fault type of the sound source device is a newly added fault type, the method also includes: sending the first audio data to a manual identification platform, the manual identification platform being used to display the first audio data of the fault type that requires manual identification to the target object; obtaining the fault type returned by the manual identification platform, and updating the fault database.

[0089] Optionally, the fault type corresponding to the audio with the highest similarity is determined as the fault type of the abnormal data. For abnormal data with the highest similarity but not meeting the corresponding requirements, the fault type corresponding to the abnormal data is recorded as a newly added abnormality, and the abnormal information is sent to the manual platform to determine the fault type through manual identification, and update the fault library at the same time. In fact, the fault types included in the fault library are not complete, and it may also appear that when calculating the similarity, the highest similarity does not meet the corresponding requirements (for example, the highest similarity is also very low), which means that the fault type corresponding to the abnormal data is a newly added abnormality (that is, a fault type that is not recorded in the fault library), so it should be sent to the manual platform, the fault type is determined through manual identification, and the fault library is updated.

[0090] Optionally, the prediction model is set in a sensor, wherein the sensor is used to collect data and to perform local real-time fault detection based on the predicted audio data output by the prediction model, and the sensor supports detection in an offline state.

[0091] Optionally, the prediction model is embedded in the intelligent sensor on the coal mine collection production line to detect the machine's sound information data on site, analyze the data and make real-time predictions. The sensor realizes the end-side edge computing, and the equipment can analyze it locally after collection. The fault can be judged locally in time without transmitting the audio occupied bandwidth back to the server. It reduces latency, saves bandwidth, and supports working in offline state.

[0092] Through the above steps, a detection method for large-scale professional equipment in the coal mining field such as coal mine roller chains can be implemented, which combines the audio front-end acquisition and processing technology and the artificial intelligence machine learning analysis algorithm, specifically including the front-end sound pickup method, the artificial intelligence audio classification algorithm, etc., to achieve unattended operation, intelligent judgment on the terminal side, and save network bandwidth. Specifically, the embodiment of the present application has the following advantages:

[0093] 1. Use the microphone array pickup model to collect and record the underground sound field information, locate the sound source, and collect the sound field, thus achieving a better noise reduction performance solution.

[0094] 2. The acoustic + artificial intelligence algorithm is used to overcome and reduce interference factors in the coal mine site environment such as dim light and heavy dust underground, replacing manual experience and other means. Artificial intelligence automatically analyzes and judges all the time, saving manpower.

[0095] 3. With high degree of automation and good effectiveness, it can predict fault points in advance based on big data analysis, realize early warning, improve accuracy and coverage, and reduce production losses caused by faults.

[0096] 4. The embodiment of the present application adopts a transfer learning algorithm, which has a further rapid convergence of accuracy improvement compared with previous methods, greatly reducing the required training data. At the same time, it simplifies the cost of early data collection and the requirement for data volume, reduces the time and cost of previous collection and training, greatly improves training efficiency and reduces dependence on data collection.

[0097] 5. The embodiments of the present application perform special processing on the dynamic differences between different machines or different production states of the same type of machines, thereby reducing the impact of distinguishing features of different models or different production states of the same model on the accuracy of the prediction results, and predicting the audio data of other machines through feature change factors, thereby improving the adaptability of the prediction model to machines of different manufacturers and the prediction accuracy of different production states of the same model.

[0098] 6. Use smart sensors with end-side edge computing capabilities in coal mine collection production lines to perform real-time fault prediction, reduce latency, save bandwidth, perform real-time analysis, and quickly detect faults.

[0099] The present application provides a fault detection device. Figure 4 is a schematic diagram of the structure of the device, such as Figure 4 As shown, the device includes: a first acquisition module 40, which is used to acquire the first audio data of the sound source device collected by the audio acquisition device and the second audio data output by the prediction model, and determine the difference value between the first audio data and the second audio data, wherein the prediction model is obtained based on historical audio data training, and is used to predict the audio data of the sound source device in each working time period, and the working state of the sound source device in each working time period is not completely the same; a first determination module 42, which is used to determine that the sound source device has a fault when the difference value exceeds a preset threshold; a second determination module 44, which is used to determine the similarity between the first audio data and the historical fault audio data in the fault database when it is determined that the sound source device has a fault; a third determination module 46, which is used to determine the target fault data from the historical fault audio data based on the similarity, and determine the fault type of the sound source device based on the fault type corresponding to the target fault data, wherein the working time period corresponding to the historical fault audio data is the same as the working time period corresponding to the first audio data, and the target fault data is the historical fault audio data in the fault database with the highest similarity to the first audio data.

[0100] According to another aspect of the embodiment of the present application, the third determination module 46 determines the fault type of the sound source device based on the fault type corresponding to the target fault data, including: when the similarity of the target fault data is greater than a preset threshold, determining that the fault type of the sound source device is the fault type corresponding to the target fault data; when the similarity of the target fault data is less than a preset threshold, determining that the fault type of the sound source device is a newly added fault type.

[0101] According to another aspect of an embodiment of the present application, before the first acquisition module 40 acquires the second audio data output by the prediction model, the method also includes: acquiring historical audio data, and extracting feature data of the historical audio data through Mel-frequency cepstral coefficients; constructing a time series based on the feature data, wherein the time series is divided into multiple groups, and the multiple groups of time series correspond to different working time periods respectively; training a basic model based on the time series to obtain a prediction model, wherein the basic model is a pre-trained long short-term memory network model.

[0102] According to another aspect of an embodiment of the present application, the sound source device includes a first device and a second device, wherein the historical device audio data used for prediction model training is the historical device audio data of the first device; obtaining the second audio data output by the prediction model includes: when the sound source device is the first device, using the data output by the prediction model as the second audio data; when the sound source device is the second device, obtaining the data output by the prediction model, and converting the data into second audio data based on a feature change factor, wherein the feature change factor is used to represent the mapping relationship between the audio data of the first device and the audio data of the second device.

[0103] According to another aspect of an embodiment of the present application, the characteristic change factor is determined in the following manner: obtaining third audio data of a first device, segmenting the third audio data, and obtaining a first data frame of the third audio data, wherein the first data frame is arranged according to a time series; performing feature extraction on the first data frame to obtain a first feature vector sequence; performing mean processing on multiple groups of first feature vector sequences to obtain a standardized first feature vector sequence; obtaining fourth audio data of a second device, segmenting the fourth audio data, and obtaining a second data frame of the fourth audio data, wherein the second data frame is arranged according to a time series; performing feature extraction on the second data frame to obtain a second feature vector sequence; performing mean processing on multiple groups of second feature vector sequences to obtain a standardized second feature vector sequence; and determining the characteristic change factors of the first device and the second device based on the discrete relationship between the standardized first feature vector sequence and the standardized second feature vector sequence.

[0104] According to another aspect of the embodiment of the present application, the second determination module 44 determines the similarity between the first audio data and the historical fault audio data in the fault database, including: obtaining a first spectrum graph of the first audio data and a second spectrum graph of the historical fault audio data; extracting feature points in the first spectrum graph as first coordinate points in a unified coordinate system, and extracting feature points in the second spectrum graph as second coordinate points in the unified coordinate system, wherein the feature points include peak points and valley points, and the first coordinate points and the second coordinate points correspond one to one; calculating the distance between the first coordinate point in the first spectrum graph and the second coordinate point corresponding to the first coordinate point in the second spectrum graph, and determining the distance between the first spectrum graph and the second spectrum graph based on the distance between the first coordinate point in the first spectrum graph and the second coordinate point corresponding to the first coordinate point in the second spectrum graph, wherein the value of the distance between the first spectrum graph and the second spectrum graph is the similarity between the first audio data and each historical fault audio data.

[0105] According to another aspect of an embodiment of the present application, a prediction model is set in a sensor, wherein the sensor is used to collect data and to perform local real-time fault detection based on the predicted audio data output by the prediction model, and the sensor supports detection in an offline state.

[0106] According to another aspect of an embodiment of the present application, after determining that the fault type of the sound source device is a newly added fault type, the method also includes: sending the first audio data to a manual identification platform, the manual identification platform being used to display the first audio data of the fault type that requires manual identification to the target object; obtaining the fault type returned by the manual identification platform, and updating the fault database.

[0107] According to another aspect of an embodiment of the present application, the audio acquisition device includes a microphone array, and obtaining first audio data of a sound source device collected by the audio acquisition device includes: when the operating amplitude of the device is less than or equal to a preset amplitude threshold, obtaining the first audio data of the sound source device collected by the microphone array arranged by a near-field model; when the operating amplitude of the device is greater than the preset amplitude threshold, obtaining the first audio data of the sound source device collected by the microphone array arranged by a far-field model.

[0108] It should be noted that the various modules in the above-mentioned fault detection device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0109] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0110] An embodiment of the present application provides a non-volatile storage medium, in which a program is stored, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the following fault detection method: obtaining first audio data of a sound source device collected by an audio collection device and second audio data output by a prediction model, and determining a difference value between the first audio data and the second audio data, wherein the prediction model is obtained based on historical audio data training and is used to predict the audio data of the sound source device in each working time period, and the working states of the sound source device in each working time period are not completely the same; when the difference value exceeds a preset threshold, determining that the sound source device has a fault; when it is determined that the sound source device has a fault, determining the similarity between the first audio data and the historical fault audio data in a fault database; determining target fault data from the historical fault audio data based on the similarity, and determining the fault type of the sound source device based on the fault type corresponding to the target fault data, wherein the working time period corresponding to the historical fault audio data is the same as the working time period corresponding to the first audio data, and the target fault data is the historical fault audio data in the fault database that has the highest similarity to the first audio data.

[0111] An embodiment of the present application provides an electronic device, including: a memory and a processor, the processor is used to run a program stored in the memory, wherein the following fault detection method is executed when the program is run: obtaining first audio data of a sound source device collected by an audio collection device and second audio data output by a prediction model, and determining a difference value between the first audio data and the second audio data, wherein the prediction model is obtained based on historical audio data training and is used to predict audio data of the sound source device in each working time period, and the working states of the sound source device in each working time period are not completely the same; when the difference value exceeds a preset threshold, determining that the sound source device has a fault; when it is determined that the sound source device has a fault, determining the similarity between the first audio data and the historical fault audio data in a fault database; determining target fault data from the historical fault audio data based on the similarity, and determining the fault type of the sound source device based on the fault type corresponding to the target fault data, wherein the working time period corresponding to the historical fault audio data is the same as the working time period corresponding to the first audio data, and the target fault data is the historical fault audio data in the fault database that has the highest similarity to the first audio data.

[0112] An embodiment of the present application provides a computer program product, including a computer program, which implements the following fault detection method when executed by a processor: obtaining first audio data of a sound source device collected by an audio collection device and second audio data output by a prediction model, and determining a difference value between the first audio data and the second audio data, wherein the prediction model is obtained based on historical audio data training and is used to predict the audio data of the sound source device in each working time period, and the working states of the sound source device in each working time period are not completely the same; when the difference value exceeds a preset threshold, determining that the sound source device has a fault; when it is determined that the sound source device has a fault, determining the similarity between the first audio data and the historical fault audio data in a fault database; determining target fault data from the historical fault audio data based on the similarity, and determining the fault type of the sound source device based on the fault type corresponding to the target fault data, wherein the working time period corresponding to the historical fault audio data is the same as the working time period corresponding to the first audio data, and the target fault data is the historical fault audio data in the fault database that has the highest similarity to the first audio data.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0114] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0115] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0116] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0117] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A fault detection method, characterized in that: include: Acquire first audio data of a sound source device collected by an audio collection device and second audio data output by a prediction model, and determine a difference value between the first audio data and the second audio data, wherein the prediction model is obtained by training based on historical audio data and is used to predict the audio data of the sound source device in each working time period, and the working states of the sound source device in each working time period are not completely the same; When the difference value exceeds a preset threshold, determining that the sound source device has a fault; In the case where it is determined that the sound source device has a fault, determining the similarity between the first audio data and historical fault audio data in a fault database; Determine target fault data from the historical fault audio data based on the similarity, and determine the fault type of the sound source device based on the fault type corresponding to the target fault data, wherein the working time period corresponding to the historical fault audio data is the same as the working time period corresponding to the first audio data, and the target fault data is the historical fault audio data in the fault database that has the highest similarity to the first audio data.

2. The fault detection method according to claim 1, characterized in that: Determining the fault type of the sound source device according to the fault type corresponding to the target fault data includes: When the similarity of the target fault data is greater than a preset threshold, the fault type of the sound source device is determined to be the fault type corresponding to the target fault data; when the similarity of the target fault data is less than a preset threshold, the fault type of the sound source device is determined to be a newly added fault type.

3. The fault detection method according to claim 1, characterized in that: Before acquiring the second audio data output by the prediction model, the method further includes: Acquire historical audio data, and extract feature data of the historical audio data through Mel-frequency cepstrum coefficients; Constructing a time series according to the characteristic data, wherein the time series is divided into multiple groups, and the multiple groups of time series correspond to different working time periods respectively; The prediction model is obtained by training a basic model according to the time series, wherein the basic model is a pre-trained long short-term memory network model.

4. The fault detection method according to claim 1, characterized in that: The sound source device includes a first device and a second device, wherein the historical device audio data used in the prediction model training is the historical device audio data of the first device; and obtaining the second audio data output by the prediction model includes: When the sound source device is the first device, using the data output by the prediction model as the second audio data; When the sound source device is the second device, data output by the prediction model is obtained, and the data is converted into the second audio data based on a feature change factor, wherein the feature change factor is used to represent a mapping relationship between the audio data of the first device and the audio data of the second device.

5. The fault detection method according to claim 4, characterized in that: The characteristic variation factor is determined by: Acquire third audio data of the first device, and segment the third audio data to obtain first data frames of the third audio data, wherein the first data frames are arranged in time series; Performing feature extraction on the first data frame to obtain a first feature vector sequence; Performing mean processing on multiple groups of the first feature vector sequences to obtain standardized first feature vector sequences; Acquire fourth audio data of the second device, and segment the fourth audio data to obtain second data frames of the fourth audio data, wherein the second data frames are arranged in a time series; Performing feature extraction on the second data frame to obtain a second feature vector sequence; Performing mean processing on multiple groups of the second feature vector sequences to obtain standardized second feature vector sequences; The characteristic variation factors of the first device and the second device are determined according to a discrete relationship between the standardized first characteristic vector sequence and the standardized second characteristic vector sequence.

6. The fault detection method according to claim 1, characterized in that: Determining the similarity between the first audio data and the historical fault audio data in the fault database includes: Obtain a first spectrum graph of the first audio data and a second spectrum graph of the historical fault audio data; extract characteristic points in the first spectrum graph as first coordinate points in a unified coordinate system, and extract characteristic points in the second spectrum graph as second coordinate points in the unified coordinate system, wherein the characteristic points include peak points and valley points, and the first coordinate points correspond to the second coordinate points one by one; Calculate the distance between a first coordinate point in the first spectrum graph and a second coordinate point corresponding to the first coordinate point in the second spectrum graph, and determine the distance between the first spectrum graph and the second spectrum graph based on the distance between the first coordinate point in the first spectrum graph and the second coordinate point corresponding to the first coordinate point in the second spectrum graph, wherein the value of the distance between the first spectrum graph and the second spectrum graph is the similarity between the first audio data and each of the historical fault audio data.

7. The fault detection method according to claim 1, characterized in that: The method further comprises: The prediction model is set in a sensor, wherein the sensor is used to collect data and to perform local real-time fault detection based on the predicted audio data output by the prediction model, and the sensor supports detection in an offline state.

8. The fault detection method according to claim 2, characterized in that: After determining that the fault type of the sound source device is a newly added fault type, the method further includes: Sending the first audio data to a manual identification platform, where the manual identification platform is used to display the first audio data requiring manual identification of the fault type to a target object; The fault type returned by the manual identification platform is obtained, and the fault database is updated.

9. The fault detection method according to claim 1, characterized in that: The audio acquisition device includes a microphone array, and obtaining first audio data of a sound source device acquired by the audio acquisition device includes: When the working amplitude of the device is less than or equal to a preset amplitude threshold, obtaining first audio data of the sound source device collected by a microphone array arranged by a near-field model; When the working amplitude of the device is greater than a preset amplitude threshold, first audio data of the sound source device collected by a microphone array arranged by a far-field model is obtained.

10. A fault detection device, characterized in that: include: A first acquisition module is used to acquire first audio data of a sound source device collected by an audio acquisition device and second audio data output by a prediction model, and determine a difference value between the first audio data and the second audio data, wherein the prediction model is obtained by training based on historical audio data and is used to predict the audio data of the sound source device in each working time period, and the working states of the sound source device in each working time period are not completely the same; A first determination module, configured to determine that a fault exists in the sound source device when the difference value exceeds a preset threshold; A second determination module is used to determine the similarity between the first audio data and the historical fault audio data in the fault database when it is determined that the sound source device has a fault; A third determination module is used to determine target fault data from the historical fault audio data based on the similarity, and determine the fault type of the sound source device based on the fault type corresponding to the target fault data, wherein the working time period corresponding to the historical fault audio data is the same as the working time period corresponding to the first audio data, and the target fault data is the historical fault audio data in the fault database that has the highest similarity to the first audio data.

11. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the fault detection method according to any one of claims 1 to 9.

12. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the fault detection method according to any one of claims 1 to 9 is executed when the program is run.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the fault detection method according to any one of claims 1 to 9 are implemented.

Images