Equipment fault diagnosis and positioning method and system based on microphone array

Through the fault diagnosis and positioning method based on microphone array, combined with three-dimensional laser scanner and neural network, the positioning difficulties in the noise environment during equipment fault diagnosis are solved, the precise positioning of equipment faults and the automatic identification of components are achieved, and the work is improved efficiency.

CN119049511AActive Publication Date: 2024-11-29FANDE INTELLIGENT TESTING TECHNOLOGY (SHANGHAI) CO LTD

Patent Information

Application Number
CN202411134381.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-29
Estimated Expiration
2044-08-19

AI Technical Summary

Technical Problem

Existing equipment fault diagnosis technology is difficult to accurately determine abnormal parts in complex noise environments, and linear or planar microphone arrays have limitations in sound source positioning, affecting the accuracy of fault positioning.

Method used

Using equipment fault diagnosis and positioning methods based on microphone arrays, the sound signal data is obtained for preprocessing and sound source separation by establishing a spatial coordinate system, combining a three-dimensional laser scanner to obtain component point cloud data, and using neural network and sound source positioning model Calculate the coordinates of the fault point to achieve accurate positioning of the faulty components.

Benefits of technology

It improves the separation accuracy of fault sound and the positioning accuracy of components, realizes the accurate pairing of automated fault points and components, and improves work efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119049511A_ABST
    Figure CN119049511A_ABST
Patent Text Reader

Abstract

The invention relates to an equipment fault diagnosis and positioning method based on a microphone array, and belongs to the technical field of sound recognition and fault diagnosis. The method comprises the following steps: establishing a space coordinate system, determining space coordinates of microphones, and calculating the distance between the microphones; acquiring equipment sound data through the microphone array, and preprocessing the sound data through the preprocessing model; separating the preprocessed sound data through a sound source separation model to obtain separation signal data, and performing fault diagnosis on the separation signal data to obtain fault separation signal data; performing calculation according to the fault separation signal data and the distance data to obtain fault point coordinate data, obtaining equipment component point cloud data through a three-dimensional laser scanner, identifying the point cloud data to obtain an identification result, and storing the identification result and the coordinate data into a database; and performing database retrieval according to the fault point coordinate data to obtain a fault component name. And equipment fault diagnosis and positioning based on sound are realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of sound recognition and fault diagnosis, and specifically relates to a device fault diagnosis and positioning method and system based on a microphone array. Background Art

[0002] Equipment fault diagnosis technology is a comprehensive discipline. After decades of development, it has gradually formed a relatively complete discipline system with vibration diagnosis, temperature monitoring and non-destructive testing as the main methods, and other technologies or methods as the auxiliary methods. Modern equipment fault diagnosis technology not only includes a variety of methods such as model-based, signal-based, knowledge-based and intelligent optimization-based, but also covers the latest research progress and engineering practical technologies such as fuzzy mathematics, artificial neural networks, expert systems, wavelet transforms, and information fusion. These technologies are widely used in machinery, energy, transportation, aerospace and other fields, and are of great significance to improving equipment operation efficiency and extending service life.

[0003] However, the existing equipment fault diagnosis technology has the following problems:

[0004] (1) When diagnosing faults by collecting the operating sounds of large equipment, the operating sounds of different components of the equipment are combined into an integrated sound data and collected by the sound collection device. Diagnosing faults of the equipment based on the integrated sound can only determine whether the equipment is abnormal, but cannot determine the abnormal part.

[0005] (2) Currently, microphone arrays are arranged in a planar or linear manner. However, microphone arrays arranged in this manner have limitations in sound source localization. In complex noise environments, linear or planar microphone arrays are difficult to distinguish between target sounds and background noise, which affects the quality of speech recognition and sound processing. Summary of the invention

[0006] In order to solve the above problems existing in the prior art, the present invention provides a device fault diagnosis and positioning method and system based on a microphone array.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] 1. A device fault diagnosis and location method based on a microphone array, characterized by comprising:

[0009] S1: establishing a spatial coordinate system, determining the spatial coordinates of each microphone in the microphone array according to the spatial coordinate system, and calculating the distance data between every two microphones according to the spatial coordinates;

[0010] S2: Acquire sound signal data of the device through the microphone array, transmit the sound signal data to the smart terminal, and the smart terminal calculates preprocessed sound data according to the sound signal data through a preprocessing model;

[0011] S3: the intelligent terminal calculates separation signal data according to the pre-processed sound data by using a sound source separation model, and calculates fault separation signal data according to the separation signal data by using a fault diagnosis model;

[0012] S4: Calculate the coordinate data of the fault point through the sound source localization model according to the fault separation signal data and the distance data; obtain the component point cloud data and component space coordinates of the device through a three-dimensional laser scanner, calculate the component recognition result through the recognition model according to the component point cloud data, and store the component space coordinates in the database according to the component recognition result;

[0013] S5: The intelligent terminal retrieves the database according to the fault point coordinate data to obtain the name of the fault component, and transmits the name of the fault component to the human-computer interaction interface through the internal interface.

[0014] Specifically, the preprocessing model is used to remove reverberation and enhance the signal of the sound signal data, wherein the reverberation removal uses the signal received in the past time to predict the late reverberation at the tail of the measured signal, and then subtracts the late reverberation signal from the measured signal to achieve the reverberation removal function, and the specific steps are:

[0015] S201: First, obtain time-frequency domain signal data through short-time Fourier transform according to the sound signal data. The time-frequency domain signal expression is:

[0016]

[0017] Among them, X m (n, k) represents the time-frequency domain signal of the mth microphone, n is the number of signal frames, k is the number of frequency points of the signal, L is the signal length, l is the variable parameter, S(nl, k) represents the sound source signal after being transformed into the time-frequency domain by short-time Fourier transform; r m (l, k) represents the room impulse response of the mth microphone in the array and the sound source;

[0018] S202: Using the current delay to distinguish the early reverberation signal and the late reverberation signal of the time-frequency domain signal to obtain a distinguished time-frequency domain signal, wherein the expression of the distinguished time-frequency domain signal is:

[0019]

[0020] Among them, X m(n, k) represents the time-frequency domain signal of the mth microphone, n is the number of signal frames, k is the number of frequency points of the signal, L is the signal length, l is the variable parameter, S(nl, k) represents the sound source signal after being transformed into the time-frequency domain by short-time Fourier transform; r m (l, k) represents the room impulse response of the mth microphone in the array and the sound source, and a is the linear delay;

[0021] S203: Finally, the sound signal data is subtracted from the late reverberation signal to obtain dereverberation sound data. The dereverberation sound data is expressed as:

[0022]

[0023] Where D is the dedeverberation sound data, X m (n, k) represents the time-frequency domain signal of the mth microphone, n is the number of signal frames, k is the number of frequency points of the signal, L is the signal length, l is the variable parameter, S(nl, k) represents the sound source signal after being transformed into the time-frequency domain by short-time Fourier transform; r m (l, k) represents the room impulse response of the mth microphone in the array and the sound source, and a is the linear delay;

[0024] Specifically, the signal enhancement performs modal decomposition on the dereverberation sound data by an empirical mode decomposition method to obtain an intrinsic mode function component, calculates the correlation coefficient between the intrinsic mode function component and the dereverberation sound data by a correlation coefficient method, and compares the correlation coefficient with a preset threshold to obtain a comparison result, and according to the comparison result, removes pseudo components with low correlation coefficients in the intrinsic mode data components to obtain the preprocessed sound data, wherein the correlation coefficient calculation formula is:

[0025]

[0026] Among them, Q is the correlation coefficient, t is the variable parameter, T is the number of intrinsic mode functions, and Y

[0027] (t) is the dereverberation sound data, h i (t) is the intrinsic mode function.

[0028] Specifically, the sound source separation model decomposes the frequency spectrum of the preprocessed sound data into a non-negative matrix decomposition basis, and obtains a separation signal amplitude spectrum through non-negative matrix decomposition basis clustering according to the spatial characteristics of the non-negative matrix decomposition basis, and then obtains the separation signal data according to the separation signal amplitude spectrum and the phase spectrum of the preprocessed sound data.

[0029] Specifically, the fault diagnosis model is implemented based on a neural network, and the specific steps are as follows:

[0030] S501: firstly, performing frame division and windowing processing on the separated signal data to obtain processed signal data, and then performing Fourier transform on the processed signal data to obtain transformed data;

[0031] S502: Acquire spectral line energy data of the transformed data, and obtain filtered energy data by calculation according to the spectral line energy data. The filter energy calculation formula is:

[0032]

[0033] Among them, S(i) is the filter energy, E(i, k) is the spectral line energy data, k is the variable parameter, H(k) is the filter expression, and N is the number of frames;

[0034] S503: Calculate characteristic parameters according to the filter energy data and the discrete cosine transform cepstrum calculation formula, wherein the discrete cosine transform cepstrum calculation formula is:

[0035]

[0036] Wherein, Mf(i,n) is the characteristic parameter, i represents the i-th frame, n represents the spectrum line after DCT, m represents the m-th filter, S(i,m) is the filter energy in the m-th filter of the i-th frame, and M is the number of frames;

[0037] S504: Calculate the differential characteristic parameters using a differential formula according to the characteristic parameters, and the differential formula is expressed as:

[0038]

[0039] Among them, d t is the tth first-order difference, C t is the tth characteristic parameter, C t+1 is the t+1th characteristic parameter, C t-1 is the t-1th characteristic parameter, C t+k is the t+kth characteristic parameter, C t-k is the tkth characteristic parameter, K is the first-order derivative time difference, k is the variable parameter, and Q is the order of the characteristic parameter;

[0040] S505: The differential feature parameters are input into the neural network. After receiving the differential feature parameters, the input layer of the neural network passes the differential feature parameters to the hidden layer. The hidden layer calculates the hidden layer output result according to the differential feature parameters, connection weights, thresholds and activation functions, and uses the hidden layer output result as the input of the output layer. The fault separation signal data is calculated according to the hidden layer output result, the connection weights between the output layers and the output layer threshold.

[0041] Specifically, the sound source localization model calculation steps are as follows:

[0042] S601: intercepting a signal spectrum of equal length from a spectrum according to a signal bandwidth of the fault separation signal data to obtain effective spectrum data, and performing non-integer multiple downsampling on the effective spectrum data to obtain fixed-length spectrum data;

[0043] S602: normalizing the fixed-length spectrum data to obtain normalized spectrum data, and performing secondary smoothing on the normalized spectrum data to obtain smoothed spectrum data;

[0044] S603: Obtain regularized cost path data of the smoothed spectrum data between the main microphone and the auxiliary microphone by calculating through a regularized cost path algorithm, and determine the same fault signal according to the regularized cost path data, wherein the regularized cost path algorithm calculation formula is:

[0045]

[0046] Where DTW(.) is the regularized cost path data, S i is the i-th smoothed spectrum data of the main microphone, Q xy is the yth smoothed spectrum data of the xth auxiliary microphone, k is a variable parameter, K is the path length of the smoothed spectrum data, and d(pk) represents the distance between two points S i and Q xy The measured distance between

[0047] S604: Calculate the time difference data between the same fault signal reaching the main microphone and each of the auxiliary microphones according to the cross-correlation function, and calculate the fault point coordinate data through a coordinate calculation model according to the time difference data and the distance data.

[0048] Specifically, the specific calculation steps of the recognition model are:

[0049] S701: performing thinning and smoothing processing on the component point cloud data to obtain pre-processed point cloud data;

[0050] S702: Obtain segmented point cloud data by using a progressive morphological filtering algorithm according to the pre-processed point cloud data;

[0051] S703: Calculating a covariance matrix according to the segmented point cloud data, performing principal component analysis on the covariance matrix to obtain a normal vector, and performing boundary separation on the segmented point cloud data according to the normal vector and a preset boundary determination angle threshold to obtain discrete boundary point cloud data;

[0052] S704: Obtain component outline data through a radius filtering algorithm according to the discrete boundary point cloud data, and compare the component outline data with the component outline data in a database to obtain the component recognition result.

[0053] A device fault diagnosis and positioning system based on a microphone array, comprising:

[0054] A coordinate establishment module, used to establish a spatial coordinate system, determine the spatial coordinates of each microphone in the microphone array according to the spatial coordinate system, and calculate the distance data between every two microphones according to the spatial coordinates;

[0055] The data acquisition module is used to acquire the sound signal data of the device through the microphone array, and transmit the sound signal data to the smart terminal, and the smart terminal calculates the preprocessed sound data according to the sound signal data through the preprocessing model;

[0056] The separation module module is used for the intelligent terminal to obtain separation signal data by using a sound source separation model according to the pre-processed sound data, and to obtain fault separation signal data by using a fault diagnosis model according to the separation signal data;

[0057] The positioning module is used to calculate the fault point coordinate data through the sound source positioning model according to the fault separation signal data and the distance data; obtain the component point cloud data and component space coordinates of the device through a three-dimensional laser scanner, calculate the component recognition result through the recognition model according to the component point cloud data, and store the component space coordinates in the database according to the component recognition result;

[0058] The interactive module is used for the intelligent terminal to retrieve the database according to the fault point coordinate data to obtain the name of the fault component, and transmit the name of the fault component to the human-computer interaction interface through the internal interface.

[0059] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the device fault diagnosis and positioning method based on a microphone array as described above is implemented.

[0060] A storage medium containing computer executable instructions, characterized in that the computer executable instructions are used to execute the device fault diagnosis and positioning method based on microphone array as described above when executed by a computer processor.

[0061] The beneficial effects of the present invention are:

[0062] (1) Six microphones are placed at equal intervals on the six sides of the device to form a stereo microphone array to collect sound data from the six sides of the device respectively. The sound data is passed through a preprocessing model to obtain preprocessed sound data, and then the sound is separated through a sound source separation model to obtain separated signal data. Then, a fault diagnosis model is used to perform fault diagnosis on each separated signal data to obtain fault separated signal data. The fault separated signal data is then transmitted to a sound source positioning model to obtain fault point coordinate data, thereby achieving separation of fault sounds and improving the precise positioning of the fault point.

[0063] (2) The point cloud data and spatial coordinates of equipment components are obtained through a three-dimensional laser scanner, and the point cloud data is input into the recognition model to obtain the component recognition results. The component recognition results and component spatial coordinates are stored in the database. The database is searched according to the coordinates of the fault point to obtain the name of the faulty component, thereby realizing the precise matching of automated fault points and components, avoiding personnel from inspecting all components inside the equipment one by one, and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0065] Figure 1 The present invention is a flowchart of a method for diagnosing and locating device faults based on a microphone array. DETAILED DESCRIPTION

[0066] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0067] See also Figure 1 , a device fault diagnosis and positioning method based on a microphone array, comprising:

[0068] 1. A device fault diagnosis and location method based on a microphone array, characterized by comprising:

[0069] S1: establishing a spatial coordinate system, determining the spatial coordinates of each microphone in the microphone array according to the spatial coordinate system, and calculating the distance data between every two microphones according to the spatial coordinates;

[0070] S2: Acquire sound signal data of the device through the microphone array, transmit the sound signal data to the smart terminal, and the smart terminal calculates preprocessed sound data according to the sound signal data through a preprocessing model;

[0071] S3: the intelligent terminal calculates separation signal data according to the pre-processed sound data by using a sound source separation model, and calculates fault separation signal data according to the separation signal data by using a fault diagnosis model;

[0072] S4: Calculate the coordinate data of the fault point through the sound source localization model according to the fault separation signal data and the distance data; obtain the component point cloud data and component space coordinates of the device through a three-dimensional laser scanner, calculate the component recognition result through the recognition model according to the component point cloud data, and store the component space coordinates in the database according to the component recognition result;

[0073] S5: The intelligent terminal retrieves the database according to the fault point coordinate data to obtain the name of the fault component, and transmits the name of the fault component to the human-computer interaction interface through the internal interface.

[0074] In this embodiment, the microphone array is composed of six microphones, which are located in six directions of the detection device at the same distance. The microphones located directly in front and directly above the device are set as main microphones, and the remaining four microphones are auxiliary microphones. The front microphone, the left microphone, the right microphone and the rear microphone constitute a four-element cross microphone array No. 1, and the top microphone, the bottom microphone, the left microphone and the right microphone constitute a four-element cross microphone array No. 2.

[0075] Specifically, the preprocessing model is used to remove reverberation and enhance the signal of the sound signal data, wherein the reverberation removal uses the signal received in the past time to predict the late reverberation at the tail of the measured signal, and then subtracts the late reverberation signal from the measured signal to achieve the reverberation removal function, and the specific steps are:

[0076] S201: First, obtain time-frequency domain signal data through short-time Fourier transform according to the sound signal data. The time-frequency domain signal expression is:

[0077]

[0078] Among them, X m (n, k) represents the time-frequency domain signal of the mth microphone, n is the number of signal frames, k is the number of frequency points of the signal, L is the signal length, l is the variable parameter, S(nl, k) represents the sound source signal after being transformed into the time-frequency domain by short-time Fourier transform; r m (l, k) represents the room impulse response of the mth microphone in the array and the sound source;

[0079] S202: Using the current delay to distinguish the early reverberation signal and the late reverberation signal of the time-frequency domain signal to obtain a distinguished time-frequency domain signal, wherein the expression of the distinguished time-frequency domain signal is:

[0080]

[0081] Among them, X m (n, k) represents the time-frequency domain signal of the mth microphone, n is the number of signal frames, k is the number of frequency points of the signal, L is the signal length, l is the variable parameter, S(nl, k) represents the sound source signal after being transformed into the time-frequency domain by short-time Fourier transform; r m (l, k) represents the room impulse response of the mth microphone in the array and the sound source, and a is the linear delay;

[0082] S203: Finally, the sound signal data is subtracted from the late reverberation signal to obtain dereverberation sound data. The dereverberation sound data is expressed as:

[0083]

[0084] Where D is the dedeverberation sound data, X m (n, k) represents the time-frequency domain signal of the mth microphone, n is the number of signal frames, k is the number of frequency points of the signal, L is the signal length, l is the variable parameter, S(nl, k) represents the sound source signal after being transformed into the time-frequency domain by short-time Fourier transform; r m (l, k) represents the room impulse response of the mth microphone in the array and the sound source, and a is the linear delay;

[0085] Specifically, the signal enhancement performs modal decomposition on the dereverberation sound data by an empirical mode decomposition method to obtain an intrinsic mode function component, calculates the correlation coefficient between the intrinsic mode function component and the dereverberation sound data by a correlation coefficient method, and compares the correlation coefficient with a preset threshold to obtain a comparison result, and according to the comparison result, removes pseudo components with low correlation coefficients in the intrinsic mode data components to obtain the preprocessed sound data, wherein the correlation coefficient calculation formula is:

[0086]

[0087] Among them, Q is the correlation coefficient, t is the variable parameter, T is the number of intrinsic mode functions, and Y

[0088] (t) is the dereverberation sound data, h i (t) is the intrinsic mode function.

[0089] Specifically, the sound source separation model decomposes the frequency spectrum of the preprocessed sound data into a non-negative matrix decomposition basis, and obtains a separation signal amplitude spectrum through non-negative matrix decomposition basis clustering according to the spatial characteristics of the non-negative matrix decomposition basis, and then obtains the separation signal data according to the separation signal amplitude spectrum and the phase spectrum of the preprocessed sound data.

[0090] Specifically, the fault diagnosis model is implemented based on a neural network, and the specific steps are as follows:

[0091] S501: firstly, performing frame division and windowing processing on the separated signal data to obtain processed signal data, and then performing Fourier transform on the processed signal data to obtain transformed data;

[0092] S502: Acquire spectral line energy data of the transformed data, and obtain filtered energy data by calculation according to the spectral line energy data. The filter energy calculation formula is:

[0093]

[0094] Among them, S(i) is the filter energy, E(i, k) is the spectral line energy data, k is the variable parameter, H(k) is the filter expression, and N is the number of frames;

[0095] S503: Calculate characteristic parameters according to the filter energy data and the discrete cosine transform cepstrum calculation formula, wherein the discrete cosine transform cepstrum calculation formula is:

[0096]

[0097] Wherein, Mf(i,n) is the characteristic parameter, i represents the i-th frame, n represents the spectrum line after DCT, m represents the m-th filter, S(i,m) is the filter energy in the m-th filter of the i-th frame, and M is the number of frames;

[0098] S504: Calculate the differential characteristic parameters using a differential formula according to the characteristic parameters, and the differential formula is expressed as:

[0099]

[0100] Among them, d t is the tth first-order difference, C t is the tth characteristic parameter, C t+1 is the t+1th characteristic parameter, C t-1 is the t-1th characteristic parameter, C t+k is the t+kth characteristic parameter, C t-k is the tkth characteristic parameter, K is the first-order derivative time difference, k is the variable parameter, and Q is the order of the characteristic parameter;

[0101] S505: The differential feature parameters are input into the neural network. After receiving the differential feature parameters, the input layer of the neural network passes the differential feature parameters to the hidden layer. The hidden layer calculates the hidden layer output result according to the differential feature parameters, connection weights, thresholds and activation functions, and uses the hidden layer output result as the input of the output layer. The fault separation signal data is calculated according to the hidden layer output result, the connection weights between the output layers and the output layer threshold.

[0102] Specifically, the specific calculation steps of the sound source localization model are as follows:

[0103] S601: intercepting a signal spectrum of equal length from a spectrum according to a signal bandwidth of the fault separation signal data to obtain effective spectrum data, and performing non-integer multiple downsampling on the effective spectrum data to obtain fixed-length spectrum data;

[0104] S602: normalizing the fixed-length spectrum data to obtain normalized spectrum data, and performing secondary smoothing on the normalized spectrum data to obtain smoothed spectrum data;

[0105] S603: Obtain regularized cost path data of the smoothed spectrum data between the main microphone and the auxiliary microphone in the first four-element cross microphone array and the second four-element cross microphone array by calculating through a regularized cost path algorithm, and determine the same fault signal according to the regularized cost path data, wherein the regularized cost path algorithm calculation formula is:

[0106]

[0107] Where DTW(.) is the regularized cost path data, S i is the i-th smoothed spectrum data of the main microphone, Q xy is the yth smoothed spectrum data of the xth auxiliary microphone, k is a variable parameter, K is the path length of the smoothed spectrum data, and d(pk) represents the distance between two points S i and Q xy The measured distance between

[0108] S604: Calculate the time difference data between the same fault signal arriving at the main microphone and each of the auxiliary microphones in the first four-element cross microphone array and the second four-element cross microphone array according to the cross-correlation function, and calculate the fault point coordinate data through a coordinate calculation model according to the time difference data and the distance data.

[0109] In this embodiment, the expression of the cross-correlation function is:

[0110] R X1X2 =E{(X1(t)X2(ta)},

[0111] Where R x1x2 is the cross-correlation function value, a is the delay difference, E{.} represents the expected value operation, when the cross-correlation function value is the maximum, the delay difference at this time is the time difference data;

[0112] The coordinate calculation model calculates the No. 1 sound source coordinate and the No. 2 sound source coordinate according to the coordinate calculation formula, calculates the average value of the No. 1 sound source coordinate and the No. 2 sound source coordinate, and uses the average value as the fault point coordinate data; the No. 1 sound source coordinate is the sound source coordinate calculated by the No. 1 four-element cross microphone array through the coordinate calculation model, and the No. 2 sound source coordinate is the sound source coordinate calculated by the No. 2 four-element cross microphone array through the coordinate calculation model; the coordinate calculation formula is:

[0113]

[0114] Among them, c is the sound speed, d is the distance data, θ is the pitch angle, φ is the azimuth, in the No. 1 four-element cross microphone array, X, Y, Z are the No. 1 sound source coordinates, a31, a21, a41 are the time difference between the left microphone and the microphone directly in front, the time difference between the right microphone and the microphone directly in front, and the time difference between the rear microphone and the microphone directly in front; in the No. 2 four-element cross microphone array, X, Y, Z are the No. 1 sound source coordinates, a31, a21, a41 are the time difference between the left microphone and the microphone directly above, the time difference between the right microphone and the microphone directly above, and the time difference between the lower microphone and the microphone directly above.

[0115] Specifically, the specific calculation steps of the recognition model are:

[0116] S701: performing thinning and smoothing processing on the component point cloud data to obtain pre-processed point cloud data;

[0117] S702: Obtain segmented point cloud data by using a progressive morphological filtering algorithm according to the pre-processed point cloud data;

[0118] S703: Calculating a covariance matrix according to the segmented point cloud data, performing principal component analysis on the covariance matrix to obtain a normal vector, and performing boundary separation on the segmented point cloud data according to the normal vector and a preset boundary determination angle threshold to obtain discrete boundary point cloud data;

[0119] S704: Obtain component outline data through a radius filtering algorithm according to the discrete boundary point cloud data, and compare the component outline data with the component outline data in a database to obtain the component recognition result.

[0120] A device fault diagnosis and positioning system based on a microphone array, comprising:

[0121] A coordinate establishment module, used to establish a spatial coordinate system, determine the spatial coordinates of each microphone in the microphone array according to the spatial coordinate system, and calculate the distance data between every two microphones according to the spatial coordinates;

[0122] The data acquisition module is used to acquire the sound signal data of the device through the microphone array, and transmit the sound signal data to the smart terminal, and the smart terminal calculates the preprocessed sound data according to the sound signal data through the preprocessing model;

[0123] The separation module module is used for the intelligent terminal to obtain separation signal data by using a sound source separation model according to the pre-processed sound data, and to obtain fault separation signal data by using a fault diagnosis model according to the separation signal data;

[0124] The positioning module is used to calculate the fault point coordinate data through the sound source positioning model according to the fault separation signal data and the distance data; obtain the component point cloud data and component space coordinates of the device through a three-dimensional laser scanner, calculate the component recognition result through the recognition model according to the component point cloud data, and store the component space coordinates in the database according to the component recognition result;

[0125] The interactive module is used for the intelligent terminal to retrieve the database according to the fault point coordinate data to obtain the name of the fault component, and transmit the name of the fault component to the human-computer interaction interface through the internal interface.

[0126] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the device fault diagnosis and positioning method based on a microphone array as described above is implemented.

[0127] A storage medium containing computer executable instructions, characterized in that the computer executable instructions are used to execute the device fault diagnosis and positioning method based on microphone array as described above when executed by a computer processor.

[0128] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0129] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0130] The program code included on the computer readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages-such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0131] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A device fault diagnosis and positioning method based on a microphone array, characterized in that: include: S1: establishing a spatial coordinate system, determining the spatial coordinates of each microphone in the microphone array according to the spatial coordinate system, and calculating the distance data between every two microphones according to the spatial coordinates; S2: Acquire sound signal data of the device through the microphone array, transmit the sound signal data to the smart terminal, and the smart terminal calculates preprocessed sound data according to the sound signal data through a preprocessing model; S3: the intelligent terminal calculates separation signal data according to the pre-processed sound data by using a sound source separation model, and calculates fault separation signal data according to the separation signal data by using a fault diagnosis model; S4: Calculate the coordinate data of the fault point through the sound source localization model according to the fault separation signal data and the distance data; obtain the component point cloud data and component space coordinates of the device through a three-dimensional laser scanner, calculate the component recognition result through the recognition model according to the component point cloud data, and store the component space coordinates in the database according to the component recognition result; S5: The intelligent terminal retrieves the database according to the fault point coordinate data to obtain the name of the fault component, and transmits the name of the fault component to the human-computer interaction interface through the internal interface.

2. The device fault diagnosis and positioning method based on microphone array according to claim 1 is characterized in that: The preprocessing model is used to remove reverberation and enhance the signal of the sound signal data, wherein the reverberation removal uses the signal received in the past time to predict the late reverberation at the tail of the measured signal, and then subtracts the late reverberation signal from the measured signal to achieve the reverberation removal function, and the specific steps are: S201: First, obtain time-frequency domain signal data through short-time Fourier transform according to the sound signal data. The time-frequency domain signal expression is: Among them, X m (n, k) represents the time-frequency domain signal of the mth microphone, n is the number of signal frames, k is the number of frequency points of the signal, L is the signal length, l is the variable parameter, S(nl, k) represents the sound source signal after being transformed into the time-frequency domain by short-time Fourier transform; r m (l, k) represents the room impulse response of the mth microphone in the array and the sound source; S202: Using the current delay to distinguish the early reverberation signal and the late reverberation signal of the time-frequency domain signal to obtain a distinguished time-frequency domain signal, wherein the expression of the distinguished time-frequency domain signal is: Among them, X m (n, k) represents the time-frequency domain signal of the mth microphone, n is the number of signal frames, k is the number of frequency points of the signal, L is the signal length, l is the variable parameter, S(nl, k) represents the sound source signal after being transformed into the time-frequency domain by short-time Fourier transform; r m (l, k) represents the room impulse response of the mth microphone in the array and the sound source, and a is the linear delay; S203: Finally, the sound signal data is subtracted from the late reverberation signal to obtain dereverberation sound data. The dereverberation sound data is expressed as: Where D is the dedeverberation sound data, X m (n, k) represents the time-frequency domain signal of the mth microphone, n is the number of signal frames, k is the number of frequency points of the signal, L is the signal length, l is the variable parameter, S(nl, k) represents the sound source signal after being transformed into the time-frequency domain by short-time Fourier transform; r m (l, k) represents the room impulse response of the mth microphone in the array and the sound source, and a is the linear delay.

3. The device fault diagnosis and positioning method based on microphone array according to claim 2 is characterized in that: The signal enhancement performs modal decomposition on the dereverberation sound data by an empirical mode decomposition method to obtain an intrinsic mode function component, calculates a correlation coefficient between the intrinsic mode function component and the dereverberation sound data by a correlation coefficient method, compares the correlation coefficient with a preset threshold to obtain a comparison result, and removes pseudo components with low correlation coefficients in the intrinsic mode data components according to the comparison result to obtain the preprocessed sound data, wherein the correlation coefficient calculation formula is: Where Q is the correlation coefficient, t is the variable parameter, T is the number of intrinsic mode functions, Y(t) is the dereverberation sound data, and h i (t) is the intrinsic mode function.

4. The device fault diagnosis and positioning method based on microphone array according to claim 1 is characterized in that: The sound source separation model decomposes the frequency spectrum of the preprocessed sound data into a non-negative matrix decomposition basis, and obtains a separation signal amplitude spectrum through non-negative matrix decomposition basis clustering according to the spatial characteristics of the non-negative matrix decomposition basis, and obtains the separation signal data according to the separation signal amplitude spectrum and the phase spectrum of the preprocessed sound data.

5. The device fault diagnosis and positioning method based on microphone array according to claim 1, characterized in that: The fault diagnosis model is implemented based on a neural network, and the specific steps are as follows: S501: performing frame division and windowing processing on the separated signal data to obtain processed signal data, and then performing Fourier transform on the processed signal data to obtain transformed data; S502: Acquire spectral line energy data of the transformed data, and obtain filtered energy data by calculation according to the spectral line energy data. The filter energy calculation formula is: Among them, S(i) is the filter energy, E(i, k) is the spectral line energy data, k is the variable parameter, H(k) is the filter expression, and N is the number of frames; S503: Calculate characteristic parameters according to the filter energy data and the discrete cosine transform cepstrum calculation formula, wherein the discrete cosine transform cepstrum calculation formula is: Wherein, Mf(i,n) is the characteristic parameter, i represents the i-th frame, n represents the spectrum line after DCT, m represents the m-th filter, S(i,m) is the filter energy in the m-th filter of the i-th frame, and M is the number of frames; S504: Calculate the differential characteristic parameters using a differential formula according to the characteristic parameters, and the differential formula is expressed as: Among them, d t is the tth first-order difference, C t is the tth characteristic parameter, C t+1 is the t+1th characteristic parameter, C t-1 is the t-1th characteristic parameter, C t+k is the t+kth characteristic parameter, C t-k is the tkth characteristic parameter, K is the first-order derivative time difference, k is the variable parameter, and Q is the order of the characteristic parameter; S505: Input the differential feature parameters into the neural network. After receiving the differential feature parameters, the input layer of the neural network passes the differential feature parameters to the hidden layer. The hidden layer calculates the hidden layer output result according to the differential feature parameters, connection weights, thresholds and activation functions, and uses the hidden layer output result as the input of the output layer. The fault separation signal data is calculated according to the hidden layer output result, the connection weights between the output layers and the output layer threshold.

6. The device fault diagnosis and location method based on microphone array according to claim 1, characterized in that: The specific steps of calculating the sound source localization model are: S601: intercepting a signal spectrum of equal length from a spectrum according to a signal bandwidth of the fault separation signal data to obtain effective spectrum data, and performing non-integer multiple downsampling on the effective spectrum data to obtain fixed-length spectrum data; S602: normalizing the fixed-length spectrum data to obtain normalized spectrum data, and performing secondary smoothing on the normalized spectrum data to obtain smoothed spectrum data; S603: Obtain regularized cost path data of the smoothed spectrum data between the main microphone and the auxiliary microphone by calculating through a regularized cost path algorithm, and determine the same fault signal according to the regularized cost path data, wherein the regularized cost path algorithm calculation formula is: Where DTW(.) is the regularized cost path data, S i is the i-th smoothed spectrum data of the main microphone, Q xy is the yth smoothed spectrum data of the xth auxiliary microphone, k is a variable parameter, K is the path length of the smoothed spectrum data, and d(pk) represents the distance between two points S i and Q xy The measured distance between S604: Calculate the time difference data between the same fault signal reaching the main microphone and each of the auxiliary microphones according to the cross-correlation function, and calculate the fault point coordinate data through a coordinate calculation model according to the time difference data and the distance data.

7. The device fault diagnosis and location method based on microphone array according to claim 1, characterized in that: The specific calculation steps of the recognition model are: S701: performing thinning and smoothing processing on the component point cloud data to obtain pre-processed point cloud data; S702: Obtain segmented point cloud data by using a progressive morphological filtering algorithm according to the pre-processed point cloud data; S703: Calculating a covariance matrix according to the segmented point cloud data, performing principal component analysis on the covariance matrix to obtain a normal vector, and performing boundary separation on the segmented point cloud data according to the normal vector and a preset boundary determination angle threshold to obtain discrete boundary point cloud data; S704: Obtain component outline data through a radius filtering algorithm according to the discrete boundary point cloud data, and compare the component outline data with the component outline data in a database to obtain the component recognition result.

8. A device fault diagnosis and positioning system based on a microphone array, characterized in that: include: A coordinate establishment module, used to establish a spatial coordinate system, determine the spatial coordinates of each microphone in the microphone array according to the spatial coordinate system, and calculate the distance data between every two microphones according to the spatial coordinates; The data acquisition module is used to acquire the sound signal data of the device through the microphone array, and transmit the sound signal data to the smart terminal, and the smart terminal calculates the preprocessed sound data according to the sound signal data through the preprocessing model; The separation module module is used for the intelligent terminal to obtain separation signal data by using a sound source separation model according to the pre-processed sound data, and to obtain fault separation signal data by using a fault diagnosis model according to the separation signal data; The positioning module is used to calculate the fault point coordinate data through the sound source positioning model according to the fault separation signal data and the distance data; Acquire component point cloud data and component space coordinates of the device through a three-dimensional laser scanner, calculate component recognition results based on the component point cloud data through a recognition model, and store the component space coordinates in a database based on the component recognition results; The interactive module is used for the intelligent terminal to retrieve the database according to the fault point coordinate data to obtain the name of the fault component, and transmit the name of the fault component to the human-computer interaction interface through the internal interface.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the device fault diagnosis and positioning method based on the microphone array as described in any one of claims 1-7 is implemented.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the device fault diagnosis and positioning method based on microphone array as described in any one of claims 1-7 when executed by a computer processor.

Citation Information

Patent Citations

  • Improved sound source localization method based on progressive serial orthogonalization blind source separation algorithm, and implementation system for same

    CN107644650A

  • Conveying belt tearing detection system based on three-dimensional laser scanning technology and detection method thereof

    CN112213317A

  • Multi-channel non-negative matrix factorization method and system based on frequency domain convolution transfer function

    CN114220453A

  • Audio detection and positioning method and system based on deep learning, and intelligent terminal

    CN114265009A

  • Method and device for positioning abnormal fault sound of power transmission channel

    CN114280413A

Cited By

  • Dynamic gain equalization method, device, equipment and product for sound equipment

    CN119996894A

  • Space anomaly acoustic diagnosis method and system based on space-time frequency feature fusion

    CN121009465A

  • Motor abnormal sound intelligent identification method based on time-frequency characteristic and hearing adaptation

    CN121234309A

  • Motor abnormal sound intelligent identification method based on time-frequency characteristics and auditory adaptation

    CN121234309B