Sound anomaly detection method and device based on self-supervised learning, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a sound anomaly detection method and device based on self-supervised learning, equipment and a medium, and the method comprises the steps: obtaining a first sound sample set of mechanical equipment in a normal operation state and a sound sample of to-be-tested mechanical equipment; performing data enhancement processing on the first sound sample set to obtain a second sound sample set; inputting the second sound sample set into the initial encoder neural network model to extract a feature vector; constructing a loss function and updating the encoder neural network model; inputting a sound sample of to-be-tested mechanical equipment into the trained encoder neural network model to extract a feature vector; and calculating the mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set, and judging whether the operation state of the mechanical equipment is normal or not according to a calculation result. According to the method, the fault detection accuracy of the industrial mechanical equipment is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a sound anomaly detection method, device, equipment and medium based on self-supervised learning. Background Art

[0002] With the rapid development of machine learning, more and more industrial production lines use intelligent anomaly detection algorithms to monitor the normal operation of industrial machinery and equipment and detect the failure of industrial machinery and equipment. At present, many intelligent anomaly detection algorithms are implemented based on computer vision monitoring technology. However, in some complex environmental scenarios, such as insufficient light, obstructions, and heavy dust, computer vision monitoring may be interfered and unable to directly detect the failure of industrial machinery and equipment. It is necessary to use sound anomaly detection algorithms to detect the failure of mechanical equipment by analyzing the sound emitted by the mechanical equipment during operation to determine whether the equipment has a fault.

[0003] However, in actual situations, it is difficult to obtain sound sample data of mechanical equipment in abnormal operating conditions. The number of samples in fault conditions is far less than the number of samples in normal conditions. The sound anomaly detection algorithm requires a sufficient number of samples in abnormal operating conditions to learn the characteristic information of fault sounds. If the number of samples in abnormal operating conditions of mechanical equipment is small, the sound anomaly detection algorithm may not be able to accurately learn these features, resulting in inaccurate fault detection of industrial mechanical equipment. Summary of the invention

[0004] The present invention provides a sound anomaly detection method, device, equipment and medium based on self-supervised learning to solve the technical problem that when the number of samples of the existing mechanical equipment in the abnormal operation state is small, the fault detection of industrial mechanical equipment is inaccurate.

[0005] In a first aspect, a sound anomaly detection method based on self-supervised learning is provided, which is applied to fault detection of industrial machinery equipment, including:

[0006] Acquire a first sound sample set of the mechanical device in a normal operating state, and a sound sample of the mechanical device to be tested;

[0007] Performing data enhancement processing on the first sound sample set acquired under normal operating conditions to obtain a second sound sample set under normal operating conditions;

[0008] Inputting the second sound sample set in the normal operating state into the initial encoder neural network model, and extracting the feature vector of the second sound sample set in the normal operating state;

[0009] Constructing a loss function of the encoder neural network model, and updating the encoder neural network model during training based on the constructed loss function to obtain a trained encoder neural network model;

[0010] Inputting the sound sample of the mechanical equipment to be tested into the trained encoder neural network model to extract the feature vector of the sound sample of the mechanical equipment to be tested;

[0011] The Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions is calculated, and whether the operating state of the mechanical equipment is normal is determined according to the calculation result.

[0012] In a second aspect, a sound anomaly detection device based on self-supervised learning is provided, wherein the device is used to implement the sound anomaly detection method based on self-supervised learning as described in any one of the first aspects above, including:

[0013] An acquisition module, used to acquire a first sound sample set of the mechanical device in a normal operating state, and a sound sample of the mechanical device to be tested;

[0014] A processing module, configured to perform data enhancement processing on the first sound sample set acquired under normal operating conditions to obtain a second sound sample set under normal operating conditions;

[0015] A first extraction module, used for inputting the second sound sample set in the normal operating state into the initial encoder neural network model, and extracting the feature vector of the second sound sample set in the normal operating state;

[0016] A construction module is used to construct a loss function of the encoder neural network model, and update the encoder neural network model during training based on the constructed loss function to obtain a trained encoder neural network model;

[0017] A second extraction module, used for inputting the sound sample of the mechanical equipment to be tested into the trained encoder neural network model to extract the feature vector of the sound sample of the mechanical equipment to be tested;

[0018] The calculation module is used to calculate the Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions, and determine whether the operating state of the mechanical equipment is normal according to the calculation result.

[0019] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned sound anomaly detection method based on self-supervised learning are implemented.

[0020] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned sound anomaly detection method based on self-supervised learning are implemented.

[0021] In the scheme implemented by the above-mentioned sound anomaly detection method, device, equipment and medium based on self-supervised learning, the client can obtain a first sound sample set of the mechanical equipment in a normal operating state, and a sound sample of the mechanical equipment to be tested; perform data enhancement processing on the first sound sample set obtained in a normal operating state to obtain a second sound sample set in a normal operating state; input the second sound sample set in the normal operating state into the initial encoder neural network model, and extract the feature vector of the second sound sample set in the normal operating state; construct a loss function of the encoder neural network model, and update the encoder neural network model during training based on the constructed loss function to obtain a trained encoder neural network model; and input the second sound sample set in the normal operating state into the initial encoder neural network model, and extract the feature vector of the second sound sample set in the normal operating state. The sound samples of the mechanical equipment are input into the trained encoder neural network model, and the feature vector of the sound samples of the mechanical equipment to be tested is extracted; the Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions is calculated, and whether the operating state of the mechanical equipment is normal is judged according to the calculation result. In the present invention, the faults of industrial mechanical equipment are detected by performing data enhancement processing on the first sound sample set obtained under normal operating conditions, and by calculating the Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions, thereby effectively improving the accuracy of industrial mechanical equipment fault detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0023] Figure 1 is a schematic diagram of an application environment of a sound anomaly detection method based on self-supervised learning in one embodiment of the present invention;

[0024] Figure 2 is a flow chart of a method for detecting sound anomalies based on self-supervised learning in one embodiment of the present invention;

[0025] Figure 3 yes Figure 2 A schematic flow chart of a specific implementation of step S20;

[0026] Figure 4 yes Figure 2 A schematic flow chart of a specific implementation of step S60;

[0027] Figure 5 is a structural diagram of a sound anomaly detection device based on self-supervised learning in one embodiment of the present invention;

[0028] Figure 6 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0029] Figure 7 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] The sound anomaly detection method based on self-supervised learning provided by the embodiment of the present invention can be applied in the following aspects: Figure 1In an application environment, a client communicates with a server through a network. The server can receive a first sound sample set of a mechanical device in a normal operating state and a sound sample of a mechanical device to be tested through a client; perform data enhancement processing on the first sound sample set obtained in the normal operating state to obtain a second sound sample set in the normal operating state; input the second sound sample set in the normal operating state into the initial encoder neural network model, and extract the feature vector of the second sound sample set in the normal operating state; construct a loss function of the encoder neural network model, and update the encoder neural network model during training based on the constructed loss function to obtain a trained encoder neural network model; input the sound sample of the mechanical device to be tested into the trained encoder neural network model, and extract the feature vector of the sound sample of the mechanical device to be tested; calculate the Mahalanobis distance between the feature vector of the sound sample of the mechanical device to be tested and the feature vector of the second sound sample set in the normal operating state, and judge whether the operating state of the mechanical device is normal according to the calculation result. In the present invention, by performing data enhancement processing on the first sound sample set obtained in the normal operating state, and by calculating the Mahalanobis distance between the feature vector of the sound sample of the mechanical device to be tested and the feature vector of the second sound sample set in the normal operating state, the fault of the industrial mechanical device is detected, and the accuracy of the fault detection of the industrial mechanical device is effectively improved. The present invention is described in detail below through specific embodiments.

[0032] See also Figure 2 As shown, Figure 2 A flow chart of a method for detecting anomalies of sound based on self-supervised learning provided by an embodiment of the present invention includes the following steps:

[0033] S10: Acquire a first sound sample set of the mechanical device in a normal operating state, and a sound sample of the mechanical device to be tested.

[0034] Since the sound characteristics of mechanical equipment in normal operation are very similar, and the sound in fault state is ever-changing, the training set of the present invention only uses the sound samples of mechanical equipment in normal operation, and collects N sound samples of mechanical equipment in normal operation as the training set, as shown in formula (1):

[0035]

[0036] By collecting a large number of sounds during normal operation, the present invention can understand the sound feature range of mechanical equipment during normal operation, and provide a reference standard for subsequent judgment of abnormal sounds. The present invention obtains the first sound sample set of mechanical equipment in normal operation, and can collect the sound of normal operation of mechanical equipment under different working environments, different operating time periods and different load conditions. For example, the first sound sample set can be collected when the equipment is just started, when it is running stably, and when it is close to full load operation, to ensure the diversity of samples.

[0037] S20: Performing data enhancement processing on the first sound sample set acquired under the normal operating state to obtain a second sound sample set under the normal operating state.

[0038] After the first sound sample set of the mechanical equipment in normal operation is obtained in the above step S10, data enhancement processing can be performed on the first sound sample set in normal operation. In mechanical equipment fault detection, the sound samples in normal operation are often large in number, but there may be a problem of insufficient diversity. Data enhancement processing can effectively increase the number and diversity of samples, and improve the generalization ability and robustness of the model. When there are many samples in normal operation and few samples in abnormal operation, data enhancement processing of the sound sample set in normal operation can enable the model to better learn various changes in the normal state during training, thereby more accurately identifying the sound samples in the normal operation state. Among them, if Figure 3 As shown, Figure 3 yes Figure 2 FIG. 1 is a flow chart of a specific implementation of step S20 in FIG. 1 ; in step S20, data enhancement processing is performed on the first sound sample set obtained under normal operating conditions to obtain a second sound sample set under normal operating conditions, including the following steps:

[0039] S21: Perform tone conversion processing on the first sound sample set obtained in the normal operating state to obtain a tone-converted sound sample set. Specifically, the present invention can adjust the tone of the audio by changing the frequency characteristics of the sound, and can increase or decrease the tone of the audio. For example, for a sound sample of a mechanical device in normal operation, its tone can be increased by 10%, 20%, 30%, etc. to obtain a tone-converted sound sample set.

[0040] S22: Performing audio speed processing on the first sound sample set obtained in the normal operating state to obtain a sound sample set after audio speed processing. Specifically, in the present invention, for the first sound sample set, different degrees of fast-play and slow-play operations can be performed respectively. For example, the playback speed of the sound sample can be accelerated by 10%, 50%, etc., or slowed down by 20%, 70%, etc., to generate audio samples of different speeds to obtain a sound sample set after audio speed processing.

[0041] S23: inject noise into the first sound sample set obtained in the normal operating state to obtain a sound sample set after the noise is injected. Specifically, in the present invention, noises of different types and intensities can be injected into the sound samples in the first sound sample set. Common noise types such as white noise and pink noise can be selected to generate sound sample sets with different noises to obtain a sound sample set after the noise is injected.

[0042] S24: Fusing the first sound sample set under normal operation, the sound sample set after tone conversion processing, the sound sample set after audio speed processing, and the sound sample set after noise injection to obtain a second sound sample set under normal operation. Specifically, in the present invention, the first sound sample set under normal operation can be fused with the sample set after data enhancement processing to obtain a richer and more diverse sound sample set, which can provide more training data for the encoder neural network model and improve the generalization ability and robustness of the model.

[0043] S30: Inputting the second sound sample set under normal operating conditions into the initial encoder neural network model, and extracting a feature vector of the second sound sample set under normal operating conditions.

[0044] In order to explore the intrinsic similarity and invariance of normal samples, the present invention adopts a feature extraction method of self-supervised learning to train an encoder neural network model. First, an initial encoder neural network model is used to extract a feature vector of a second sound sample set under normal operating conditions, wherein the encoder neural network model can receive the second sound sample set under normal operating conditions as input, and then undergo processing by multiple hidden layers, each of which performs different degrees of abstraction and feature extraction on the input. For example, in the first few layers, some simple audio features may be extracted, such as frequency components, amplitude changes, etc.; as the number of hidden layers increases, the encoder neural network model can learn more advanced semantic features, such as sound patterns and rhythmic regularities when a specific device is operating normally.

[0045] The feature vector of the second sound sample set extracted under normal operating conditions is shown in formula (2):

[0046] Z i =Encoder(Xi ) (2)

[0047] Among them, Z i is the second sound sample set X extracted under normal operating conditions i The eigenvector of the eigenvector of .

[0048] S40: construct a loss function of the encoder neural network model, and update the encoder neural network model during training based on the constructed loss function to obtain a trained encoder neural network model.

[0049] After extracting the feature vector of the second sound sample set under normal operation in the above step S30, a loss function of the encoder neural network model is constructed, and the encoder neural network model during training is updated based on the constructed loss function. The calculation formula of the loss function Loss of the encoder neural network model is shown in formula (3):

[0050]

[0051] Among them, sim(.) represents the cosine similarity function, Z i and Z j Represents the negative sample pairs of different sounds of mechanical equipment in normal operation, Z i and represents the positive sample pair of the same sound of the mechanical equipment in normal operation, and ρ represents the adjustment parameter when training the encoder neural network model.

[0052] The present invention uses positive and negative sample pairs and a cosine similarity function to calculate the loss function, so that the encoder neural network model can better mine the intrinsic characteristics of the sound sample in the self-supervised learning process and improve the ability to distinguish between normal samples and abnormal samples.

[0053] S50: Inputting the sound sample of the mechanical device to be tested into the trained encoder neural network model, and extracting the feature vector of the sound sample of the mechanical device to be tested.

[0054] After the trained encoder neural network model is obtained in step S40, the sound sample of the mechanical device to be tested can be input into the trained encoder neural network model to extract the feature vector of the sound sample of the mechanical device to be tested. The feature vector of the sound sample of the mechanical device to be tested extracted is shown in formula (5):

[0055] Z test =Encoder(X test ) (5)

[0056] S60: Calculate the Mahalanobis distance between the feature vector of the sound sample of the mechanical device to be tested and the feature vector of the second sound sample set under normal operating conditions, and determine whether the operating state of the mechanical device is normal based on the calculation result. Figure 4 As shown, Figure 4 yes Figure 2 FIG. 1 is a flow chart of a specific implementation of step S60 in FIG. 1 ; step S60, that is, calculating the Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions, comprises the following steps:

[0057] S61: Calculate the feature vector of the sound sample of the mechanical equipment to be tested, and calculate the mean and covariance of the feature vector of the second sound sample set under normal operating conditions.

[0058] In mechanical equipment fault detection, the mean value can be used as a benchmark reference value for the sound characteristics under normal operating conditions. When the feature vector of the sound sample of the mechanical equipment to be tested differs greatly from the mean value, the mechanical equipment may fail.

[0059] In step S61, the calculation formula of the mean value u of the feature vector of the second sound sample set in the normal operating state is:

[0060]

[0061] Among them, Z i represents the calculated feature vector of the second sound sample set under the normal operating state, and N represents the number of samples of the second sound sample set under the normal operating state.

[0062] In mechanical equipment fault detection, the covariance matrix can help us understand the degree of correlation between different sound features. In step S61, the calculation formula of the covariance ∑ of the feature vector of the second sound sample set under normal operating conditions is:

[0063]

[0064] Wherein, μ represents the mean value of the feature vector of the second sound sample set under normal operating conditions.

[0065] S62: Calculate the Mahalanobis distance between the feature vector of the sound sample of the mechanical device to be tested and the feature vector of the second sound sample set under normal operating conditions according to the calculated mean and covariance of the feature vector of the sound sample of the mechanical device to be tested and the feature vector of the second sound sample set under normal operating conditions.

[0066] It should be noted that the Mahalanobis distance is an effective method for measuring the difference between two random variables, which takes into account the correlation between the variables. In the present invention, by calculating the Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions, it can be determined whether the sound sample of the mechanical equipment to be tested is normal.

[0067] In step S62, the Mahalanobis distance S between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operation is calculated. test The calculation formula is:

[0068] S test =(Z test -μ) T ∑ -1 (Z test -u) (8)

[0069] Among them, Z test represents the feature vector of the sound sample to be tested, u represents the mean of the feature vector of the second sound sample set under normal operating conditions, and ∑ represents the covariance of the feature vector of the second sound sample set under normal operating conditions.

[0070] The method of judging whether the operating state of the mechanical equipment is normal according to the calculation result includes: presetting an abnormal distance threshold between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under the normal operating state; if the Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under the normal operating state is greater than the preset abnormal distance threshold, it means that the sound sample of the mechanical equipment to be tested is far from the second sound sample set under the normal operating state, and the operating state of the mechanical equipment is abnormal; otherwise, the operating state of the mechanical equipment is normal. By presetting the abnormal distance threshold, the present invention can promptly issue an alarm when a mechanical equipment fails, reminding the staff to perform inspection and maintenance, avoiding further deterioration of the failure, thereby reducing the risk of equipment damage and production interruption.

[0071] It can be seen that in the above scheme, by performing data enhancement processing on the first sound sample set obtained under normal operating conditions, and by calculating the Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions, the fault of the industrial mechanical equipment is detected, thereby effectively improving the accuracy of industrial mechanical equipment fault detection.

[0072] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0073] In one embodiment, a sound anomaly detection device based on self-supervised learning is provided, and the sound anomaly detection device based on self-supervised learning corresponds one-to-one to the sound anomaly detection method based on self-supervised learning in the above embodiment. Figure 5 As shown, Figure 5 1 is a schematic diagram of a structure of a sound anomaly detection device based on self-supervised learning in one embodiment of the present invention, and the sound anomaly detection device based on self-supervised learning includes an acquisition module 51, a processing module 52, a first extraction module 53, a construction module 54, a second extraction module 55, and a calculation module 56. The functional modules are described in detail as follows:

[0074] An acquisition module, used to acquire a first sound sample set of the mechanical device in a normal operating state, and a sound sample of the mechanical device to be tested;

[0075] A processing module, configured to perform data enhancement processing on the first sound sample set acquired under normal operating conditions to obtain a second sound sample set under normal operating conditions;

[0076] A first extraction module, used for inputting the second sound sample set in the normal operating state into the initial encoder neural network model, and extracting the feature vector of the second sound sample set in the normal operating state;

[0077] A construction module is used to construct a loss function of the encoder neural network model, and update the encoder neural network model during training based on the constructed loss function to obtain a trained encoder neural network model;

[0078] A second extraction module, used for inputting the sound sample of the mechanical equipment to be tested into the trained encoder neural network model to extract the feature vector of the sound sample of the mechanical equipment to be tested;

[0079] The calculation module is used to calculate the Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions, and determine whether the operating state of the mechanical equipment is normal according to the calculation result.

[0080] In one embodiment, the processing module 52 is specifically configured to:

[0081] Performing tone conversion processing on the first sound sample set acquired in the normal operating state to obtain a sound sample set after tone conversion processing;

[0082] Performing audio speed processing on the first sound sample set acquired in the normal operating state to obtain a sound sample set after the audio speed processing;

[0083] Injecting noise into the first sound sample set acquired in the normal operating state to obtain a sound sample set after the noise is injected;

[0084] The first sound sample set under normal operating conditions, the sound sample set after tone conversion processing, the sound sample set after audio speed processing, and the sound sample set after noise injection are fused to obtain a second sound sample set under normal operating conditions.

[0085] In one embodiment, the construction module 54 is specifically configured to:

[0086] The calculation formula of the loss function Loss of the encoder neural network model is:

[0087]

[0088] Among them, sim(.) represents the cosine similarity function, Z i and Z j Represents the negative sample pairs of different sounds of mechanical equipment in normal operation, Z i and represents the positive sample pair of the same sound of the mechanical equipment in normal operation, and ρ represents the adjustment parameter when training the encoder neural network model.

[0089] In one embodiment, the calculation module 56 is specifically configured to:

[0090] Calculating a feature vector of the sound sample of the mechanical equipment to be tested, and calculating a mean and a covariance of the feature vector of a second sound sample set under a normal operating state;

[0091] The Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions is calculated based on the calculated mean and covariance of the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions.

[0092] In one embodiment, the calculation module 56 is specifically configured to:

[0093] The calculation formula for the mean value u of the feature vector of the second sound sample set under normal operating conditions is:

[0094]

[0095] Among them, Z irepresents a feature vector of the second sound sample set under the normal operating state calculated, and N represents the number of samples of the second sound sample set under the normal operating state;

[0096] The calculation formula of the covariance ∑ of the feature vector of the second sound sample set under normal operating conditions is:

[0097]

[0098] Wherein, μ represents the mean value of the feature vector of the second sound sample set under normal operating conditions.

[0099] In one embodiment, the calculation module 56 is specifically configured to:

[0100] The Mahalanobis distance S between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions test The calculation formula is:

[0101] S test =(Z test -μ) T ∑ -1 (Z test -u)

[0102] Among them, Z test represents the feature vector of the sound sample to be tested, u represents the mean of the feature vector of the second sound sample set under normal operating conditions, and ∑ represents the covariance of the feature vector of the second sound sample set under normal operating conditions.

[0103] In one embodiment, the calculation module 56 is specifically configured to:

[0104] Presetting an abnormal distance threshold between a feature vector of a sound sample of the mechanical device to be tested and a feature vector of a second sound sample set under a normal operating state;

[0105] If the Mahalanobis distance between the feature vector of the sound sample of the mechanical device to be tested and the feature vector of the second sound sample set under normal operating conditions is greater than a preset abnormal distance threshold, the operating state of the mechanical device is abnormal;

[0106] Otherwise, the operating status of the mechanical equipment is normal.

[0107] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, Figure 61 is a schematic diagram of the structure of a computer device in one embodiment of the present invention. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, a function or step on the sound anomaly detection service side based on self-supervised learning is implemented.

[0108] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 7 As shown, Figure 7 : is another structural diagram of a computer device in one embodiment of the present invention. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements a function or step of a sound anomaly detection client side based on self-supervised learning

[0109] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0110] Acquire a first sound sample set of the mechanical device in a normal operating state, and a sound sample of the mechanical device to be tested;

[0111] Performing data enhancement processing on the first sound sample set acquired under normal operating conditions to obtain a second sound sample set under normal operating conditions;

[0112] Inputting the second sound sample set in the normal operating state into the initial encoder neural network model, and extracting the feature vector of the second sound sample set in the normal operating state;

[0113] Constructing a loss function of the encoder neural network model, and updating the encoder neural network model during training based on the constructed loss function to obtain a trained encoder neural network model;

[0114] Inputting the sound sample of the mechanical equipment to be tested into the trained encoder neural network model to extract the feature vector of the sound sample of the mechanical equipment to be tested;

[0115] The Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions is calculated, and whether the operating state of the mechanical equipment is normal is determined according to the calculation result.

[0116] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0117] Acquire a first sound sample set of the mechanical device in a normal operating state, and a sound sample of the mechanical device to be tested;

[0118] Performing data enhancement processing on the first sound sample set acquired under normal operating conditions to obtain a second sound sample set under normal operating conditions;

[0119] Inputting the second sound sample set in the normal operating state into the initial encoder neural network model, and extracting the feature vector of the second sound sample set in the normal operating state;

[0120] Constructing a loss function of the encoder neural network model, and updating the encoder neural network model during training based on the constructed loss function to obtain a trained encoder neural network model;

[0121] Inputting the sound sample of the mechanical equipment to be tested into the trained encoder neural network model to extract the feature vector of the sound sample of the mechanical equipment to be tested;

[0122] The Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions is calculated, and whether the operating state of the mechanical equipment is normal is determined according to the calculation result.

[0123] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0124] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0125] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0126] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A sound anomaly detection method based on self-supervised learning, applied to fault detection of industrial machinery and equipment, characterized in that: include: Acquire a first sound sample set of the mechanical device in a normal operating state, and a sound sample of the mechanical device to be tested; Performing data enhancement processing on the first sound sample set acquired under normal operating conditions to obtain a second sound sample set under normal operating conditions; Inputting the second sound sample set in the normal operating state into the initial encoder neural network model, and extracting the feature vector of the second sound sample set in the normal operating state; Constructing a loss function of the encoder neural network model, and updating the encoder neural network model during training based on the constructed loss function to obtain a trained encoder neural network model; Inputting the sound sample of the mechanical equipment to be tested into the trained encoder neural network model to extract the feature vector of the sound sample of the mechanical equipment to be tested; The Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions is calculated, and whether the operating state of the mechanical equipment is normal is determined according to the calculation result.

2. The method for detecting anomalies of sound based on self-supervised learning according to claim 1, characterized in that: The step of performing data enhancement processing on the first sound sample set acquired under normal operating conditions to obtain a second sound sample set under normal operating conditions includes: Performing tone conversion processing on the first sound sample set acquired in the normal operating state to obtain a sound sample set after tone conversion processing; Performing audio speed processing on the first sound sample set acquired in the normal operating state to obtain a sound sample set after the audio speed processing; Injecting noise into the first sound sample set acquired in the normal operating state to obtain a sound sample set after the noise is injected; The first sound sample set under normal operating conditions, the sound sample set after tone conversion processing, the sound sample set after audio speed processing, and the sound sample set after noise injection are fused to obtain a second sound sample set under normal operating conditions.

3. The method for detecting anomalies of sound based on self-supervised learning according to claim 2, characterized in that: The loss function of constructing the encoder neural network model includes: The calculation formula of the loss function Loss of the encoder neural network model is: Among them, sim(.) represents the cosine similarity function, Z i and Z j Represents the negative sample pairs of different sounds of mechanical equipment in normal operation, Z i and represents the positive sample pair of the same sound of the mechanical equipment in normal operation, and ρ represents the adjustment parameter when training the encoder neural network model.

4. The method for detecting anomalies of sound based on self-supervised learning according to claim 3, characterized in that: The calculating of the Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set in a normal operating state comprises: Calculating a feature vector of the sound sample of the mechanical equipment to be tested, and calculating a mean and a covariance of the feature vector of a second sound sample set under a normal operating state; The Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions is calculated based on the calculated mean and covariance of the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions.

5. The method for detecting anomalies of sound based on self-supervised learning according to claim 4, characterized in that: Calculating the mean and covariance of the feature vector of the second sound sample set under normal operating conditions includes: The calculation formula for the mean value u of the feature vector of the second sound sample set under normal operating conditions is: Among them, Z i represents a feature vector of the second sound sample set under the normal operating state calculated, and N represents the number of samples of the second sound sample set under the normal operating state; The calculation formula of the covariance ∑ of the feature vector of the second sound sample set under normal operating conditions is: Wherein, μ represents the mean value of the feature vector of the second sound sample set under normal operating conditions.

6. The method for detecting anomalies of sound based on self-supervised learning according to claim 5, characterized in that: The step of calculating the Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operation includes: The Mahalanobis distance S between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions test The calculation formula is: S test =(From test -μ) T ∑ -1 (WITH test -at) Among them, Z test represents the feature vector of the sound sample to be tested, u represents the mean of the feature vector of the second sound sample set under normal operating conditions, and ∑ represents the covariance of the feature vector of the second sound sample set under normal operating conditions.

7. The method for detecting sound anomalies based on self-supervised learning according to claim 6, characterized in that: The step of judging whether the operating state of the mechanical equipment is normal according to the calculation result includes: Presetting an abnormal distance threshold between a feature vector of a sound sample of the mechanical device to be tested and a feature vector of a second sound sample set under a normal operating state; If the Mahalanobis distance between the feature vector of the sound sample of the mechanical device to be tested and the feature vector of the second sound sample set under normal operating conditions is greater than a preset abnormal distance threshold, the operating state of the mechanical device is abnormal; Otherwise, the operating status of the mechanical equipment is normal.

8. A sound anomaly detection device based on self-supervised learning, the device being used to implement the sound anomaly detection method based on self-supervised learning according to any one of claims 1 to 7, characterized in that: include: An acquisition module, used to acquire a first sound sample set of the mechanical device in a normal operating state, and a sound sample of the mechanical device to be tested; A processing module, configured to perform data enhancement processing on the first sound sample set acquired under normal operating conditions to obtain a second sound sample set under normal operating conditions; A first extraction module, used for inputting the second sound sample set in the normal operating state into the initial encoder neural network model, and extracting the feature vector of the second sound sample set in the normal operating state; A construction module is used to construct a loss function of the encoder neural network model, and update the encoder neural network model during training based on the constructed loss function to obtain a trained encoder neural network model; A second extraction module, used for inputting the sound sample of the mechanical equipment to be tested into the trained encoder neural network model to extract the feature vector of the sound sample of the mechanical equipment to be tested; The calculation module is used to calculate the Mahalanobis distance between the feature vector of the sound sample of the mechanical equipment to be tested and the feature vector of the second sound sample set under normal operating conditions, and determine whether the operating state of the mechanical equipment is normal according to the calculation result.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the sound anomaly detection method based on self-supervised learning are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the sound anomaly detection method based on self-supervised learning are implemented as described in any one of claims 1 to 7.

Images