Sound anomaly detection method and device based on self-supervised learning, equipment and medium

Abstract

The invention relates to the field of artificial intelligence, and discloses a sound anomaly detection method and device based on self-supervised learning, electronic equipment and a storage medium, and the method comprises the steps: carrying out the voice enhancement of a positive sample sound data set, and obtaining an enhanced sample sound data set; training a preset feature extraction model by using the positive sample sound data set and the enhanced sample sound data set to obtain a trained feature extraction model; extracting features of the positive sample sound data set and the to-be-detected sound data by using the trained feature extraction model to obtain a standard sound wave feature data set and to-be-detected sound wave feature data; calculating a mean value and a covariance of the sound wave characteristic data set to obtain a standard sound wave mean value and a standard sound wave covariance; according to the standard sound wave mean value and the standard sound wave covariance, the distance between the to-be-detected sound wave feature data and the sound wave feature data set is calculated, and when the distance is larger than a preset threshold value, it is judged that the to-be-detected sound data is abnormal data. According to the invention, the accuracy of sound anomaly detection can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a sound anomaly detection method, device, electronic device and readable storage medium based on self-supervised learning. Background Art

[0002] Sound anomaly detection refers to the process of detecting sound data to be tested based on the gap between sound data and sound data. For example, in the field of medical devices, the sound of the pacemaker to be tested when it is working can be compared with the sound of a normal pacemaker when it is working to determine whether the pacemaker to be tested has any abnormalities.

[0003] Currently, the most common sound anomaly detection methods are to collect abnormal sound data and use the abnormal sound data to train specific models to achieve the purpose of sound anomaly detection. However, the existing methods also have some problems. For example, there are many normal samples and few abnormal samples, and supervised learning cannot be used directly, resulting in poor model training results and low accuracy of sound anomaly detection. Summary of the invention

[0004] The present invention provides a sound anomaly detection method, device, electronic device and readable storage medium based on self-supervised learning, which aims to improve the accuracy of sound anomaly detection.

[0005] To achieve the above object, the present invention provides a method for detecting sound anomalies based on self-supervised learning, the method comprising:

[0006] Acquire a positive sample sound data set and sound data to be tested, and perform speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set;

[0007] Using the positive sample sound data set and the enhanced sample sound data set to train a preset feature extraction model to obtain a trained feature extraction model;

[0008] The trained feature extraction model is used to extract the features of the positive sample sound data set and the sound data to be tested, respectively, to obtain a standard sound wave feature data set and sound wave feature data to be tested;

[0009] Calculating the mean and covariance of the acoustic wave feature data set respectively to obtain a standard acoustic wave mean and a standard acoustic wave covariance;

[0010] The distance between the sound wave feature data to be measured and the sound wave feature data set is calculated according to the standard sound wave mean and the standard sound wave covariance, and when the distance is greater than a preset threshold, the sound data to be measured is determined to be abnormal data.

[0011] Optionally, the using the positive sample sound data set and the enhanced sample sound data set to train a preset feature extraction model to obtain a trained feature extraction model includes:

[0012] Using the encoding layer in the preset feature extraction model, respectively encode the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set to obtain a positive sample sound encoding vector and an enhanced sample sound encoding vector;

[0013] Using the convolution layer in the preset feature extraction model to perform convolution operations on the positive sample sound coding vector and the enhanced sample sound coding vector respectively, to obtain a positive sample sound feature vector and an enhanced sample sound feature vector;

[0014] According to the positive sample sound feature vector and the enhanced sample sound feature vector, a preset loss function is used to calculate the loss value between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set, and the parameters of the preset feature extraction model are iteratively adjusted according to the loss value until the loss value meets the preset conditions, thereby obtaining a trained feature extraction model.

[0015] Optionally, the calculating, according to the positive sample sound feature vector and the enhanced sample sound feature vector, a loss value between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set using a preset loss function comprises:

[0016] The loss value Loss(·) between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set is calculated using the following formula:

[0017]

[0018] Among them, sim represents the cosine similarity function, z i represents the positive sample sound feature vector corresponding to the i-th positive sample sound data in the positive sample sound data set, represents the enhanced sample sound feature vector of the enhanced sample sound data corresponding to the i-th positive sample sound data, z j represents the positive sample sound feature vector corresponding to the j-th positive sample sound data in the positive sample sound data set, and j≠i, ρ represents a preset scaling parameter.

[0019] Optionally, performing speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set includes:

[0020] Performing tone conversion on the positive sample sound data in the positive sample sound data set to obtain a first enhanced sample sound data set;

[0021] Performing audio adjustment on the positive sample sound data in the positive sample sound data set to obtain a second enhanced sample sound data set;

[0022] injecting white noise data into the positive sample sound data in the positive sample sound data set to obtain a third enhanced sample sound data set;

[0023] The first enhanced sample sound data set, the second enhanced sample sound data set and the third enhanced sample sound data set are collected to obtain an enhanced sample sound data set.

[0024] Optionally, the obtaining of the positive sample sound data set and the sound data to be tested includes:

[0025] Acquire positive sample sound model data and to-be-tested sound model data;

[0026] Periodically scanning the positive sample sound model data and the sound model data to be tested respectively to obtain a positive sample sound discrete signal and a sound discrete signal to be tested;

[0027] Classifying the instantaneous values ​​in the positive sample sound discrete signal and the sound discrete signal to be tested into different levels according to the magnitudes of the instantaneous values ​​in the positive sample sound discrete signal and the sound discrete signal to be tested;

[0028] The levels of instantaneous values ​​in the positive sample sound discrete signal and the sound discrete signal to be tested are respectively converted into binary codes to obtain a positive sample sound data set and a sound data set to be tested.

[0029] Optionally, respectively calculating the mean and covariance of the acoustic wave feature data set to obtain a standard acoustic wave mean and a standard acoustic wave covariance includes:

[0030] Calculating the sum vector of each acoustic wave feature data in the acoustic wave feature data set, and calculating the mean of the acoustic wave feature data set according to the number of acoustic wave feature data in the acoustic wave feature data set and the sum vector of each acoustic wave feature data, to obtain a standard acoustic wave mean vector;

[0031] Calculating a difference vector between each acoustic wave feature data in the acoustic wave feature data set and the standard acoustic wave mean, and transposing the difference to obtain a transposed difference vector;

[0032] multiplying the difference vector by the transposed difference vector to obtain a difference product vector;

[0033] The sum of the difference product vectors corresponding to each acoustic wave feature data in the acoustic wave feature data set is calculated to obtain a standard acoustic wave covariance.

[0034] Optionally, calculating the distance between the acoustic wave feature data to be measured and the acoustic wave feature data set according to the standard acoustic wave mean and the standard acoustic wave covariance includes:

[0035] Calculating a difference vector of the sound wave to be measured between the characteristic data of the sound wave to be measured and the mean vector of the standard sound wave, and transposing the difference vector of the sound wave to be measured to obtain a transposed difference vector of the sound wave to be measured;

[0036] The sound wave difference vector to be measured is multiplied by the inverse of the standard sound wave covariance vector, and the multiplication result is multiplied by the inverse difference vector of the sound wave to be measured to obtain the distance between the sound wave feature data to be measured and the sound wave feature data set.

[0037] In order to solve the above problems, the present invention also provides a sound anomaly detection device based on self-supervised learning, the device comprising:

[0038] A speech data enhancement module is used to obtain a positive sample sound data set and sound data to be tested, and to perform speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set;

[0039] A feature extraction model training module, used to train a preset feature extraction model using the positive sample sound data set and the enhanced sample sound data set to obtain a trained feature extraction model;

[0040] The sound data detection module to be tested is used to use the trained feature extraction model to respectively extract the features of the positive sample sound data set and the sound data to be tested, obtain the standard sound wave feature data set and the sound wave feature data to be tested, respectively calculate the mean and covariance of the sound wave feature data set to obtain the standard sound wave mean and the standard sound wave covariance, calculate the distance between the sound wave feature data to be tested and the sound wave feature data set according to the standard sound wave mean and the standard sound wave covariance, and when the distance is greater than a preset threshold, determine that the sound data to be tested is abnormal data.

[0041] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0042] a memory storing at least one computer program; and

[0043] The processor executes the computer program stored in the memory to implement the above-mentioned sound anomaly detection method based on self-supervised learning.

[0044] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned sound anomaly detection method based on self-supervised learning.

[0045] The embodiment of the present invention uses the standard sound wave data of a normal instrument to train a feature extraction model, uses a multi-task self-supervised model to obtain the similarity and invariance of the standard sound wave data of the normal instrument, and further, calculates the mean and covariance of the standard sound wave feature data set from which the feature data is extracted, thereby avoiding the appearance of extreme data in the standard sound wave data of the normal instrument, thereby causing sound anomaly detection errors based on self-supervised learning, and improving the accuracy of sound anomaly detection based on self-supervised learning. Finally, the distance between the sound wave feature data to be tested and the sound wave feature data set is calculated, and whether the instrument to be tested is abnormal is determined based on the distance, thereby reducing the demand for abnormal samples during model training and improving the accuracy of sound anomaly detection based on self-supervised learning. Therefore, the sound anomaly detection method, device, equipment and storage medium based on self-supervised learning provided by the present invention can improve the accuracy of sound anomaly detection based on self-supervised learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a flow chart of a method for detecting sound anomalies based on self-supervised learning provided by an embodiment of the present invention;

[0047] Figure 2 and Figure 3 A detailed implementation flow chart of one of the steps in the sound anomaly detection method based on self-supervised learning provided by one embodiment of the present invention;

[0048] Figure 4 A schematic diagram of a module of a sound anomaly detection device based on self-supervised learning provided by an embodiment of the present invention;

[0049] Figure 5 A schematic diagram of the internal structure of an electronic device for implementing a sound anomaly detection method based on self-supervised learning provided by an embodiment of the present invention;

[0050] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0051] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0052] The embodiment of the present invention provides a method for detecting sound anomalies based on self-supervised learning. The execution subject of the method for detecting sound anomalies based on self-supervised learning includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided in the embodiment of the present application. In other words, the method for detecting sound anomalies based on self-supervised learning can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server may include an independent server, or may include a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0053] Reference Figure 1 FIG. 1 is a flow chart of a method for detecting anomalies of sound based on self-supervised learning provided by an embodiment of the present invention. In the embodiment of the present invention, the method for detecting anomalies of sound based on self-supervised learning includes:

[0054] S1. Obtain a positive sample sound data set and sound data to be tested, and perform speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set.

[0055] In an embodiment of the present invention, the sound data included in the positive sample sound data set may be the sound digital data of a normal instrument. The sound data to be tested may be the sound digital data of an instrument that is uncertain whether it can still operate normally and needs to be tested. For example, in the medical field, whether the operation of medical equipment is normal is extremely important for diagnosis, and it is necessary to detect whether the equipment is normal through sound. Therefore, it is necessary to obtain the sound data to be tested during the operation of the equipment to detect whether the new instrument can be used normally.

[0056] In an optional embodiment of the present invention, the sound model data of the normal instrument and the instrument to be detected are converted into sound digital data that can be written into a computer using a pulse code modulation method, thereby realizing the digital-to-analog conversion of the sound waveform, so that the positive sample sound data and the sound data to be detected can be input into the model.

[0057] Specifically, refer to Figure 2 As shown, the obtaining of the positive sample sound data set and the sound data to be tested includes:

[0058] S11, obtaining positive sample sound model data and sound model data to be tested;

[0059] S12, periodically scanning the positive sample sound model data and the sound model data to be tested respectively to obtain a positive sample sound discrete signal and a sound discrete signal to be tested;

[0060] S13, classifying the instantaneous values ​​in the positive sample sound discrete signal and the instantaneous values ​​in the sound discrete signal to be tested into different levels according to the magnitudes of the instantaneous values ​​in the positive sample sound discrete signal and the instantaneous values ​​in the sound discrete signal to be tested;

[0061] S14, converting the levels of instantaneous values ​​in the positive sample sound discrete signal and the sound discrete signal to be tested into binary codes respectively, to obtain a positive sample sound data set and a sound data set to be tested.

[0062] In an optional embodiment of the present invention, the instantaneous voltage values ​​of the positive sample sound model data and the sound model data to be tested can be collected continuously at fixed time intervals through high-frequency detection, and the instantaneous voltage values ​​are then divided into different levels. The collected instantaneous voltage values ​​are divided into different levels using the rounding principle to reduce the workload. Finally, each sampled voltage level needs to be converted into a binary code recognizable by a computer, thereby completing the conversion of the reference audio data into a reference audio digital signal, and then writing the positive sample sound data set and the sound data to be tested into the computer.

[0063] The embodiment of the present invention performs speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set, thereby enriching the types of training texts, improving the efficiency of model self-supervision, and improving the fault tolerance of the model.

[0064] Further, as an optional embodiment of the present invention, performing speech enhancement on the positive sample sound dataset to obtain an enhanced sample sound dataset includes:

[0065] Performing tone conversion on the positive sample sound data in the positive sample sound data set to obtain a first enhanced sample sound data set;

[0066] Performing audio adjustment on the positive sample sound data in the positive sample sound data set to obtain a second enhanced sample sound data set;

[0067] injecting white noise data into the positive sample sound data in the positive sample sound data set to obtain a third enhanced sample sound data set;

[0068] The first enhanced sample sound data set, the second enhanced sample sound data set and the third enhanced sample sound data set are collected to obtain an enhanced sample sound data set.

[0069] In the embodiment of the present invention, the audio adjustment generally refers to methods such as audio fast play, audio slow play and audio reverse play.

[0070] In an optional embodiment of the present invention, the positive sample sound data is enhanced by tone conversion, audio fast playback, audio slow playback, audio reverse playback and white noise injection to obtain enhanced sample sound data, so that the model can more accurately identify the sound data of normal instruments, thereby finding the sound data of abnormal instruments, and improving the accuracy of model sound recognition.

[0071] Furthermore, in an optional embodiment of the present invention, the audio adjustment of the positive sample sound data may be performed by a manual adjustment method, or the audio adjustment of the positive sample sound data may be performed according to a set script.

[0072] S2. Use the positive sample sound data set and the enhanced sample sound data set to train a preset feature extraction model to obtain a trained feature extraction model.

[0073] In an embodiment of the present invention, the preset feature extraction model may be a speech recognition model constructed based on a neural network, and may be constructed based on a Transformer structure.

[0074] In an optional embodiment of the present invention, the positive sample sound data set and the enhanced sample sound data set are used to train a preset feature extraction model to ensure the applicability of the feature extraction model, thereby improving the accuracy of the feature extraction model.

[0075] Further, as an optional embodiment of the present invention, refer to Figure 3 As shown, the method of using the positive sample sound data set and the enhanced sample sound data set to train a preset feature extraction model to obtain a trained feature extraction model includes:

[0076] S21, using the encoding layer in the preset feature extraction model to encode the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set, respectively, to obtain a positive sample sound encoding vector and an enhanced sample sound encoding vector;

[0077] S22, using the convolution layer in the preset feature extraction model to perform convolution operations on the positive sample sound coding vector and the enhanced sample sound coding vector respectively, to obtain a positive sample sound feature vector and an enhanced sample sound feature vector;

[0078] S23. According to the positive sample sound feature vector and the enhanced sample sound feature vector, a preset loss function is used to calculate the loss value between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set, and the preset feature extraction model is iteratively adjusted according to the loss value until the loss value meets the preset conditions, thereby obtaining a trained feature extraction model.

[0079] In the embodiment of the present invention, the encoding layer may be used to convert a signal into another signal that can be received by the model. The convolution layer may be used to compress the signal.

[0080] Further, as an optional embodiment of the present invention, the calculating, according to the positive sample sound feature vector and the enhanced sample sound feature vector, a loss value between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set using a preset loss function includes:

[0081] The loss value Loss(·) between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set is calculated using the following formula:

[0082]

[0083] Among them, sim represents the cosine similarity function, z i represents the positive sample sound feature vector corresponding to the i-th positive sample sound data in the positive sample sound data set, represents the enhanced sample sound feature vector of the enhanced sample sound data corresponding to the i-th positive sample sound data, z j represents the positive sample sound feature vector corresponding to the j-th positive sample sound data in the positive sample sound data set, and j≠i, ρ represents a preset scaling parameter.

[0084] In an optional embodiment of the present invention, in order to make the distribution of the loss function smoother or sharper, it is necessary to introduce a scaling parameter, wherein the scaling parameter is an artificially set model hyperparameter, and the value of the scaling parameter ρ is different in different scenarios.

[0085] In an optional implementation of the present invention, the positive sample sound data set and the enhanced sample sound data set are used as sample sets and input into a preset feature extraction model for training to obtain a trained feature extraction model, thereby improving the accuracy of the feature extraction model and also improving the accuracy of the sound data detection results when the instrument is abnormal.

[0086] S3. Using the trained feature extraction model, respectively extract features of the positive sample sound data set and the sound data to be tested, to obtain a standard sound wave feature data set and sound wave feature data to be tested.

[0087] In an optional embodiment of the present invention, in order to determine the difference between the sound data to be tested and the positive sample sound data, the features of the positive sample sound data set and the sound data to be tested can be respectively extracted by utilizing the trained feature extraction model, thereby converting the comparison of the sound data into a conversion of feature vectors, simplifying the difficulty of comparing the sound data and improving the accuracy of the sound data comparison results.

[0088] Furthermore, as an optional embodiment of the present invention, the step of using the trained feature extraction model to respectively extract features of the positive sample sound data set and the sound data to be tested is similar to the process of extracting positive sample sound feature vectors and enhanced sample sound feature vectors, and therefore will not be repeated herein.

[0089] S4. Calculate the mean and covariance of the acoustic wave feature data set respectively to obtain a standard acoustic wave mean and a standard acoustic wave covariance.

[0090] In detail, as an optional embodiment of the present invention, respectively calculating the mean and covariance of the acoustic wave feature data set to obtain a standard acoustic wave mean and a standard acoustic wave covariance includes:

[0091] Calculating the sum vector of each acoustic wave feature data in the acoustic wave feature data set, and calculating the mean of the acoustic wave feature data set according to the number of acoustic wave feature data in the acoustic wave feature data set and the sum vector of each acoustic wave feature data, to obtain a standard acoustic wave mean vector;

[0092] Calculating a difference vector between each acoustic wave feature data in the acoustic wave feature data set and the standard acoustic wave mean, and transposing the difference to obtain a transposed difference vector;

[0093] multiplying the difference vector by the transposed difference vector to obtain a difference product vector;

[0094] The sum of the difference product vectors corresponding to each acoustic wave feature data in the acoustic wave feature data set is calculated to obtain a standard acoustic wave covariance.

[0095] In an optional embodiment of the present invention, the mean of the acoustic wave feature data set can be calculated using the following formula to obtain the standard acoustic wave mean μ:

[0096]

[0097] Among them, z i represents the positive sample sound feature vector corresponding to the ith positive sample sound data in the positive sample sound data set, and N represents the total amount of positive sample sound data in the positive sample sound data set.

[0098] Furthermore, in an optional embodiment of the present invention, the covariance of the acoustic wave feature data set can be calculated using the following formula to obtain a standard acoustic wave covariance Σ:

[0099]

[0100] Among them, z i represents the positive sample sound feature vector corresponding to the i-th positive sample sound data in the positive sample sound data set, μ represents the standard sound wave mean, T represents transposition, and N represents the total amount of positive sample sound data in the positive sample sound data set.

[0101] In an optional embodiment of the present invention, since the acoustic wave feature data set cannot be completely consistent, in order to improve the fault tolerance of the acoustic wave feature data set, the mean and covariance of the acoustic wave feature data set are respectively calculated using the formula for calculating the average value and the formula for calculating the covariance, thereby reducing the error caused by the inconsistency of the acoustic wave feature data in the acoustic wave feature data set and improving the accuracy of sound abnormality detection.

[0102] S5. Calculate the distance between the sound wave feature data to be measured and the sound wave feature data set according to the standard sound wave mean and the standard sound wave covariance, and determine that the sound data to be measured is abnormal data when the distance is greater than a preset threshold.

[0103] In the embodiment of the present invention, the preset threshold may be a numerical value of different sizes set by a business person according to different scenarios.

[0104] The embodiment of the present invention calculates the distance between the sound wave feature data to be measured and the sound wave feature data set according to the standard sound wave mean and the standard sound wave covariance, thereby solving the problem of too few abnormal samples and inability to perform model training during sound anomaly detection, and improving the accuracy of sound anomaly detection.

[0105] In an optional embodiment of the present invention, the distance between the acoustic wave characteristic data to be measured and the acoustic wave characteristic data set may be calculated using a Mahalanobis distance formula.

[0106] Further, as an optional embodiment of the present invention, the calculating the distance between the acoustic wave feature data to be measured and the acoustic wave feature data set according to the standard acoustic wave mean and the standard acoustic wave covariance includes:

[0107] Calculating a difference vector of the sound wave to be measured between the characteristic data of the sound wave to be measured and the mean vector of the standard sound wave, and transposing the difference vector of the sound wave to be measured to obtain a transposed difference vector of the sound wave to be measured;

[0108] The sound wave difference vector to be measured is multiplied by the inverse of the standard sound wave covariance vector, and the multiplication result is multiplied by the inverse difference vector of the sound wave to be measured to obtain the distance between the sound wave feature data to be measured and the sound wave feature data set.

[0109] Specifically, in the embodiment of the present invention, the distance S between the acoustic wave characteristic data to be measured and the acoustic wave characteristic data set can be calculated using the following formula: test :

[0110] S test =(z test -μ) T Σ -1 (z test -μ)

[0111] Among them, z test represents the characteristic data of the sound wave to be measured, μ represents the mean of the standard sound wave, T represents the transpose, and Σ represents the covariance of the standard sound wave.

[0112] In the embodiment of the present invention, when the distance between the sound wave characteristic data to be measured and the sound wave characteristic data set is larger, it means that the sound wave characteristic data to be measured is far different from the sound wave characteristic data in the sound wave characteristic data set. Therefore, the sound data to be measured is judged to be abnormal data. Conversely, when the distance between the sound wave characteristic data to be measured and the sound wave characteristic data set is smaller, it means that the sound wave characteristic data to be measured is close to the sound wave characteristic data in the sound wave characteristic data set. Therefore, the sound data to be measured is judged to be normal data.

[0113] like Figure 4 , which is a functional module diagram of the sound anomaly detection device based on self-supervised learning of the present invention.

[0114] The sound anomaly detection device 100 based on self-supervised learning of the present invention can be installed in an electronic device. According to the functions to be implemented, the sound anomaly detection device 100 based on self-supervised learning can include a speech data enhancement module 101, a feature extraction model training module 102 and a sound data detection module 103 to be tested. The module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0115] In this embodiment, the functions of each module / unit are as follows:

[0116] The speech data enhancement module 101 is used to obtain a positive sample sound data set and sound data to be tested, and perform speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set.

[0117] In an embodiment of the present invention, the sound data included in the positive sample sound data set may be the sound digital data of a normal instrument. The sound data to be tested may be the sound digital data of an instrument that is uncertain whether it can still operate normally and needs to be tested. For example, in the medical field, whether the operation of medical equipment is normal is extremely important for diagnosis, and it is necessary to detect whether the equipment is normal through sound. Therefore, it is necessary to obtain the sound data to be tested during the operation of the equipment to detect whether the new instrument can be used normally.

[0118] In an optional embodiment of the present invention, the sound model data of the normal instrument and the instrument to be detected are converted into sound digital data that can be written into a computer using a pulse code modulation method, thereby realizing the digital-to-analog conversion of the sound waveform, so that the positive sample sound data and the sound data to be detected can be input into the model.

[0119] Specifically, the obtaining of the positive sample sound data set and the sound data to be tested includes:

[0120] Acquire positive sample sound model data and to-be-tested sound model data;

[0121] Periodically scanning the positive sample sound model data and the sound model data to be tested respectively to obtain a positive sample sound discrete signal and a sound discrete signal to be tested;

[0122] Classifying the instantaneous values ​​in the positive sample sound discrete signal and the sound discrete signal to be tested into different levels according to the magnitudes of the instantaneous values ​​in the positive sample sound discrete signal and the sound discrete signal to be tested;

[0123] The levels of instantaneous values ​​in the positive sample sound discrete signal and the sound discrete signal to be tested are respectively converted into binary codes to obtain a positive sample sound data set and a sound data set to be tested.

[0124] In an optional embodiment of the present invention, the instantaneous voltage values ​​of the positive sample sound model data and the sound model data to be tested can be collected continuously at fixed time intervals through high-frequency detection, and the instantaneous voltage values ​​are then divided into different levels. The collected instantaneous voltage values ​​are divided into different levels using the rounding principle to reduce the workload. Finally, each sampled voltage level needs to be converted into a binary code recognizable by a computer, thereby completing the conversion of the reference audio data into a reference audio digital signal, and then writing the positive sample sound data set and the sound data to be tested into the computer.

[0125] The embodiment of the present invention performs speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set, thereby enriching the types of training texts, improving the efficiency of model self-supervision, and improving the fault tolerance of the model.

[0126] Further, as an optional embodiment of the present invention, performing speech enhancement on the positive sample sound dataset to obtain an enhanced sample sound dataset includes:

[0127] Performing tone conversion on the positive sample sound data in the positive sample sound data set to obtain a first enhanced sample sound data set;

[0128] Performing audio adjustment on the positive sample sound data in the positive sample sound data set to obtain a second enhanced sample sound data set;

[0129] injecting white noise data into the positive sample sound data in the positive sample sound data set to obtain a third enhanced sample sound data set;

[0130] The first enhanced sample sound data set, the second enhanced sample sound data set and the third enhanced sample sound data set are collected to obtain an enhanced sample sound data set.

[0131] In the embodiment of the present invention, the audio adjustment generally refers to methods such as audio fast play, audio slow play and audio reverse play.

[0132] In an optional embodiment of the present invention, the positive sample sound data is enhanced by tone conversion, audio fast playback, audio slow playback, audio reverse playback and white noise injection to obtain enhanced sample sound data, so that the model can more accurately identify the sound data of normal instruments, thereby finding the sound data of abnormal instruments, and improving the accuracy of model sound recognition.

[0133] Furthermore, in an optional embodiment of the present invention, the audio adjustment of the positive sample sound data may be performed by a manual adjustment method, or the audio adjustment of the positive sample sound data may be performed according to a set script.

[0134] The feature extraction model training module 102 is used to train a preset feature extraction model using the positive sample sound data set and the enhanced sample sound data set to obtain a trained feature extraction model.

[0135] In an embodiment of the present invention, the preset feature extraction model may be a speech recognition model constructed based on a neural network, and may be constructed based on a Transformer structure.

[0136] In an optional embodiment of the present invention, the positive sample sound data set and the enhanced sample sound data set are used to train a preset feature extraction model to ensure the applicability of the feature extraction model, thereby improving the accuracy of the feature extraction model.

[0137] Further, as an optional embodiment of the present invention, the using the positive sample sound data set and the enhanced sample sound data set to train a preset feature extraction model to obtain a trained feature extraction model includes:

[0138] Using the encoding layer in the preset feature extraction model, respectively encode the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set to obtain a positive sample sound encoding vector and an enhanced sample sound encoding vector;

[0139] Using the convolution layer in the preset feature extraction model to perform convolution operations on the positive sample sound coding vector and the enhanced sample sound coding vector respectively, to obtain the positive sample sound feature vector and the enhanced sample sound feature vector;

[0140] According to the positive sample sound feature vector and the enhanced sample sound feature vector, a preset loss function is used to calculate the loss value between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set, and the parameters of the preset feature extraction model are iteratively adjusted according to the loss value until the loss value meets the preset conditions, thereby obtaining a trained feature extraction model.

[0141] In the embodiment of the present invention, the encoding layer may be used to convert a signal into another signal that can be received by the model. The convolution layer may be used to compress the signal.

[0142] Further, as an optional embodiment of the present invention, the calculating, according to the positive sample sound feature vector and the enhanced sample sound feature vector, a loss value between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set using a preset loss function includes:

[0143] The loss value Loss(·) between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set is calculated using the following formula:

[0144]

[0145] Among them, sim represents the cosine similarity function, z i represents the positive sample sound feature vector corresponding to the i-th positive sample sound data in the positive sample sound data set, represents the enhanced sample sound feature vector of the enhanced sample sound data corresponding to the i-th positive sample sound data, z jrepresents the positive sample sound feature vector corresponding to the j-th positive sample sound data in the positive sample sound data set, and j≠i, ρ represents a preset scaling parameter.

[0146] In an optional embodiment of the present invention, in order to make the distribution of the loss function smoother or sharper, it is necessary to introduce a scaling parameter, wherein the scaling parameter is an artificially set model hyperparameter, and the value of the scaling parameter ρ is different in different scenarios.

[0147] In an optional implementation of the present invention, the positive sample sound data set and the enhanced sample sound data set are used as sample sets and input into a preset feature extraction model for training to obtain a trained feature extraction model, thereby improving the accuracy of the feature extraction model and also improving the accuracy of the sound data detection results when the instrument is abnormal.

[0148] The sound data to be tested detection module 103 is used to use the trained feature extraction model to respectively extract the features of the positive sample sound data set and the sound data to be tested, obtain a standard sound wave feature data set and a sound wave feature data to be tested, respectively calculate the mean and covariance of the sound wave feature data set, obtain a standard sound wave mean and a standard sound wave covariance, calculate the distance between the sound wave feature data to be tested and the sound wave feature data set based on the standard sound wave mean and the standard sound wave covariance, and when the distance is greater than a preset threshold, determine that the sound data to be tested is abnormal data.

[0149] In an optional embodiment of the present invention, in order to determine the difference between the sound data to be tested and the positive sample sound data, the features of the positive sample sound data set and the sound data to be tested can be respectively extracted by utilizing the trained feature extraction model, thereby converting the comparison of the sound data into a conversion of feature vectors, simplifying the difficulty of comparing the sound data and improving the accuracy of the sound data comparison results.

[0150] Furthermore, as an optional embodiment of the present invention, the step of using the trained feature extraction model to respectively extract features of the positive sample sound data set and the sound data to be tested is similar to the process of extracting positive sample sound feature vectors and enhanced sample sound feature vectors, and therefore will not be repeated herein.

[0151] In detail, as an optional embodiment of the present invention, respectively calculating the mean and covariance of the acoustic wave feature data set to obtain a standard acoustic wave mean and a standard acoustic wave covariance includes:

[0152] Calculating the sum vector of each acoustic wave feature data in the acoustic wave feature data set, and calculating the mean of the acoustic wave feature data set according to the number of acoustic wave feature data in the acoustic wave feature data set and the sum vector of each acoustic wave feature data, to obtain a standard acoustic wave mean vector;

[0153] Calculating a difference vector between each acoustic wave feature data in the acoustic wave feature data set and the standard acoustic wave mean, and transposing the difference to obtain a transposed difference vector;

[0154] multiplying the difference vector by the transposed difference vector to obtain a difference product vector;

[0155] The sum of the difference product vectors corresponding to each acoustic wave feature data in the acoustic wave feature data set is calculated to obtain a standard acoustic wave covariance.

[0156] In an optional embodiment of the present invention, the mean of the acoustic wave feature data set can be calculated using the following formula to obtain the standard acoustic wave mean μ:

[0157]

[0158] Among them, z i represents the positive sample sound feature vector corresponding to the ith positive sample sound data in the positive sample sound data set, and N represents the total amount of positive sample sound data in the positive sample sound data set.

[0159] Furthermore, in an optional embodiment of the present invention, the covariance of the acoustic wave feature data set can be calculated using the following formula to obtain a standard acoustic wave covariance Σ:

[0160]

[0161] Among them, z i represents the positive sample sound feature vector corresponding to the i-th positive sample sound data in the positive sample sound data set, μ represents the standard sound wave mean, T represents transposition, and N represents the total amount of positive sample sound data in the positive sample sound data set.

[0162] In an optional embodiment of the present invention, since the acoustic wave feature data set cannot be completely consistent, in order to improve the fault tolerance of the acoustic wave feature data set, the mean and covariance of the acoustic wave feature data set are respectively calculated using the formula for calculating the average value and the formula for calculating the covariance, thereby reducing the error caused by the inconsistency of the acoustic wave feature data in the acoustic wave feature data set and improving the accuracy of sound abnormality detection.

[0163] In the embodiment of the present invention, the preset threshold may be a numerical value of different sizes set by a business person according to different scenarios.

[0164] The embodiment of the present invention calculates the distance between the sound wave feature data to be measured and the sound wave feature data set according to the standard sound wave mean and the standard sound wave covariance, thereby solving the problem of too few abnormal samples and inability to perform model training during sound anomaly detection, and improving the accuracy of sound anomaly detection.

[0165] In an optional embodiment of the present invention, the distance between the acoustic wave characteristic data to be measured and the acoustic wave characteristic data set may be calculated using a Mahalanobis distance formula.

[0166] Further, as an optional embodiment of the present invention, the calculating the distance between the acoustic wave feature data to be measured and the acoustic wave feature data set according to the standard acoustic wave mean and the standard acoustic wave covariance includes:

[0167] Calculating a difference vector of the sound wave to be measured between the characteristic data of the sound wave to be measured and the mean vector of the standard sound wave, and transposing the difference vector of the sound wave to be measured to obtain a transposed difference vector of the sound wave to be measured;

[0168] The sound wave difference vector to be measured is multiplied by the inverse of the standard sound wave covariance vector, and the multiplication result is multiplied by the inverse difference vector of the sound wave to be measured to obtain the distance between the sound wave feature data to be measured and the sound wave feature data set.

[0169] Specifically, in the embodiment of the present invention, the distance S between the acoustic wave characteristic data to be measured and the acoustic wave characteristic data set can be calculated using the following formula: test :

[0170] S test =(z test -μ) T Σ -1 (z test -μ)

[0171] Among them, z test represents the characteristic data of the sound wave to be measured, μ represents the mean of the standard sound wave, T represents the transpose, and Σ represents the covariance of the standard sound wave.

[0172] In the embodiment of the present invention, when the distance between the sound wave characteristic data to be measured and the sound wave characteristic data set is larger, it means that the sound wave characteristic data to be measured is far different from the sound wave characteristic data in the sound wave characteristic data set. Therefore, the sound data to be measured is judged to be abnormal data. Conversely, when the distance between the sound wave characteristic data to be measured and the sound wave characteristic data set is smaller, it means that the sound wave characteristic data to be measured is close to the sound wave characteristic data in the sound wave characteristic data set. Therefore, the sound data to be measured is judged to be normal data.

[0173] like Figure 5 , which is a schematic diagram of the structure of an electronic device for implementing a sound anomaly detection method based on self-supervised learning according to the present invention.

[0174] The electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a sound anomaly detection program based on self-supervised learning.

[0175] Among them, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 may be an internal storage unit of an electronic device in some embodiments, such as a mobile hard disk of the electronic device. The memory 11 may also be an external storage device of an electronic device in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a sound anomaly detection program based on self-supervised learning, but also can be used to temporarily store data that has been output or is to be output.

[0176] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (such as a sound anomaly detection program based on self-supervised learning, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device and process data.

[0177] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The communication bus 12 is configured to realize connection and communication between the memory 11 and at least one processor 10, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0178] Figure 5 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0179] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0180] Optionally, the communication interface 13 may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices.

[0181] Optionally, the communication interface 13 may also include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0182] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0183] The sound anomaly detection program based on self-supervised learning stored in the memory 11 of the electronic device is a combination of multiple computer programs. When running in the processor 10, it can achieve:

[0184] Acquire a positive sample sound data set and sound data to be tested, and perform speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set;

[0185] Using the positive sample sound data set and the enhanced sample sound data set to train a preset feature extraction model to obtain a trained feature extraction model;

[0186] The trained feature extraction model is used to extract the features of the positive sample sound data set and the sound data to be tested, respectively, to obtain a standard sound wave feature data set and sound wave feature data to be tested;

[0187] Calculating the mean and covariance of the acoustic wave feature data set respectively to obtain a standard acoustic wave mean and a standard acoustic wave covariance;

[0188] The distance between the sound wave feature data to be measured and the sound wave feature data set is calculated according to the standard sound wave mean and the standard sound wave covariance, and when the distance is greater than a preset threshold, the sound data to be measured is determined to be abnormal data.

[0189] Specifically, the specific implementation method of the processor 10 for the above computer program can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0190] Furthermore, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium can be non-volatile or volatile. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0191] The embodiment of the present invention may also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program may implement:

[0192] Acquire a positive sample sound data set and sound data to be tested, and perform speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set;

[0193] Using the positive sample sound data set and the enhanced sample sound data set to train a preset feature extraction model to obtain a trained feature extraction model;

[0194] The trained feature extraction model is used to extract the features of the positive sample sound data set and the sound data to be tested, respectively, to obtain a standard sound wave feature data set and sound wave feature data to be tested;

[0195] Calculating the mean and covariance of the acoustic wave feature data set respectively to obtain a standard acoustic wave mean and a standard acoustic wave covariance;

[0196] The distance between the sound wave feature data to be measured and the sound wave feature data set is calculated according to the standard sound wave mean and the standard sound wave covariance, and when the distance is greater than a preset threshold, the sound data to be measured is determined to be abnormal data.

[0197] Furthermore, the computer-usable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0198] In the several embodiments provided by the present invention, it should be understood that the disclosed electronic device, apparatus and method can be implemented in other ways. For example, the apparatus embodiment described above is only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0199] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0200] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0201] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0202] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0203] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0204] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A sound anomaly detection method based on self-supervised learning, characterized in that: The method comprises: Acquire a positive sample sound data set and sound data to be tested, and perform speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set; Using the positive sample sound data set and the enhanced sample sound data set to train a preset feature extraction model to obtain a trained feature extraction model; The trained feature extraction model is used to extract the features of the positive sample sound data set and the sound data to be tested, respectively, to obtain a standard sound wave feature data set and sound wave feature data to be tested; Calculating the mean and covariance of the acoustic wave feature data set respectively to obtain a standard acoustic wave mean and a standard acoustic wave covariance; The distance between the sound wave feature data to be measured and the sound wave feature data set is calculated according to the standard sound wave mean and the standard sound wave covariance, and when the distance is greater than a preset threshold, the sound data to be measured is determined to be abnormal data.

2. The method for detecting anomalies of sound based on self-supervised learning according to claim 1, characterized in that: The method of using the positive sample sound data set and the enhanced sample sound data set to train a preset feature extraction model to obtain a trained feature extraction model includes: Using the encoding layer in the preset feature extraction model, respectively encode the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set to obtain a positive sample sound encoding vector and an enhanced sample sound encoding vector; Using the convolution layer in the preset feature extraction model to perform convolution operations on the positive sample sound coding vector and the enhanced sample sound coding vector respectively, to obtain a positive sample sound feature vector and an enhanced sample sound feature vector; According to the positive sample sound feature vector and the enhanced sample sound feature vector, a preset loss function is used to calculate the loss value between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set, and the parameters of the preset feature extraction model are iteratively adjusted according to the loss value until the loss value meets the preset conditions, thereby obtaining a trained feature extraction model.

3. The method for detecting anomalies of sound based on self-supervised learning according to claim 2, characterized in that: The method of calculating the loss value between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set using a preset loss function according to the positive sample sound feature vector and the enhanced sample sound feature vector comprises: The loss value Loss(·) between the positive sample sound data in the positive sample sound data set and the enhanced sample sound data in the enhanced sample sound data set is calculated using the following formula: Among them, sim represents the cosine similarity function, z i represents the positive sample sound feature vector corresponding to the i-th positive sample sound data in the positive sample sound data set, represents the enhanced sample sound feature vector of the enhanced sample sound data corresponding to the i-th positive sample sound data, z j represents the positive sample sound feature vector corresponding to the j-th positive sample sound data in the positive sample sound data set, and j≠i, ρ represents a preset scaling parameter.

4. The method for detecting anomalies of sound based on self-supervised learning according to claim 1, characterized in that: The performing speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set includes: Performing tone conversion on the positive sample sound data in the positive sample sound data set to obtain a first enhanced sample sound data set; Performing audio adjustment on the positive sample sound data in the positive sample sound data set to obtain a second enhanced sample sound data set; injecting white noise data into the positive sample sound data in the positive sample sound data set to obtain a third enhanced sample sound data set; The first enhanced sample sound data set, the second enhanced sample sound data set and the third enhanced sample sound data set are collected to obtain an enhanced sample sound data set.

5. The method for detecting anomalies of sound based on self-supervised learning according to claim 1, characterized in that: The obtaining of the positive sample sound data set and the sound data to be tested comprises: Acquire positive sample sound model data and to-be-tested sound model data; Periodically scanning the positive sample sound model data and the sound model data to be tested respectively to obtain a positive sample sound discrete signal and a sound discrete signal to be tested; Classifying the instantaneous values ​​in the positive sample sound discrete signal and the sound discrete signal to be tested into different levels according to the magnitudes of the instantaneous values ​​in the positive sample sound discrete signal and the sound discrete signal to be tested; The levels of instantaneous values ​​in the positive sample sound discrete signal and the sound discrete signal to be tested are respectively converted into binary codes to obtain a positive sample sound data set and a sound data set to be tested.

6. The method for detecting anomalies of sound based on self-supervised learning according to claim 1, characterized in that: The step of respectively calculating the mean and covariance of the acoustic wave feature data set to obtain a standard acoustic wave mean and a standard acoustic wave covariance includes: Calculating the sum vector of each acoustic wave feature data in the acoustic wave feature data set, and calculating the mean of the acoustic wave feature data set according to the number of acoustic wave feature data in the acoustic wave feature data set and the sum vector of each acoustic wave feature data, to obtain a standard acoustic wave mean vector; Calculating a difference vector between each acoustic wave feature data in the acoustic wave feature data set and the standard acoustic wave mean, and transposing the difference to obtain a transposed difference vector; multiplying the difference vector by the transposed difference vector to obtain a difference product vector; The sum of the difference product vectors corresponding to each acoustic wave feature data in the acoustic wave feature data set is calculated to obtain a standard acoustic wave covariance.

7. The method for detecting anomalies of sound based on self-supervised learning according to claim 1, characterized in that: The step of calculating the distance between the acoustic wave feature data to be measured and the acoustic wave feature data set according to the standard acoustic wave mean and the standard acoustic wave covariance includes: Calculating a difference vector of the sound wave to be measured between the characteristic data of the sound wave to be measured and the mean vector of the standard sound wave, and transposing the difference vector of the sound wave to be measured to obtain a transposed difference vector of the sound wave to be measured; The measured sound wave difference vector is multiplied by the inverse of the standard sound wave covariance vector, and the multiplication result is multiplied by the measured sound wave inverted difference vector to obtain the distance between the measured sound wave feature data and the sound wave feature data set.

8. A sound anomaly detection device based on self-supervised learning, characterized in that: The device comprises: A speech data enhancement module is used to obtain a positive sample sound data set and sound data to be tested, and to perform speech enhancement on the positive sample sound data set to obtain an enhanced sample sound data set; A feature extraction model training module, used to train a preset feature extraction model using the positive sample sound data set and the enhanced sample sound data set to obtain a trained feature extraction model; The sound data to be tested detection module is used to use the trained feature extraction model to respectively extract the features of the positive sample sound data set and the sound data to be tested, obtain a standard sound wave feature data set and the sound wave feature data to be tested, respectively calculate the mean and covariance of the sound wave feature data set, obtain the standard sound wave mean and the standard sound wave covariance, calculate the distance between the sound wave feature data to be tested and the sound wave feature data set according to the standard sound wave mean and the standard sound wave covariance, and when the distance is greater than a preset threshold, determine that the sound data to be tested is abnormal data.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to perform the sound anomaly detection method based on self-supervised learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting sound anomalies based on self-supervised learning according to any one of claims 1 to 7 is implemented.

Images