Voiceprint recognition model, voiceprint recognition training method, voiceprint recognition method, program product, medium and device
Through the voiceprint recognition model, the device operation voiceprint is classified using feature extraction networks and residual networks, which solves the complex problem of equipment failure monitoring in the existing technology and achieves efficient and accurate fault recognition.
Patent Information
- Application Number
- CN202411364082.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-28
- Publication Date
- 2025-02-14
AI Technical Summary
In the prior art, equipment failure monitoring is more complex, relying on the experience of multiple sensors and engineers, and is costly and inefficient.
The voiceprint recognition model is adopted to classify the device's operating voiceprint through feature extraction networks and residual networks, and the differentiated features of real-time voiceprints compared to normal operating voiceprints are trained to improve the recognition accuracy of the model.
It realizes accurate identification of equipment failures, reduces hardware costs and engineers' experience requirements, and improves monitoring efficiency.
Smart Images
Figure CN119446184A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of equipment monitoring, and in particular to a voiceprint recognition model and training method, a voiceprint recognition method, a program product, a medium and equipment. Background Art
[0002] With the rapid development of science and technology, today's mechanical equipment is becoming more and more sophisticated and expensive, which also puts higher requirements on the reliability of mechanical equipment. Therefore, in order to ensure the normal operation of mechanical equipment, mechanical equipment is generally inspected manually on a regular basis to identify whether there is a fault. However, fault identification requires operators to have rich experience and also relies on engineers to monitor as frequently as possible.
[0003] Subsequently, more intelligent detection methods emerged. Thanks to the development of sensors, existing detection methods have gradually become automated. By installing sensors on mechanical equipment, the operating conditions of mechanical equipment, such as temperature, voltage, current, vibration intensity, and voiceprint, can be monitored in real time, and then uploaded to the designated server and software platform through the communication module or network, which can basically achieve real-time remote monitoring. Engineers can integrate the data through these monitoring data and analyze the fault conditions of mechanical equipment. Although this method achieves a certain degree of automated monitoring, it has high hardware requirements (multiple sensors, etc.) and still requires certain fault analysis experience of engineers. Therefore, an accurate and economical fault identification method is needed. Summary of the invention
[0004] In view of the deficiencies in the prior art, the present invention provides a voiceprint recognition model and training method, a voiceprint recognition method, a program product, a medium and a device, which mainly solve the complex technical problem of equipment fault monitoring in the prior art.
[0005] The objective of the present invention is achieved through the following solutions:
[0006] In a first aspect, according to an embodiment of the present invention, there is provided a method for training a voiceprint recognition model, wherein the voiceprint recognition model classifies device faults based on device operation voiceprints, and the training method comprises: S1, initializing the voiceprint recognition model, wherein the voiceprint recognition model comprises a feature extraction network and a residual network, wherein the feature extraction network is used to extract features of an input voiceprint, and the residual network is used to calculate residual features of a voiceprint to be identified relative to a reference voiceprint; S2, constructing a training data set, wherein the training data set comprises measured voiceprints of devices under different faults, and the fault type corresponding to each measured voiceprint is used as its label, and the training data set also comprises measured voiceprints of devices without faults as reference voiceprints; S3, training the voiceprint recognition model based on the constructed training data set, and during the training process, a preset loss function is used to calculate the loss and update the model parameters.
[0007] According to an embodiment of the present invention, the feature extraction network includes multiple convolutional layers, and each convolutional layer is connected to a batch normalization layer and an activation function layer.
[0008] According to an embodiment of the present invention, the residual features of the voiceprint to be identified relative to the reference voiceprint are calculated as follows: S21, respectively obtaining features of the voiceprint to be identified and the reference voiceprint; S22, calculating the difference between the features of the voiceprint to be identified and the features of the reference voiceprint.
[0009] According to an embodiment of the present invention, the loss is a cross entropy loss.
[0010] According to an embodiment of the present invention, the cross entropy loss is:
[0011]
[0012] Among them, y represents the label failure type, Indicates the predicted failure type.
[0013] In a second aspect, according to another embodiment of the present invention, a voiceprint recognition model is provided for classifying device failures based on the device operation voiceprint. The model is configured to be trained using the method described in the first aspect, and includes: a feature extraction network for extracting features of an input voiceprint; and a residual network for calculating the residual features of the voiceprint to be identified relative to a reference voiceprint.
[0014] In a third aspect, according to another embodiment of the present invention, a voiceprint recognition method is provided for classifying equipment failures based on equipment operation voiceprints, the method comprising: P1, obtaining equipment operation voiceprints and reference voiceprints; and P2, using a voiceprint recognition model as described in the second aspect to classify equipment operation voiceprints.
[0015] In a fourth aspect, according to another embodiment of the present invention, a computer program product is provided, wherein the computer program product includes a computer program code, and when the computer program code runs on an electronic device, the electronic device executes the method as described in any one of the first aspect or the third aspect.
[0016] In the fifth aspect, according to another embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program can be executed by a processor to implement the steps of the method described in the first aspect or the third aspect.
[0017] In the sixth aspect, according to another embodiment of the present invention, an electronic device is provided, comprising: one or more processors; and a memory, wherein the memory is used to store executable instructions; the one or more processors are configured to implement the steps of the method described in the first aspect or the third aspect by executing the executable instructions.
[0018] Compared with the prior art, the present invention has the following beneficial effects: when conducting voiceprint recognition training, by using the distinguishing features of real-time voiceprints relative to normal operating voiceprints for training, the model can be more focused on the differences in device voiceprints under different faults, thereby making the model more accurate in identifying voiceprints and, in turn, more accurate in identifying faults. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The embodiments of the present invention are further described below with reference to the accompanying drawings:
[0020] Figure 1 The figure is a flow chart of a method for training a voiceprint recognition model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0021] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0022] As mentioned in the background technology, there are relatively complex problems with the automatic detection methods in the prior art. For example, through the integration of multiple sensors, the monitoring of multiple equipment parameters is achieved. Although this method enriches the information that engineers learn, the hardware cost increases, and it also puts higher requirements on engineers to integrate multiple information.
[0023] The applicant also found that there are also studies on single sensors (such as voiceprint sensors). First, during the operation of mechanical equipment such as transformers, the iron core, winding and other structures will vibrate and transmit mechanical waves. The generated vibration and audible sound signals contain a large amount of equipment status information. For example, experiments have shown that various defects such as transformer overload, unbalanced load, harmonic load, severe overheating, DC bias, partial discharge, loose winding core, loose accessories, etc. are all related to the sound and vibration of transformer operation. In addition, the sound sensor does not need to contact the live equipment, does not change the operating status of the equipment, and is easy to install; it can track the changes in the operating status of the equipment in real time and has high detection sensitivity. Existing sound research methods include the following methods.
[0024] 1. Time domain analysis: This is the most direct method, which directly analyzes the audio signal in the time domain. For example, we can extract the statistical features of the audio signal, such as mean, variance, peak, etc. In addition, we can also extract the dynamic features of the audio signal, such as zero crossing rate (ZCR), short time energy (STE), etc. These features can reflect the changing characteristics of the audio signal in the time domain.
[0025] 2. Frequency domain analysis: Frequency domain analysis is to convert the audio signal from the time domain to the frequency domain for analysis. Commonly used methods include Fourier Transform (FT) and its variants such as Short Time Fourier Transform (STFT). Through these methods, we can get the spectrum of the audio signal, that is, the intensity of the signal at each frequency. The spectrum can reflect the frequency characteristics of the audio signal, such as the scale of music, human voice, etc.
[0026] 3. Cepstrum analysis: Cepstrum analysis is a method for extracting periodic features in audio signals. The most commonly used one is Mel Frequency Cepstral Coefficients (MFCC), which is widely used in tasks such as speech recognition. MFCC first uses a filter bank to filter the spectrum, then takes the logarithm, then performs a discrete cosine transform (DCT), and finally takes the first few coefficients as features. These features can reflect the periodic characteristics in the audio signal, such as the phonemes in speech.
[0027] 4. Feature extraction methods based on deep learning: In recent years, deep learning has also achieved great success in audio feature extraction. For example, Convolutional Neural Networks (CNN) can automatically learn the time-frequency characteristics of audio signals, and Recurrent Neural Networks (RNN) can learn the timing characteristics of audio signals. These methods require a large amount of data for training, but once trained, they can extract very effective features.
[0028] However, even relatively complex deep learning-based sound recognition methods only achieve unsatisfactory recognition accuracy, and this is based on a large number of training samples and a long training time.
[0029] In response to the problem that the above-mentioned sound recognition method is not accurate enough, the applicant has found that it is possible to further focus on the difference between real-time voiceprints and normal voiceprints. This can highlight the difference in device voiceprints under different faults, especially when the strength of the normal voiceprint of the device is large, or the change trends between the voiceprints are similar. In addition, generally speaking, when only some components fail or a single component fails and the fault sound generated by the faulty component is not obvious, the overall sound of the device generated is still similar to the sound of the normal device. Figure 1 As shown, according to an embodiment of the present invention, a training method for a voiceprint recognition model is provided, wherein the voiceprint recognition model classifies device faults based on device operation voiceprints, and the training method comprises: S1, initializing the voiceprint recognition model, wherein the voiceprint recognition model comprises a feature extraction network and a residual network, wherein the feature extraction network is used to extract features of the input voiceprint, and the residual network is used to calculate the residual features of the voiceprint to be identified relative to the reference voiceprint; S2, constructing a training data set, wherein the training data set comprises the measured voiceprints of devices under different faults, and the fault type corresponding to each measured voiceprint is used as its label, and the training data set also comprises the measured voiceprints of the device without faults as the reference voiceprint; S3, training the voiceprint recognition model based on the constructed training data set, and using a preset loss function to calculate the loss during the training process, and updating the model parameters. When performing voiceprint recognition training, by using the distinguishing features of the real-time voiceprint relative to the normal operation voiceprint for training, the model can be more focused on the differences in the device voiceprints under different faults, so that the model can recognize the voiceprint more accurately, and thus more accurately recognize the fault. Faults include, for example, transformer overload, unbalanced load, harmonic load, severe overheating, DC bias, partial discharge, loose winding core, loose accessories, etc.
[0030] The sound is collected by a sound sensor. Example sound sensors include bone conduction sensors or ultra-high frequency sensors. After the sound is collected, the sound is pre-processed such as noise reduction and silence removal to extract useful sound samples. In addition, the neural network models that can be used include Gaussian mixture models, hidden Markov models, and deep neural network models.
[0031] According to an embodiment of the present invention, the feature extraction network includes multiple convolutional layers, each of which is connected to a batch normalization layer and an activation function layer, wherein the activation function is a ReLU activation function.
[0032] According to an embodiment of the present invention, the residual features of the voiceprint to be identified relative to the reference voiceprint are calculated as follows: S21, respectively obtaining the features of the voiceprint to be identified and the reference voiceprint; S22, calculating the difference between the features of the voiceprint to be identified and the features of the reference voiceprint. Using the distinguishing features of the real-time voiceprint relative to the normal operation voiceprint for training can make the model more focused on the differences in the device voiceprints under different faults. This also helps to reduce the model network layers and shorten the training time.
[0033] According to an embodiment of the present invention, the loss is a cross entropy loss. In addition, other losses effective for speech recognition may also be used, such as mean square error loss and CTC (Connectionist Temporal Classification) loss.
[0034] According to an embodiment of the present invention, the cross entropy loss is:
[0035]
[0036] Among them, y represents the label failure type, Indicates the predicted failure type.
[0037] According to another embodiment of the present invention, a voiceprint recognition model is provided for classifying device failures based on device operation voiceprints. The model is configured to be trained using the method described above, and includes: a feature extraction network for extracting features of an input voiceprint; and a residual network for calculating the residual features of the voiceprint to be identified relative to a reference voiceprint.
[0038] According to another embodiment of the present invention, a voiceprint recognition method is provided for classifying device failures based on device operation voiceprints, the method comprising: P1, obtaining device operation voiceprints and reference voiceprints; and P2, using a voiceprint recognition model as described above to classify device operation voiceprints.
[0039] According to another embodiment of the present invention, a computer program product is provided, wherein the computer program product comprises a computer program code, and when the computer program code runs on an electronic device, the electronic device executes the method as described in any one of the preceding or third aspects.
[0040] According to another embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. The computer program can be executed by a processor to implement the steps of the above-mentioned method.
[0041] According to another embodiment of the present invention, there is provided an electronic device, comprising: one or more processors; and a memory, wherein the memory is used to store executable instructions; the one or more processors are configured to implement the steps of the aforementioned method by executing the executable instructions.
[0042] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.
[0043] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.
[0044] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for training a voiceprint recognition model, wherein the voiceprint recognition model classifies device failures based on device operation voiceprints, characterized in that: The training method comprises: S1. Initializing the voiceprint recognition model, which includes a feature extraction network and a residual network, wherein the feature extraction network is used to extract features of an input voiceprint, and the residual network is used to calculate the residual features of the voiceprint to be recognized relative to the reference voiceprint; S2. Construct a training data set, wherein the training data set includes measured voiceprints of devices under different faults, the fault type corresponding to each measured voiceprint is used as its label, and the training data set also includes measured voiceprints of devices without faults as reference voiceprints; S3. The voiceprint recognition model is trained based on the constructed training data set. During the training process, a preset loss function is used to calculate the loss and update the model parameters.
2. A method for training a voiceprint recognition model as claimed in claim 1, characterized in that: The feature extraction network includes multiple convolutional layers, and each convolutional layer is connected to a batch normalization layer and an activation function layer.
3. The method for training a voiceprint recognition model according to claim 1, characterized in that: The residual feature of the voiceprint to be identified relative to the reference voiceprint is calculated as: S21, respectively obtaining the characteristics of the voiceprint to be identified and the reference voiceprint; S22. Calculate the difference between the characteristics of the voiceprint to be identified and the characteristics of the reference voiceprint.
4. The method for training a voiceprint recognition model according to claim 1, characterized in that: The loss is the cross entropy loss.
5. A method for training a voiceprint recognition model as claimed in claim 4, characterized in that: The cross entropy loss is: Among them, y represents the label failure type, Indicates the predicted failure type.
6. A voiceprint recognition model that classifies device failures based on device operation voiceprints, characterized in that: The model is configured to be trained using the method according to any one of claims 1 to 5, and comprises: Feature extraction network, used to extract features of input voiceprint; The residual network is used to calculate the residual features of the voiceprint to be identified relative to the reference voiceprint.
7. A voiceprint recognition method for classifying device failures based on device operation voiceprints, characterized in that: The method comprises: P1. Obtain device operation voiceprint and reference voiceprint; and P2. Use a voiceprint recognition model as described in claim 6 to classify the device operation voiceprint.
8. A computer program product, characterized in that The computer program product comprises a computer program code, and when the computer program code is run on an electronic device, the electronic device executes the method as claimed in any one of claims 1-5 and 7.
9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method in any one of claims 1-5 and 7.
10. An electronic device, characterized in that: include: one or more processors; as well as A memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1-5 and 7 by executing the executable instructions.