An industrial equipment fault diagnosis-oriented small sample voiceprint recognition method
Patent Information
- Application Number
- CN202611496675.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-24
- Publication Date
- 2026-10-30
AI Technical Summary
[0006]为了解决现有技术中存在的故障声纹样本少、跨设备和跨工况分布差异大，以及深浅层特征迁移效果不足的问题，本发明提出了一种面向工业设备故障诊断的小样本声纹识别方法，能够在目标域故障声纹样本较少的情况下，依次完成统一时频表征构建、浅层通用特征分布校正、多频带故障相关特征增强、深层跨域分布匹配以及联合训练分类输出，实现跨域故障声纹识别
[0041]本发明的有益效果：本发明通过构建由浅层通用特征对齐和深层领域自适应组成的跨域迁移链路，减小源域与目标域之间的分布偏移；通过连续小波变换构建统一时频表征，并结合多频带信号分解提取多尺度声纹特征，增强故障相关信息表达；通过多尺度时频坐标注意力模块对时间片段、频带特征和通道特征进行联合建模，在保留时频位置信息的同时强化故障相关区域的特征表达；通过分类损失和域差异损失联合训练，有利于改善小样本及复杂噪声条件下目标域工业设备声纹识别的泛化能力。
Smart Images
Figure CN123017085A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial equipment fault diagnosis and intelligent acoustic signal processing technology, and in particular to a small-sample voiceprint recognition method for industrial equipment fault diagnosis. Background Technology
[0002] Industrial equipment generates acoustic signals that reflect its operating status and fault characteristics during operation. Voiceprint recognition methods based on acoustic signals have application value in industrial equipment condition monitoring and fault diagnosis.
[0003] Existing industrial equipment voiceprint recognition methods typically rely on sufficient and well-labeled sample data to train models. However, in actual industrial settings, the number of fault samples is often limited, especially early faults, intermittent faults, and abnormal samples under specific operating conditions, leading to a significant small sample problem for voiceprint recognition models. Furthermore, differences between different equipment, operating conditions, and data acquisition environments can cause significant distribution shifts between the source and target data domains, making it difficult to directly apply a recognition model trained in one scenario to fault identification tasks under different equipment or operating conditions.
[0004] Furthermore, industrial environments are typically characterized by strong background noise, transient impacts, and aliasing, making it difficult for traditional single-time-domain or single-frequency-domain feature extraction methods to fully represent the key details in fault acoustic signatures. This results in insufficient robustness and generalization ability of the model in complex noise environments. While existing transfer learning methods can alleviate cross-domain recognition problems to some extent, most methods focus on single-distribution adaptation at the deep layers or ends of the network, making it difficult to simultaneously address the distribution correction of shallow general features and the cross-domain adaptation of deep discriminative features. For small-sample acoustic signature recognition tasks in industrial equipment, problems remain, including difficulty in effectively correcting shallow feature distribution biases, insufficient cross-domain adaptability of deep discriminative features, and inadequate utilization of multi-scale frequency band information.
[0005] Therefore, there is an urgent need to provide an industrial equipment voiceprint recognition method that can adapt to small samples, cross-device, cross-operating conditions and complex noise environments, so as to improve the stability and generalization ability of target domain voiceprint recognition. Summary of the Invention
[0006] To address the problems of limited fault acoustic signature samples, significant distribution differences across equipment and operating conditions, and insufficient deep and shallow feature transfer in existing technologies, this invention proposes a small-sample acoustic signature recognition method for industrial equipment fault diagnosis. This method, even with limited fault acoustic signature samples in the target domain, sequentially completes unified time-frequency representation construction, shallow general feature distribution correction, multi-frequency band fault-related feature enhancement, deep cross-domain distribution matching, and joint training classification output, achieving cross-domain fault acoustic signature recognition. The specific technical solution is as follows:
[0007] A small-sample voiceprint recognition method for fault diagnosis of industrial equipment includes:
[0008] Step 1: Collect source domain acoustic signature data and target domain acoustic signature data during the operation of industrial equipment and perform preprocessing with unified time-frequency representation to obtain source domain CWT time-frequency map and target domain CWT time-frequency map;
[0009] Step 2: Use a shared shallow feature extraction unit to extract shallow features from the source domain CWT time-frequency map and the target domain CWT time-frequency map, and use a general feature transfer module to transfer features to reduce the distribution offset between features and complete the shallow distribution alignment.
[0010] Step 3: The aligned source domain shallow features and target domain shallow features are sequentially processed by multi-band signal decomposition and multi-scale parallel feature extraction. Then, a multi-scale time-frequency coordinate attention module is used to enhance feature attention along the time dimension, frequency dimension and channel dimension to obtain the enhanced multi-scale voiceprint features of the source domain and target domain respectively.
[0011] Step 4: Using a domain-specific adaptive transfer module, deep feature extraction is performed on the enhanced source and target domain multi-scale voiceprint features respectively. Domain difference constraints are implemented by constructing a domain difference loss to achieve cross-domain distribution matching, thereby obtaining the domain-invariant voiceprint feature representation of the corresponding domain.
[0012] Step 5: Based on the classification loss and combined with the domain difference loss, guide the joint optimization of the voiceprint recognition model, and use the optimized model to output the target domain voiceprint recognition result.
[0013] Furthermore, step 1 specifically includes:
[0014] Step 1.1: Collect the raw acoustic signals of the source domain industrial equipment and the target domain industrial equipment during operation: source domain acoustic data and target domain acoustic data. The source domain acoustic data is an acoustic sample with category label collected under known equipment or known operating conditions. The target domain acoustic data is an acoustic sample collected under the equipment to be identified or the operating conditions to be identified.
[0015] Step 1.2: Process the raw acoustic signal into a unified input format so that the source domain acoustic data and the target domain acoustic data have a consistent sampling organization, sample length and batch structure, forming source domain preprocessed samples and target domain preprocessed samples.
[0016] Step 1.3: Perform continuous wavelet transform on the source domain preprocessed sample and the target domain preprocessed sample respectively using preset wavelet basis functions, so that the time variation information and frequency distribution information in the one-dimensional voiceprint signal are synchronously mapped to the two-dimensional time-frequency feature space, and the source domain CWT time-frequency map and the target domain CWT time-frequency map are obtained.
[0017] Step 1.4: Save the source domain CWT time-frequency map and the target domain CWT time-frequency map according to the sample order and batch order, so that each time-frequency map maintains a correspondence with its original voiceprint preprocessing sample.
[0018] Furthermore, the shared shallow feature extraction unit in step 2 includes a first shared convolutional layer and a second shared deep convolutional layer. The first shared convolutional layer is used to perform preliminary time-frequency response extraction on the source domain CWT time-frequency map and the target domain CWT time-frequency map. The second shared deep convolutional layer is used to further extract time-frequency local structural features or channel correlation features. The shallow feature extraction of the source domain and the shallow feature extraction of the target domain share the network parameters of the first shared convolutional layer and the second shared deep convolutional layer.
[0019] Furthermore, the general feature transfer module in step 2 is connected to the output of the shared shallow feature extraction unit and includes: a shallow feature center calculation unit and a center offset correction unit. The shallow feature center calculation unit calculates the shallow feature centers of the two domains respectively based on the shallow features of the source domain and the shallow features of the target domain. The center offset correction unit constructs a shallow alignment loss based on the offset between the shallow feature centers. By minimizing the shallow alignment loss, the offset between the shallow feature centers of the source domain and the target domain is reduced, and the shallow features are mapped and corrected accordingly to obtain the aligned shallow features of the source domain and the shallow features of the target domain.
[0020] Furthermore, step 3 specifically includes:
[0021] Step 3.1: Perform multi-band signal decomposition on the aligned source domain shallow features and target domain shallow features to obtain high-frequency sub-band features and low-frequency sub-band features, and obtain multi-band acoustic features based on convolution processing and feature reconstruction at different scales.
[0022] Step 3.2: Input the multi-band voiceprint features into the multi-scale parallel feature extraction branch to extract local time-frequency texture, medium-range time-frequency structure, wide-range frequency band correlation information and extended receptive field features respectively, and then fuse them;
[0023] Step 3.3: Input the fused features into the multi-scale time-frequency coordinate attention module, generate corresponding attention weights along the time dimension, frequency dimension and channel dimension respectively, and then fuse the time, frequency and channel attention weights and apply them to the fused features to obtain recalibrated features. Add the residuals of the recalibrated features to the multi-band voiceprint features to obtain enhanced multi-scale voiceprint features. The enhanced multi-scale voiceprint features retain the basic multi-band features, enhance the fault-related time segments, frequency regions and effective channel responses, and suppress noise-related features.
[0024] Furthermore, the multi-band signal decomposition in step 3.1 is implemented using a wavelet filter bank, which decomposes the input shallow features into high-frequency sub-band features and low-frequency sub-band features. The high-frequency sub-band features are transformed by channel dimension transformation using 1×1 convolution, and the low-frequency sub-band features are extracted by local time-frequency structure features using 5×5 convolution. Subsequently, the processed high-frequency and low-frequency sub-band features are reconstructed by inverse wavelet transform to obtain multi-band voiceprint features, and the dimensions of the multi-band voiceprint features are aligned.
[0025] Furthermore, the domain-specific adaptive transfer module includes a deep feature mapping unit and a domain difference measurement unit, and step 4 specifically includes:
[0026] Step 4.1: Input the enhanced multi-scale voiceprint features into the deep feature mapping unit to perform deep feature mapping, and obtain the source domain deep features and the target domain deep features respectively.
[0027] Step 4.2: Input the source domain deep features and the target domain deep features into the domain difference measurement unit, calculate the distribution difference between the two deep features through the domain difference measurement function, and construct the domain difference loss based on the distribution difference to perform domain difference constraint.
[0028] Step 4.3: By reducing the domain difference loss mentioned above, the deep features of the source domain and the deep features of the target domain are brought closer together in the deep feature space, thereby obtaining the domain-invariant voiceprint feature representation of the corresponding domain.
[0029] Furthermore, if the domain difference metric function in step 4.2 uses the maximum mean difference, then the domain difference loss is expressed as:
[0030]
[0031]
[0032]
[0033] in, Indicates the first Deep features of the source domain Indicates the first Deep features of the target domain; This represents the kernel mapping function that maps deep features to the reproducing kernel Hilbert space; Represents the regenerating kernel Hilbert space determined by the kernel mapping function; and These represent the mean embeddings of deep features in the source domain and deep features in the target domain in the reproducing kernel Hilbert space, respectively. and These represent the number of samples in the source domain and the number of samples in the target domain, respectively. Represents the regenerated nucleus Hilbert space Norms in; This represents the domain difference loss obtained by using the maximum mean difference. By minimizing this loss, the distribution difference of deep features in the source and target domains in the kernel mapping space is reduced.
[0034] Furthermore, step 5 specifically includes:
[0035] Step 5.1: Construct a classification loss based on the source domain sample labels in the source domain voiceprint data, and input the domain-invariant voiceprint features of the source domain into the classifier so that the classifier learns the correspondence between the source domain voiceprint data and the fault category.
[0036] Step 5.2: Weight the classification loss and the domain difference loss to form a joint optimization objective;
[0037] Step 5.3: Based on the joint optimization objective, train and update the parameters of the voiceprint recognition model, which consists of a shared shallow feature extraction unit, a general feature transfer module, a multi-scale time-frequency coordinate attention module, a domain-specific adaptive transfer module, and a classifier.
[0038] During training, the parameters are jointly updated by minimizing the joint optimization objective, so that the voiceprint recognition model can reduce the feature distribution shift between the source domain and the target domain while maintaining the ability to distinguish fault categories in the source domain.
[0039] Step 5.4: After training is completed, the target domain voiceprint data to be identified is input into the trained model, and the classifier outputs the target domain voiceprint recognition result.
[0040] Furthermore, the classifier includes a shared fully connected layer, where both the domain-invariant voiceprint features of the source domain and the domain-invariant voiceprint features of the target domain are input to the shared fully connected layer. The domain-invariant voiceprint features of the source domain are output as source domain predicted labels through the shared fully connected layer and used to construct a classification loss with the real source domain sample labels. The domain-invariant voiceprint features of the target domain are output as target domain voiceprint recognition results through the same shared fully connected layer.
[0041] The beneficial effects of this invention are as follows: This invention reduces the distribution offset between the source and target domains by constructing a cross-domain migration link composed of shallow general feature alignment and deep domain adaptation; it enhances the expression of fault-related information by constructing a unified time-frequency representation through continuous wavelet transform and extracting multi-scale acoustic signature features by combining multi-band signal decomposition; it strengthens the feature expression of fault-related regions by jointly modeling time segments, frequency band features, and channel features through a multi-scale time-frequency coordinate attention module, while preserving time-frequency location information; and it improves the generalization ability of acoustic signature recognition of industrial equipment in the target domain under small sample and complex noise conditions through joint training of classification loss and domain difference loss. Attached Figure Description
[0042] Figure 1 This is a schematic diagram of the overall process of a small-sample voiceprint recognition method for fault diagnosis of industrial equipment according to the present invention.
[0043] Figure 2 This is a schematic diagram of the two-stage transfer learning algorithm for small-sample voiceprint recognition of the present invention.
[0044] Figure 3 This is a schematic diagram of the structure of the multi-band decomposition and multi-scale time-frequency coordinate attention module in this invention.
[0045] Figure 4 This is a schematic diagram of the training and recognition phases of the method of the present invention. Detailed Implementation
[0046] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.
[0047] like Figure 1 As shown, this embodiment provides a small-sample voiceprint recognition method for fault diagnosis of industrial equipment, including the following steps:
[0048] Step 1, Cross-Domain Voiceprint Data Acquisition and Unified Time-Frequency Representation: This step sequentially completes voiceprint data acquisition, unified preprocessing, sample structuring, and continuous wavelet transform (CWT) to obtain standardized cross-domain time-frequency representations of the source and target domain voiceprint data, providing a consistent time-frequency input basis for subsequent cross-domain feature transfer. This step specifically includes:
[0049] Step 1.1: Conduct cross-domain acoustic data acquisition for industrial equipment fault diagnosis scenarios to obtain raw acoustic signals: source domain acoustic data and target domain acoustic data; the source domain acoustic data are acoustic samples collected under known equipment or known operating conditions and have category labels, and the target domain acoustic data are acoustic samples collected under the equipment or operating conditions to be identified.
[0050] Step 1.2: Process the source domain voiceprint data and target domain voiceprint data obtained in Step 1.1 into a unified input format to ensure that the source domain voiceprint data and target domain voiceprint data have the same sampling organization form, sample length and batch structure, so as to form source domain preprocessed samples and target domain preprocessed samples that meet the requirements of subsequent feature extraction.
[0051] Step 1.3: For the source domain preprocessed samples and target domain preprocessed samples obtained in Step 1.2, perform continuous wavelet transform using preset wavelet basis functions to synchronously map the time variation information and frequency distribution information in the one-dimensional voiceprint signal to the two-dimensional time-frequency feature space, that is, perform CWT time-frequency conversion to obtain the source domain CWT time-frequency map and the target domain CWT time-frequency map.
[0052] Specifically, the continuous wavelet transform is used to perform multi-scale time-frequency analysis on the input one-dimensional voiceprint signal: for the input voiceprint signal , in scale and time translation position The continuous wavelet transform coefficients can be expressed as follows:
[0053]
[0054] in, The time variable representing the voiceprint signal; This represents the scaling parameter, used to adjust the time width of the wavelet function and the corresponding frequency analysis range; This represents the time shift parameter, used to determine the temporal location of the wavelet function in the input speaker signal; This represents the predefined wavelet basis functions; Represents the complex conjugate of wavelet basis functions; Indicates the scale of the input voiceprint signal and time location The corresponding continuous wavelet transform coefficients at that point. Furthermore, for different scales... and different times and locations The wavelet transform coefficients obtained are arranged and a two-dimensional CWT time-frequency diagram is constructed based on their amplitude or energy, thereby characterizing the frequency distribution characteristics of the voiceprint signal as it changes over time.
[0055] Step 1.4: Organize and save the source domain CWT time-frequency map and target domain CWT time-frequency map obtained in Step 1.3 according to the corresponding input samples and batch order, so that they maintain the correspondence with the source domain preprocessed samples and the target domain preprocessed samples respectively, and use them as input to the shared shallow feature extraction unit.
[0056] Step 2, Shallow General Feature Extraction and Distribution Correction: Shallow feature extraction is performed on the source domain CWT time-frequency plot and the target domain CWT time-frequency plot. Feature transfer is then used to reduce the fundamental distribution offset between the extracted shallow features from the source domain and the shallow features from the target domain, achieving shallow distribution alignment. This step specifically includes:
[0057] Step 2.1: Input the source domain CWT time-frequency map and the target domain CWT time-frequency map after organization in step 1.4 into the shared shallow feature extraction unit, and obtain the source domain shallow features and the target domain shallow features through the shared shallow feature extraction unit.
[0058] The shared shallow feature extraction unit includes a first shared convolutional layer and a second shared depth convolutional layer. The first shared convolutional layer is used to perform preliminary time-frequency response extraction on the source domain CWT time-frequency map and the target domain CWT time-frequency map. The second shared depth convolutional layer is used to further extract time-frequency local structural features or channel correlation features. The source domain branch and the target domain branch share the network parameters of the first shared convolutional layer and the second shared depth convolutional layer.
[0059] Step 2.2: A general feature transfer module is set at the output of the shared shallow feature extraction unit. The general feature transfer module includes a shallow feature center calculation unit and a center offset correction unit. The shallow feature center calculation unit calculates the shallow feature centers of the source and target domains respectively based on the shallow features of the source and target domains. The center offset correction unit constructs a shallow alignment loss based on the offset between the shallow feature centers, providing constraints for center offset correction to achieve shallow distribution alignment. Specifically, this is expressed as follows:
[0060]
[0061]
[0062]
[0063]
[0064] in, and These represent the shallow feature centers of the source domain and the shallow feature centers of the target domain, respectively. Indicates the first Shallow features of the source domain Indicates the first Shallow features of the target domain; and These represent the number of samples in the source domain and the number of samples in the target domain, respectively. This represents the offset between the shallow feature centers of two domains; This represents the shallow alignment loss, used to measure the difference between the shallow feature centers of the source and target domains.
[0065] By minimizing the shallow alignment loss, the offset between the shallow feature centers of the source domain and the target domain is reduced, and the shallow features are mapped and corrected accordingly to obtain the aligned shallow features of the source domain and the shallow features of the target domain.
[0066] In one implementation, the general feature transfer module selects the calculation method for the feature center based on the representation of the shallow features: for vector features in Euclidean space, the Euclidean mean is used; for shallow features represented by covariance matrices or manifold structures, the Riemann mean, Log-Det mean, or Log-Euclidean mean is used. The feature center is used to align the shallow centers of the source and target domains in the general feature transfer module.
[0067] Step 2.3: By reducing the shallow alignment loss mentioned above, the aligned source domain shallow features and the aligned target domain shallow features are obtained, and the aligned source domain shallow features and the target domain shallow features are used as the input features for step 3.
[0068] Step 3, Multiband Decomposition and Fault-Related Feature Enhancement: Reference Figure 2 and Figure 3 As shown, the aligned shallow features of the source domain and the shallow features of the target domain are extracted from fault-related acoustic signature information in different frequency bands through multi-band signal decomposition. Then, combined with multi-scale parallel feature extraction and a multi-scale time-frequency coordinate attention module, joint attention modeling is performed along the time, frequency, and channel dimensions to adaptively weight and fuse fault-related time-frequency regions, frequency band features, and channel features, resulting in enhanced multi-scale acoustic signature features for both the source and target domains. This step specifically includes:
[0069] Step 3.1: Perform multi-band signal decomposition on the aligned source domain shallow features and target domain shallow features output from Step 2.3 to obtain high-frequency sub-band features and low-frequency sub-band features, and obtain multi-band acoustic features based on convolution processing and feature reconstruction at different scales.
[0070] Specifically, the multi-band signal decomposition is implemented using a wavelet filter bank, decomposing the input shallow features into high-frequency sub-band features and low-frequency sub-band features. The high-frequency sub-band features include HH, HL, and LH sub-bands, while the low-frequency sub-band features include LL sub-bands. The high-frequency sub-band features are transformed using a 1×1 convolution, and the low-frequency sub-band features are extracted using a 5×5 convolution. Subsequently, the processed high-frequency and low-frequency sub-band features are reconstructed using inverse wavelet transform to obtain multi-band voiceprint features. The dimensions of these multi-band voiceprint features are then aligned to meet the input requirements of subsequent multi-scale parallel feature extraction and residual bypass processing.
[0071] Step 3.2: Input the multi-band voiceprint features into the multi-scale parallel feature extraction branch to extract local time-frequency texture, medium-range time-frequency structure, wide-range frequency band correlation information and extended receptive field features, and then fuse them.
[0072] The multi-scale parallel feature extraction branch includes a first-scale convolution branch, a second-scale convolution branch, a third-scale convolution branch, and a dilated convolution branch. The first-scale convolution branch is used to extract local time-frequency texture, the second-scale convolution branch is used to extract time-frequency structure over a medium range, the third-scale convolution branch is used to extract frequency band correlation information over a wide range, and the dilated convolution branch is used to expand the receptive field and capture fault-related responses over a longer time span or a wider frequency range. The features output by each scale branch are fed into a multi-scale fusion unit to form fused multi-scale voiceprint features, i.e., fused features.
[0073] In one embodiment, the fused features are further subjected to nonlinear transformation and dimension alignment to meet the requirements of the input multi-scale time-frequency coordinate attention module in terms of time dimension, frequency dimension, and number of channels, and are used for subsequent joint weight fusion and feature recalibration.
[0074] Step 3.3: Input the fused features into the multi-scale time-frequency coordinate attention module, generate corresponding attention weights along the time dimension, frequency dimension and channel dimension respectively, and then fuse the time, frequency and channel attention weights and apply them to the fused features to obtain recalibrated features. Add the residuals of the reconstructed or dimensionally aligned multi-band voiceprint features to obtain the enhanced multi-scale voiceprint features.
[0075] In one implementation, the fusion feature is denoted as The fused features are aggregated along the frequency and time dimensions respectively to obtain time coordinate features and frequency coordinate features, and their calculation relationship is as follows:
[0076]
[0077]
[0078] in, Indicates the fusion feature in the th Each channel, time location and frequency position Eigenvalues at; Indicates the channel index; Indicates the time location index; Indicates the frequency position index; and These represent the lengths of the fused features in the time and frequency dimensions, respectively. This represents summing the features at each frequency position along the frequency dimension. This represents the summation of features at each time location along the time dimension; and These represent the averaging process on the corresponding dimensions; Indicates the first Each channel in time position The time coordinate features obtained by aggregating along the frequency direction; Indicates the first Each channel at frequency position The frequency coordinate features are obtained by aggregating along the time direction. These time coordinate features and frequency coordinate features are used to generate the time attention weights and frequency attention weights, respectively.
[0079] Temporal attention weights and frequency attention weights are generated based on temporal and frequency coordinate features, and channel attention weights are generated based on fusion features. The calculation relationships are as follows:
[0080]
[0081]
[0082]
[0083] in, Indicates the first Each channel in time position Time attention weighting; Indicates the first Each channel at frequency position Frequency of attention weights; Indicates the first Channel attention weights for each channel; Indicates by the first Each channel in time position The time coordinate features obtained by aggregating along the frequency direction; Indicates by the first Each channel at frequency position Frequency coordinate features obtained by aggregation along the time direction; Indicates the fusion characteristics of each channel The overall fusion characteristics of the composition; This indicates pooling operations performed along the time and frequency dimensions. , and These represent the weight generation mappings in the time direction, frequency direction, and channel direction, respectively; This represents a non-linear activation function.
[0084] The three types of attention weights mentioned above are jointly weighted, and the weighted result is added to the multi-band voiceprint features:
[0085]
[0086]
[0087]
[0088] in, This indicates that the joint attention weight, constructed based on the aforementioned time attention weight, frequency attention weight, and channel attention weight, is used to comprehensively reflect the importance of the corresponding time-frequency position and feature channel; The recalibrated features are obtained by applying the joint attention weights to the fused features. This represents the multi-band voiceprint features after reconstruction or dimension alignment; This represents the final enhanced multi-scale voiceprint features.
[0089] Through the above-mentioned joint weight calculation, feature recalibration and residual summation process, the fault-related time segment, frequency region and effective channel response are enhanced while retaining the basic characteristics of the multi-band system, and noise-related features are suppressed.
[0090] Step 3.4: Output the enhanced multi-scale voiceprint features obtained in Step 3.3 according to the source domain branch and the target domain branch respectively, and use them as input for the deep domain adaptive transfer in Step 4.
[0091] Step 4, Deep Cross-Domain Distribution Matching: The enhanced multi-scale voiceprint features of the source and target domains are input into a domain-specific adaptive transfer module. This module includes a deep feature mapping unit and a domain difference measurement unit. The deep feature mapping unit obtains deep features of the source and target domains respectively, and the domain difference measurement unit applies domain difference constraints to these features to achieve cross-domain distribution matching and obtain a domain-invariant voiceprint feature representation. This step specifically includes:
[0092] Step 4.1: Input the enhanced multi-scale voiceprint features output in Step 3.4 into the deep feature mapping unit. The deep feature mapping unit performs deep feature mapping on the input features based on the deep mapping parameters of the shared shallow feature extraction unit to obtain the source domain deep features and the target domain deep features respectively.
[0093] Step 4.2: Input the source domain deep features and the target domain deep features into the domain difference measurement unit, calculate the distribution difference between the two deep features through the domain difference measurement function, and perform domain difference constraint by constructing a domain difference loss based on the distribution difference.
[0094] The domain difference loss can be expressed by the following formula:
[0095]
[0096] in, Indicates domain difference loss, Represents deep features of the source domain. Represents deep features of the target domain. This represents the domain difference metric function used to measure the difference between deep features in the source domain and deep features in the target domain. Cross-domain distribution matching constraints can be obtained using the above formula.
[0097] In one implementation, the domain difference measure function One of the following methods can be used: adversarial training, maximum mean difference, relevance alignment, or learned maximum mean difference (MMD). The following explanation uses maximum mean difference as an example, which measures the distributional difference between deep features in the source domain and deep features in the target domain. The domain difference loss is expressed as:
[0098]
[0099]
[0100]
[0101] in, Indicates the first Deep features of the source domain Indicates the first Deep features of the target domain; This represents the kernel mapping function that maps deep features to the reproducing kernel Hilbert space; This represents the regenerating kernel Hilbert space determined by the kernel mapping function. and These represent the mean embeddings of deep features in the source domain and deep features in the target domain in the reproducing kernel Hilbert space, respectively. and These represent the number of samples in the source domain and the number of samples in the target domain, respectively. Represents the regenerated nucleus Hilbert space Norms in; This represents the domain difference loss obtained by using the maximum mean difference. By minimizing this loss, the distribution difference of deep features in the source and target domains in the kernel mapping space is reduced.
[0102] Step 4.3: By reducing the domain difference loss mentioned above, the deep features of the source domain and the deep features of the target domain are brought closer together in the deep feature space, reducing the cross-domain distribution shift in the deep discriminative features, thereby further transforming the enhanced multi-scale voiceprint features output in Step 3 into domain-invariant voiceprint feature representations that can be used for joint training and classification recognition.
[0103] Step 5, Joint training of the model and classification output: such as Figure 4 As shown, a joint training and recognition output mechanism is constructed based on classification loss and combined with the domain difference loss, and the voiceprint recognition model is jointly optimized. The optimized model is then used to output the target domain voiceprint recognition result. This step specifically includes:
[0104] Step 5.1: Construct a classification loss based on the source domain sample labels, and input the domain-invariant voiceprint features of the source domain into the classifier so that the classifier learns the correspondence between the source domain voiceprint data and the fault category.
[0105] The classification loss can be expressed by the following formula:
[0106]
[0107] Where K represents the number of fault categories, Indicates the number of samples in the source domain. This indicates that the i-th source domain sample belongs to the k-th class. When the i-th source domain sample belongs to the k-th class... The value is 1 if it is set to 1, otherwise the value is 0. This represents the predicted probability that the i-th source domain sample belongs to the k-th class. The classification loss described above constrains the classifier to learn the correspondence between source domain voiceprint features and fault categories.
[0108] The classifier includes a shared fully connected layer. Both the domain-invariant voiceprint features of the source domain and the domain-invariant voiceprint features of the target domain are input into the shared fully connected layer. The domain-invariant voiceprint features of the source domain are output as source domain predicted labels through the shared fully connected layer and used to construct a classification loss with the real source domain sample labels. The domain-invariant voiceprint features of the target domain are output as target domain voiceprint recognition results through the same shared fully connected layer.
[0109] Step 5.2: Combine the classification loss and the domain difference loss in a weighted manner to form a joint optimization objective.
[0110] The joint optimization objective can be expressed by the following formula:
[0111]
[0112] in, This represents the classification loss constructed based on labeled samples from the source domain. Indicates domain difference loss, The weighting coefficients representing the domain difference loss are used to adjust the relative contributions of classification discrimination constraints and cross-domain distribution matching constraints to the joint optimization objective.
[0113] Step 5.3: Based on the joint optimization objective, train and update the parameters of the shared shallow feature extraction unit, the general feature transfer module, the multi-scale time-frequency coordinate attention module, the domain-specific adaptive transfer module, and the classifier.
[0114] The set of model parameters participating in the joint optimization and its optimization objective can be expressed as:
[0115] , ;
[0116] in, The parameters representing the shared shallow feature extraction units. This represents the parameters of the general feature transfer module. The parameters represent the multi-scale time-frequency coordinate attention module. These represent parameters for a domain-specific adaptive migration module. Indicates the parameters of the classifier; This represents the overall parameter set composed of the parameters from the above modules. This indicates that the joint optimization objective The optimal model parameters that achieve the minimum value. Indicates in the parameter set Searching for the objective function The smallest parameter.
[0117] During training, the joint optimization objective is minimized. By jointly updating the above parameters, the model can maintain its ability to distinguish fault categories in the source domain while reducing the feature distribution shift between the source and target domains.
[0118] Step 5.4: After training, the target domain voiceprint data to be identified is input into the trained model in the order of steps 1 to 4, and the classifier outputs the target domain voiceprint recognition result. Thus, even with a small number of target domain samples, the model can still utilize source domain supervision information and cross-domain distribution matching constraints to complete target domain fault identification.
[0119] In step 5.4, the classifier determines the final identification category based on the predicted probability that the target domain sample belongs to each fault category. The target domain voiceprint recognition result can be expressed by the following formula:
[0120]
[0121] in, Indicates the first The fault category ultimately predicted from each target domain sample; The output of the trained classifier is the first... The target domain sample belongs to the first Predicted probability of type of failure; Indicates the fault category index, Indicates the total number of fault categories; Indicates the index of the sample to be identified in the target domain; This indicates that the category with the highest predicted probability is selected from all fault categories. The final fault category corresponding to the target domain sample is determined through the above maximum probability decision.
[0122] Through the above steps, the present invention can reduce the distribution offset between the source domain and the target domain when there are few fault acoustic fingerprint samples in the target domain, and improve the generalization ability of industrial equipment fault acoustic fingerprint recognition under cross-equipment and cross-operating condition conditions.
[0123] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A small-sample voiceprint recognition method for fault diagnosis of industrial equipment, characterized in that, include: Step 1: Collect source domain acoustic signature data and target domain acoustic signature data during the operation of industrial equipment and perform preprocessing with unified time-frequency representation to obtain source domain CWT time-frequency map and target domain CWT time-frequency map; Step 2: Use a shared shallow feature extraction unit to extract shallow features from the source domain CWT time-frequency map and the target domain CWT time-frequency map, and use a general feature transfer module to transfer features to reduce the distribution offset between features and complete the shallow distribution alignment. Step 3: The aligned source domain shallow features and target domain shallow features are sequentially processed by multi-band signal decomposition and multi-scale parallel feature extraction. Then, a multi-scale time-frequency coordinate attention module is used to enhance feature attention along the time dimension, frequency dimension and channel dimension to obtain the enhanced multi-scale voiceprint features of the source domain and target domain respectively. Step 4: Using a domain-specific adaptive transfer module, deep feature extraction is performed on the enhanced source and target domain multi-scale voiceprint features respectively. Domain difference constraints are implemented by constructing a domain difference loss to achieve cross-domain distribution matching, thereby obtaining the domain-invariant voiceprint feature representation of the corresponding domain. Step 5: Based on the classification loss and combined with the domain difference loss, guide the joint optimization of the voiceprint recognition model, and use the optimized model to output the target domain voiceprint recognition result.
2. The method as described in claim 1, characterized in that, Step 1 specifically includes: Step 1.1: Collect the raw acoustic signals of the source domain industrial equipment and the target domain industrial equipment during operation: source domain acoustic data and target domain acoustic data. The source domain acoustic data is an acoustic sample with category label collected under known equipment or known operating conditions. The target domain acoustic data is an acoustic sample collected under the equipment to be identified or the operating conditions to be identified. Step 1.2: Perform unified input format processing on the original acoustic signal to ensure that the source domain acoustic data and the target domain acoustic data have consistent sampling organization, sample length and batch structure, forming source domain preprocessed samples and target domain preprocessed samples. Step 1.3: Perform continuous wavelet transform on the source domain preprocessed sample and the target domain preprocessed sample respectively using preset wavelet basis functions, so that the time variation information and frequency distribution information in the one-dimensional voiceprint signal are synchronously mapped to the two-dimensional time-frequency feature space, and the source domain CWT time-frequency map and the target domain CWT time-frequency map are obtained. Step 1.4: Save the source domain CWT time-frequency map and the target domain CWT time-frequency map according to the sample order and batch order, so that each time-frequency map maintains a correspondence with its original voiceprint preprocessing sample.
3. The method as described in claim 1, characterized in that, The shared shallow feature extraction unit in step 2 includes a first shared convolutional layer and a second shared deep convolutional layer. The first shared convolutional layer is used to perform preliminary time-frequency response extraction on the source domain CWT time-frequency map and the target domain CWT time-frequency map. The second shared deep convolutional layer is used to further extract time-frequency local structural features or channel correlation features. The shallow feature extraction of the source domain and the shallow feature extraction of the target domain share the network parameters of the first shared convolutional layer and the second shared deep convolutional layer.
4. The method as described in claim 3, characterized in that, The general feature transfer module in step 2 is connected to the output of the shared shallow feature extraction unit and includes a shallow feature center calculation unit and a center offset correction unit. The shallow feature center calculation unit calculates the shallow feature centers of the two domains based on the shallow features of the source domain and the shallow features of the target domain, respectively. The center offset correction unit constructs a shallow alignment loss based on the offset between the shallow feature centers. By minimizing the shallow alignment loss, the offset between the shallow feature centers of the source domain and the target domain is reduced, and the shallow features are mapped and corrected accordingly to obtain the aligned shallow features of the source domain and the shallow features of the target domain.
5. The method as described in claim 1, characterized in that, Step 3 specifically includes: Step 3.1: Perform multi-band signal decomposition on the aligned source domain shallow features and target domain shallow features to obtain high-frequency sub-band features and low-frequency sub-band features, and obtain multi-band acoustic features based on convolution processing and feature reconstruction at different scales. Step 3.2: Input the multi-band voiceprint features into the multi-scale parallel feature extraction branch to extract local time-frequency texture, medium-range time-frequency structure, wide-range frequency band correlation information and extended receptive field features respectively, and then fuse them; Step 3.3: Input the fused features into the multi-scale time-frequency coordinate attention module, generate corresponding attention weights along the time dimension, frequency dimension and channel dimension respectively, and then fuse the time, frequency and channel attention weights and apply them to the fused features to obtain recalibrated features. Add the residuals of the recalibrated features to the multi-band acoustic features to obtain enhanced multi-scale acoustic features. The enhanced multi-scale acoustic features retain the basic multi-band features, enhance the fault-related time segments, frequency regions and effective channel responses, and suppress noise-related features.
6. The method as described in claim 5, characterized in that, The multi-band signal decomposition in step 3.1 is implemented using a wavelet filter bank, which decomposes the input shallow features into high-frequency sub-band features and low-frequency sub-band features. The high-frequency sub-band features are transformed by channel dimension transformation using 1×1 convolution, and the low-frequency sub-band features are extracted by local time-frequency structure feature extraction using 5×5 convolution. Subsequently, the processed high-frequency and low-frequency sub-band features are reconstructed by inverse wavelet transform to obtain multi-band voiceprint features, and the dimensions of the multi-band voiceprint features are aligned.
7. The method as described in claim 1, characterized in that, The domain-specific adaptive transfer module includes a deep feature mapping unit and a domain difference measurement unit. Step 4 specifically includes: Step 4.1: Input the enhanced multi-scale voiceprint features into the deep feature mapping unit to perform deep feature mapping, and obtain the source domain deep features and the target domain deep features respectively. Step 4.2: Input the source domain deep features and the target domain deep features into the domain difference measurement unit, calculate the distribution difference between the two deep features through the domain difference measurement function, and construct the domain difference loss based on the distribution difference to perform domain difference constraint. Step 4.3: By reducing the domain difference loss mentioned above, the deep features of the source domain and the deep features of the target domain are brought closer together in the deep feature space, thereby obtaining the domain-invariant voiceprint feature representation of the corresponding domain.
8. The method as described in claim 7, characterized in that, The domain difference metric function in step 4.2 uses the maximum mean difference, and the domain difference loss is expressed as: ， ， ， in, Indicates the first Deep features of the source domain Indicates the first Deep features of the target domain; This represents the kernel mapping function that maps deep features to the reproducing kernel Hilbert space; Represents the regenerating kernel Hilbert space determined by the kernel mapping function; and These represent the mean embeddings of deep features in the source domain and deep features in the target domain in the reproducing kernel Hilbert space, respectively. and These represent the number of samples in the source domain and the number of samples in the target domain, respectively. Represents the regenerated nucleus Hilbert space Norms in; This represents the domain difference loss obtained by using the maximum mean difference. By minimizing this loss, the distribution difference of deep features in the source and target domains in the kernel mapping space is reduced.
9. The method as described in claim 1, characterized in that, Step 5 specifically includes: Step 5.1: Construct a classification loss based on the source domain sample labels in the source domain voiceprint data, and input the domain-invariant voiceprint features of the source domain into the classifier so that the classifier learns the correspondence between the source domain voiceprint data and the fault category. Step 5.2: Weight the classification loss and the domain difference loss to form a joint optimization objective; Step 5.3: Based on the joint optimization objective, train and update the parameters of the voiceprint recognition model, which consists of a shared shallow feature extraction unit, a general feature transfer module, a multi-scale time-frequency coordinate attention module, a domain-specific adaptive transfer module, and a classifier. During training, the parameters are jointly updated by minimizing the joint optimization objective, so that the voiceprint recognition model can reduce the feature distribution shift between the source domain and the target domain while maintaining the ability to distinguish fault categories in the source domain. Step 5.4: After training is completed, the target domain voiceprint data to be identified is input into the trained model, and the classifier outputs the target domain voiceprint recognition result.
10. The method as described in claim 9, characterized in that, The classifier includes a shared fully connected layer. Both the domain-invariant voiceprint features of the source domain and the domain-invariant voiceprint features of the target domain are input into the shared fully connected layer. The domain-invariant voiceprint features of the source domain are output as source domain predicted labels through the shared fully connected layer and used to construct a classification loss with the real source domain sample labels. The domain-invariant voiceprint features of the target domain are output as target domain voiceprint recognition results through the same shared fully connected layer.